How can you set SEOmoz to work with your dev site behind an htpasswd?
-
All sites need to be developed from the small to the grand - and this takes time. Development usually takes place on a subdomain different from our live domain. It is locked down behind an htpasswd during development so its not picked up by searching engines - that may create duplicate content issues if when the site goes live it has already scanned our site on the development server. Its also a security implementation to keep the site away from prying eyes before its ready for launch There could be security holes that have not been tweaked.
Whats the best strategy to get SEOmoz involved in this scenario. Its tools are invaluable to the SEO part of the build - but the seomoz crawler bot has a different IP address (being cloud based) - so we cannot just let a single IP address through our htpasswd.
Also is there a way to link the dev and live site in seomoz - so when it goes live to maintain all teh same logs without having to create two seperate site campaigns?
Thanks!
-
Hey Robert!
Yes, everything is good here, thanks. Just been hectic around here lately and yes, I've been feeling bad about missing my time in Q&A
-
Good answer there Sha. Glad to see your smiling face, thought you were hiding! Hope all is well down under.
-
Hi dseo241,
Given that your question specifically relates to Roger's ability to access your sites (both dev & production), I think you would be best to email the SEOmoz Help Team direct - help at seomoz.org.
I use the stand alone Crawl Test tool to run a base crawl of our dev sites, which are noindexed, but not behind an htpassword, so you would still need to ask the Help Team about accessibility. Incidentally, the SEOmoz User-agent is rogerbot.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Site Crawler not working but on-demand crawler working
Hi, In Moz pro, when using Site crawler (or recrawl), we are seeing message site is banned. But when using on-demand crawler, it could generate report successfully. I just like to know if in both these cases, it is roberbot that is used! And kindly note, site crawler was perfectly working before. So the required setup is already in place from long time. Site crawler ban issue started appearing from nov/dec 2023. . Could you please us understand how could we possibly make site-crawler work?
Moz Pro | | gilesd
I am happy to provide more details if you need any. Thanks0 -
What value am I getting from SEOmoz?
Okay, I've been using the trial version of SEOmoz for almost a week, but, I'm just not sure what to do with it to be honest. I'm not an SEO expert, so, a lot of the terminology and reports are confusing. So far the only thing I've found useful are the domain and page duplication errors, but, I can get the same info from Google Webmaster Tools. Am I missing something? We have a very simple site for a small business, not sure I can justify $99 per month for a service I don't really understand. Is this service more for SEO professionals than business owners? -Tom
Moz Pro | | TomHu0 -
How to find the authorities site to link from
Hi It seems getting quality authority backlinks is the main priority to success in your SEO which is also most difficult task as well . How do you find these authority sites that allows you to backlink from them? is there any tools to find good quality sites for backlinks? Thanks
Moz Pro | | Matt-kind0 -
Can I rely on Keyword Difficulty tool?
I just ran into a problem that I hadn't expected. Testing the Keyword Difficulty I saw the results contained a result for a page that has Domain Authority=1 and Page Authority=1. As a result, Keyword Difficulty was reduced (compared to last month), which may actually be reversed if the site is crawled. Sadly, I didn't run the report on the figures as it was a small project. Questions: Can I rely on results shown by Keyword Difficulty? Are results where Domain Authority =1 are used to calculate Keyword Difficulty? If so, why is that? Is there any difference between a page that has received no links and a page that OSE/Mozscape has no link data for? The problem Using the Keyword Difficulty tool, I found swings of up to 14% in Keyword Difficulty (between Oct -Nov). Dr Pete may suggest that this is because of changes in Google's index ( http://www.seomoz.org/blog/a-week-in-the-life-of-3-keywords ). However, It would be helpful to have a figure for Keyword Difficulty that isn't affected by the gaps in the Mozscape data. The (bad) solution You can mirror something close to Keyword Difficulty using: =(Sum of Page Authorities + Sum of Domain Authorities )/20 Right now, I have resorted to manually calculating keyword difficulty. I use the SEOMoz Page Authority & Domain Authority figures and a quick splash of Excel SUMIF and COUNTIF. I find the results don't look as 'easy' when I can ignore results where the data is unknown (PageAuthority=1 & DomainAuthority = 1). Background Info One result I still have a report on is for the phrase [fixing your business puzzle] using US Results on Google. For the specific result, I found the additional information about the site: DNS lookup shows the domain was registered in 2010
Moz Pro | | Darroch
Archive.org shows no records
OSE shows no data for the site
Site uses https
Google showing No links
Robots.txt file seems fine
No Sitemap.xml0 -
Can open site explorer miss incoming links?
In open site explorer I only see one linking domain, eventhough I know of at least 1 other. Why is that?
Moz Pro | | ResourceLab0 -
I cant work it out!!
Hi Guys,If you can please help. I am trying to use OSE for the site www matesratesuk.com it tells me there is 177 link and it is showing 1-26 of these but I cannot seem to locate the others. Am I possibly doing something wrong oris it because all these links are internally linked. Im not sure. Thanks for Looking, Craig
Moz Pro | | fenwaymedia0 -
Adjusting SEOmoz Crawling Speed
How do you adjust the SEOmoz crawling speed? SEOmoz tried to crawl 10,000 pages in 3 hours and crashed our MySQL server.
Moz Pro | | cappuccino891 -
For SEOMoz Staff - Can You Make a Better Referral Badge, PLEASE?
I am going to send referrals to SEOMOZ and I want to put a badge on my site with my referral link from this page: http://www.seomoz.org/users/referrals Can you make some banners where we can just select an image src with a call to action like "FREE SEO Tools" and "Free 30 Day Trail"? Currently the only badges I see say something wack like "I Love SEOMOZ" http://www.seomoz.org/dp/badges I need something that converts higher than "I love SEOmoz". Love yah guys!
Moz Pro | | Francisco_Meza0