How can you set SEOmoz to work with your dev site behind an htpasswd?
-
All sites need to be developed from the small to the grand - and this takes time. Development usually takes place on a subdomain different from our live domain. It is locked down behind an htpasswd during development so its not picked up by searching engines - that may create duplicate content issues if when the site goes live it has already scanned our site on the development server. Its also a security implementation to keep the site away from prying eyes before its ready for launch There could be security holes that have not been tweaked.
Whats the best strategy to get SEOmoz involved in this scenario. Its tools are invaluable to the SEO part of the build - but the seomoz crawler bot has a different IP address (being cloud based) - so we cannot just let a single IP address through our htpasswd.
Also is there a way to link the dev and live site in seomoz - so when it goes live to maintain all teh same logs without having to create two seperate site campaigns?
Thanks!
-
Hey Robert!
Yes, everything is good here, thanks. Just been hectic around here lately and yes, I've been feeling bad about missing my time in Q&A
-
Good answer there Sha. Glad to see your smiling face, thought you were hiding! Hope all is well down under.
-
Hi dseo241,
Given that your question specifically relates to Roger's ability to access your sites (both dev & production), I think you would be best to email the SEOmoz Help Team direct - help at seomoz.org.
I use the stand alone Crawl Test tool to run a base crawl of our dev sites, which are noindexed, but not behind an htpassword, so you would still need to ask the Help Team about accessibility. Incidentally, the SEOmoz User-agent is rogerbot.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Export to csv does not work
HI friends. For long time I cannot download csv reports for links. I need to export new spam links sort by spam score and compare to existing but export to csv button spins for ever and nothing happens
Moz Pro | | netcomsia0 -
Why did Moz crawl our development site?
In our Moz Pro account we have one campaign set up to track our main domain. This week Moz threw up around 400 new crawl errors, 99% of which were meta noindex issues. What happened was that somehow Moz found the development/staging site and decided to crawl that. I have no idea how it was able to do this - the robots.txt is set to disallow all and there is password protection on the site. It looks like Moz ignored the robots.txt, but I still don't have any idea how it was able to do a crawl - it should have received a 401 Forbidden and not gone any further. How do I a) clean this up without going through and manually ignoring each issue, and b) stop this from happening again? Thanks!
Moz Pro | | MultiTimeMachine0 -
I know our business listed in Yahoo and medranks.com (for example). But my open site explorer report doesn't show those. however on their sites, I see the listing. Why is this?
I know our business listed in Yahoo and medranks.com (for example). But my open site explorer report doesn't show those links on the inbound report. however on their respective sites, I see the listing when I search for us. And the link does work..... Why is this? Why don't I see it on the open site report?
Moz Pro | | cschwartzel0 -
Big changes in site titles
So as I pour though some of the diagnostics data for over 100,000 pages of my site I see thousands of page title that "could" be changed. Could this cause some lost traffic for a while due to the big changes?
Moz Pro | | dvduval0 -
Page Authority is the same on every page of my site
I'm analyzing a site and the page authority is the exact same for every page in the site. How can this be since the page authority is supposed to be unique to each page?
Moz Pro | | azjayhawk0 -
How can competition outrank you if your site has better Domain/Page Authority, More links, and More Social sharing?
Say you have a site that has better Domain/page authority, more links, more social media sharing, and a lot more indexed pages (thanks to blogging) than the competition. Of course all of these metrics are based off of data from SEOMoz open site explorer tool which I am not sure if it produces accurate data. 1. Other than exact match domains or the age of a domain what would be other reasons why competition would outrank you? 2. Can anyone suggest other ways to help increase a sites domain/page authority besides creating more indexed pages, link building, etc..?
Moz Pro | | webestate0 -
How can I reduce the number of links on a page and keep the site easy to navigate?
The SEOmoz Site Crawl indicates that we have too many on page links on over 9,970 pages. This is an ecommerce site with a large number of categories. I have a couple of questions regarding this issue: How important is the "too many on page links" factor to SEO? What are some methods of reducing the number of links when there are a large number of categories? We have main categories with dropdown menus currently and have found that they are used to browse and shop the store.
Moz Pro | | afmaury1