Can I crawl a password protected domain with SEOmoz?
-
Hi everyone,
Just wondered if anybody has been able to use the SEOmoz site crawler for password protected domains? On Screaming Frog you are prompted for the username and password when you set the crawler running, however SEOmoz doesn't. It seems you can only crawl sites that are live and publicly available - can anyone confirm if this is the case?
Cheers,
M
-
Hi,
I don't think it is possible to crawl any password protectec pages. Password protected pages are usually blocked and bots are unable access those pages.
And yes it is true that only sites that are publicly and live can be crawled.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
'domain:example.com/' is this line with a '/' at the end of the domain valid in a disavow report file ?
Hi everyone Just out of curiosity, what would happen if in my disavow report I have this line : domain:example.com**/** instead of domain:example.com as recommended by google. I was just wondering if adding a / at the end of a domain would automatically render the line invalid and ignored by Google's disavow backlinks tool. Many thanks for your thoughts
Technical SEO | | LabeliumUSA0 -
Where can I ... ?
[Spammy content removed by moderator.] Abdullah-Al-Asif-1.jpg?fit=177%2C177&ssl=1 tEjk20Fs_400x400.jpg
Technical SEO | | Abdullah-Al-Asif0 -
404 crawl errors ending with your domain name??
Hello, I have a crawl test with numerous 404 errors ending with my domain name..? Not sure what the cause is. Plugins? Ecommerce? I use Wordpress if that could lead to an answer. Thanks for your time. K
Technical SEO | | Hydraulicgirl0 -
Merging domains was probably a mistake. Can it be undone?
Hello good people, Website A was doing great in terms of traffic and ranking.
Technical SEO | | jansievers
Website B had seen a steady decline in traffic and visibility for several years, most likely due to thin content issues. Website B was merged into website A. Content was redirected on a per page basis with a rewrite rule that takes domain name and the change in URL structure into account. After merging the two websites, traffic and rankings of Website A started dropping. What would be the best course of action here? Weed out thin content and stick with the merger? Removing all of Website B's content and the redirects to Website A? ?? Thanks! Jan 1pYROKn1 -
Keyword domain usage
Hi I have a client who wants some advice on the following and I haven't come up with anything decent yet. So wanted to throw it out here. They own a domain like soccershoes.com which also houses their web site. They have now bought ussoccershoes.com because this US based search term is more popular than simply soccer shoes. How can they utilise this new domain effectively to drive traffic to their main web site? Thanks for any input.
Technical SEO | | yolkcreative0 -
How Often is Site Crawled
Good morning- I saw some errors in my first crawl and immediately removed the pages from my website. I then re-created my XML sitemap and uploaded to Google. The question I have is will the site be crawled to recognize the changes in the next day or so? The pages were just placed on the site as test pages and never removed. The initial crawl that notified me it was done found the errors and were removed. Thanks for your help. Peter
Technical SEO | | VT_Pete0 -
My domain does not come in the search results, what do I do?
Hi, I have a website called www.bollykings.com It had a pretty solid rank on google for a number of keywords but 4-5 months back, it was badly affected by the Panda update. Now it comes nowhere. I have started updating and posting new articles on it since the last two months. When I search for "bollykings" on Google.com, website does not come only in the first 40 results. What could this mean?
Technical SEO | | modifyed0 -
SEOMOZ Ranking
Im pretty new to all this so hopefuly im going the right way ive been using SEOMOZ Pro for about 3 weeks. I have a website and a blog, I noticed that my website (flash based) was not doing very well but my blog was doing well. So i decided to turn my blog into a website for my main website and my old website into another category for my services. So I have now transferred everything over to my blog and on website grader I have got my rating upto 74/100 from 17/100 which is higher than anyone else in my keyword, my traffic ranking is higher but Im not even in the top 50 in google, they do have more backlins than me significanlty more but I am working on it. So the question is it the backlinks that help MOZ ranking, and how long does it take before google recognises my links to give me a position on google. My keywords are coventry wedding photographer and my site is rnblog.co.uk
Technical SEO | | robanewman0