Can I crawl a password protected domain with SEOmoz?
-
Hi everyone,
Just wondered if anybody has been able to use the SEOmoz site crawler for password protected domains? On Screaming Frog you are prompted for the username and password when you set the crawler running, however SEOmoz doesn't. It seems you can only crawl sites that are live and publicly available - can anyone confirm if this is the case?
Cheers,
M
-
Hi,
I don't think it is possible to crawl any password protectec pages. Password protected pages are usually blocked and bots are unable access those pages.
And yes it is true that only sites that are publicly and live can be crawled.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Changed domains, saw significant drop in domain authority
Hello, A little over a month ago we switched our domain from A to B. The original was .com and the new one is at a .tech for good measure. We have meticulously done 301 redirects page by page but there has been a massive drop in traffic and I checked here to see that we've gone from a 30 domain authority to a 9. The company is a few months over 1 year old, but we're really looking at traffic accumulated in just about 8 months for the old domain. Is there any way to recover some of the old juice in this one other than the re-directs? Our number of backlinks have also severely dropped despite re-directs.
Technical SEO | | SteveSaf0 -
Old Redirected Domain is replacing my current domain on SERPs
Hello everyone, All of a sudden a 2 year old redirected domain is replacing my current domain for 2 weeks now, my site is apitus.com and my old domain is aptitus.pe (the redirect is still working), however this only happens on my country google results (google.com.pe), if you check my site on google.com, everything looks ok even with a sitelink, which I no longer have on my country search results. Back to the issue, the first thing I thought was go to Search Console and take it out from the index, so I asked for access by uploading a file but since everything on that old site redirects to my current site I can't make such action. While still waiting for such access, is there anything else I could do?. Thanks in advance. PD: I'm adding the images of my SERPs CmzN8kY G3zZwwj
Technical SEO | | JoaoCJ0 -
Domain vs Sub Domain and Rankings
Hi All Wanting some advice. I have a client which has a number of individual centres that are part of an umbrella organisation. Each individual centre has its own web site and some of these sites have similar (not duplicate content) products and services. Currently the individual centres are sub domains of the umbrella organisation. i.e. Umbrella organisation www.organisation.org.au Individual centres are sub domains i.e. www.centre1.organisation.org.au, www.centre2.organisation.org.au etc. I'm feeling that perhaps this setup might be affecting the rankings of the individual sites because they are sub domains. Would love to hear some thoughts or experience on this and whether its worth going through the process of migrating the individual centre domains. Thanks Ian
Technical SEO | | iragless0 -
Can hreflang replace canonicalisation ?
Hi Im working with a site that has ALOT of duplicate content and have recommended developer fix via correct use of Canonicalisation i.e the canonical tag. However a US version (of this UK site) is about to be developed on a subfolder (domain.com/uk/ & domain.com/US/ etc so also looking into adopting the hreflang attribute on these. Upon reading up about the hreflang attribute i see that it performs a degree of canonicalisation too. Does that mean that developing the international versions with hreflang means there's no need to apply canonicalistion tags to deal with the dupe content, since will deal with the original dupe content problems as well as the new country related dupe content, via the hreflang ? I also understand that hreflang and canonicalisation can conflict/clash on different language versions of international subfolders etc as per: http://www.youtube.com/watch?v=Igbrm1z_7Hk In this instance we are only looking at US/UK versions but very likely will want to expand into non english countries too in the future like France for example. So given both the above points if you are using hreflang is it advisable (or even best) to totally avoid the canonical tag ? I would be surprised if the answers yes, since whilst makes logical sense given the above (if the above statements are correct), that seems strange given how important and standard best practice canonical usage seems to be these days. What best ? Use the Hreflang alone, or the Canonical tag alone or both ? What does everyone else do in similar situation ? All Best Dan
Technical SEO | | Dan-Lawrence0 -
Changing Domain Name
Hi all, A client has just got their .edu domain and they want to change their current domain name (a .com) to this new .edu domain. The domain's CMS is Wordpress. Please correct me if I'm wrong, but basically I will need to create a new site (but they want to keep current design), move everything across to the new domain name, and 301 URL per URL? What about all the citations that the old URLs have gotten? The website is listed on Google listings/maps for some of their local keywords. Is there anyway to preserve this? Thank you all in advance.
Technical SEO | | EdwardDennis0 -
Can I do a redirect to a new domain name only a couple of weeks after having redirected to another domain?
I have a client with two website with very similar content. Both had a lot of inbound links and performed fairly well in SERPS. We recently combined both sites and have redirected one of the domains to the other. The traffic dipped slightly initially, but is recovering nicely. Now the client registered a new domain name he would like to use for the site. Should I wait a few weeks for everything to settle down after the first redirect/consolidation of sites before doing a new redirect to a new domain name, or should I not worry about having any issues with doing it right away?
Technical SEO | | Drewco0 -
Moving domain.com to subdomain.domain.com
Hi, I like to do something, but first like to take some opinions from seomoz. My question is: 1. I have a domain: brandtrends.com and i like to move this from brandtrends.com to trends.brand.com because: brandtrends.com is on the position #15 on the second page of SERP for my "brand" keyword. I like to move it under trends.brand.com but all inbound-links are @brandtrends.com What do you thing, if i move permanently 301 from brandtrends.com to trends.brand.com does it rank under brand.com on the 1st page of SERP..? like to rank trends.brand.com under brand.com on results page...!!! I have the backlinks of brandtrends.com on my hands too, should i leave the inbound links @brandtrends.com and the new ones i build with trends.brand.com or should i change the inbound links from brandtrends.com to trends.brand.com Hope you got it! THanks
Technical SEO | | leadsprofi0