SEOmoz crawler not crawling my site
-
We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled?
We set up the campaign to track at the root domain level.
-
Thanks, Ruth. I see what you're saying about the crawl now. I'll go ahead and reach out the help team. Hopefully we can figure it out!
-
So that's really what I meant was "as long as you're seeing the full crawl," was that if your next crawl found the whole site it would probably be fine. It sounds like that's not the case, so I think the Help Team is your best bet - they're far more able to diagnose campaign-specific issues than I am
-
Thanks for the reply Ruth!
I was a little unclear as to part of your response. What were you referring to when you said "as long as you continue seeing the full crawl, you should be good to go"?
Where should I be seeing the full crawl? When I look at the Crawl Diagnostic Summary, it notes that the last crawl was on May 10, 2013 (the day we set up the campaign) and the next crawl is to start May 18, 2013. I received email notification that SEOmoz was successfully recrawled it May 11, 2013. Similar to the May 10 crawl, the May 11 crawl only went through 2 pages.
I also checked up on our robots.txt and tags. There isn't blocking anything the crawler on that front.
-
Hmm! Well, it could be a couple of things.
If your full crawl ends up crawling the whole site, it's possible that the preliminary crawl timed out on a request to your site. As long as you continue seeing the full crawl, you should be good to go.
If your next crawl continues showing just a few pages, I would check to make sure you're not blocking robots with either the meta robots tag, a robots.txt file, or nofollowed links. If you don't uncover anything out of the ordinary there, it could be an account- or campaign-specific error on our end; please email help@seomoz.org and our super-awesome Help Team can help get that fixed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl test from tools
Hi, I notice that the crawl test which is from the Research Tools doesn't really get a new crawl even though there is 2 crawl per day. It will only provide the data which was acquire from the crawl diagnostics in my pro account. There is no point for me to get the data which I get from my crawl diagnostic isn't it? Even seomoz provided with more than 2 crawl per day also useless in this case. This whole thing doesn't make sense as the crawl diagnostics will only perform a full crawl test once every week. but even the crawl test also not helping any thing out for me.
Moz Pro | | hanzoz0 -
When I did my first crawl, I was given some errors.
Do I then need to re-crawl to make sure the errors were fixed accordingly?
Moz Pro | | immortalgamer0 -
Does SEOmoz have a Keyword Research tool similar to, say, the Google AdWords tool or the WebCEO Keyword Research Tool? And where might that be? (Sorry, I'm very new to SEOmoz Pro.)
I'm looking for an SEOmoz version of the classic WebCEO Keyword Research that would give you effective suggestions based on a keyword inquiry. I've made the switch from WebCEO, but I'm trying to find something similar to that Keyword Research tool. Am I going to just need to use the Google AdWords tool for this function or does SEOmoz have it's own version?
Moz Pro | | SmokewagonKen0 -
How do you check the outbound links of a site?
There are great tools like http://www.opensiteexplorer.org that will tell you all about the inbound links. What about the more basic and easier question: What outgoing links does this site have?
Moz Pro | | SkinLaboratory2 -
Drop in Number of Crawled pages by SEOMOZ?
I noticed that the number of Crawled Pages on my website has been 2 pages only over past week. Before that the number of crawled pages was over 1000. My site has numerous pages as it is a Travel website that pulls search results for Flights, Cars, Hotels, Cruises and Vacation packages so there is a huge Database there. Can someone help? Thanks !
Moz Pro | | sherohass0 -
Crawl Diagnostics Error Spike
With the last crawl update to one of my sites there was a huge spike in errors reported. The errors jumped by 16,659 -- majority of which are under the duplicate title and duplicate content category. When I look at the specific issues it seems that the crawler is crawling a ton of blank pages on the sites blog through pagination. The odd thing is that the site has not been updated in a while and prior to this crawl on Jun 4th there were no reports of these blank pages. Is this something that can be an error on the crawler side of things? Any suggestions on next steps would be greatly appreciated. I'm adding an image of the error spike Xovep.jpg?1 Xovep.jpg?1
Moz Pro | | VanadiumInteractive1 -
Getting your site totally indexed by SEOMOZ
Hi guys! Ijust started using SEOMOZ software and wondered how it could be that my site has over 10.000 pages but in the Pro Dashboard it only indexed about 1500 of them. I've been waiting a few weeks now but the number has been stable ever since. Is there a way to get the whole site indexed by SEOMoz software? Thanks for your answers!
Moz Pro | | ssiebn70 -
What's Your SEOmoz Favorite?
Obviously, SEOmoz is full of awesomeness. But I'm curious as to what sections, tools, etc. of SEOmoz are the users' favorites? For me, it's a close race between the blog, Linkscape/Open Site Explorer, and the search overlay tool on the Mozbar. What are your favorites?
Moz Pro | | Gyi0