Reset Crawler
-
Hello, Does anyone know how to reset the crawler? We recently uploaded our new website and deleted the current campaign but it seems the crawler is caching our old websites data and not the new so every time we try to create a new campaign with the same details, it's just pulling everything from cache it seems. Thanks
-
Hi ForzaHost!
This is Megan from the SEOmoz Help Team. I see that you've opened a ticket that my colleague, Chiaryn, is working with you on. We'll be following up with you in the ticket instead so we can look at your campaign specific information and help figure out what's happening.
Cheers!
-
Hello,
Sorry, I meant SEOMoz's Crawler.
I should have been more clear, I apologize.
Thanks.
-
ForzaHost,
One thing you might try is Fetch As Google. In GWT on dashboard you will see health. Under health click Fetch as Google and the drop down will allow you to choose what you want. (Probably Web). You can insert up to 500 directories (one at time to check the results).
Under Fetch as Google, is Index where you can see when the site was indexed, etc.
Hope this is helpful,
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Site Crawler not working but on-demand crawler working
Hi, In Moz pro, when using Site crawler (or recrawl), we are seeing message site is banned. But when using on-demand crawler, it could generate report successfully. I just like to know if in both these cases, it is roberbot that is used! And kindly note, site crawler was perfectly working before. So the required setup is already in place from long time. Site crawler ban issue started appearing from nov/dec 2023. . Could you please us understand how could we possibly make site-crawler work?
Moz Pro | | gilesd
I am happy to provide more details if you need any. Thanks0 -
Unsolved Moz crawler not working
Hi Moz crawler keep failing on my site with the error showing : Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. I'm not sure what am I missing out.. this is my robots.txt.. i don't think Im missing anything else.. https://www.wearefutureheads.com/robots.txt can the support team help ?
Moz Pro | | teikh0 -
Possible Crawling Problem with Screaming Frog and Moz Crawlers
So I'm not sure if what I'm seeing is a problem or not. As of about two weeks ago the Moz crawler has only been able to see www.mysite.com, and none of the links, content, title, ect associated with the page. Essentially the report has one line, what should be the homepage, but it's not able to pull any information from the page but does show a 200 http status code. The report shows nothing blocked by robots or any errors. When I use screaming frog to crawl the site about 75% of the time it just reports one line www.mysite.com with a 200 status code, but again the crawler is not able to actually see the html. The other 25% of the time it works perfectly fine, crawls all pages and sees all meta info and content. There are no errors in Google WMT and everything looks ok there. We have seen a traffic drop the last two weeks but I don't know if this is the reason for it. I can't publicly post the page but if someone has an idea of what might be going on I'd be happy to PM them. Thanks
Moz Pro | | CJ50 -
SEOmoz crawler not crawling my site
We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled? We set up the campaign to track at the root domain level.
Moz Pro | | IMM0 -
Seomoz crawler problems
I have had Seomoz for about a month. It has crawled about 1000 pages. I have about 10,000 pages total for the site. Why are these others being a problem? I have contacted help but the guy isn't any help, we have just been going back and forth for the last two weeks. Any suggestions?
Moz Pro | | EcommerceSite0 -
Crawler has stopped crawling my website
Hi all, I recently changed my site www.ipbskinning.com to ipbskinning.com. For some reason my weekly scan states that no pages have been crawled. I tried making a new campaign with the sub-domain ipbskinning.com but still the starter crawl returns nothing? Am I doing something wrong? :S Thanks all!
Moz Pro | | pezza34340 -
Why does SEOMoz crawler ignore robots.txt?
The SEOMoz crawler ignores robots.txt It also "indexes" pages marked as noindex. That means it is filling up the reports with things that don't matter. Is there any way to stop it doing that?
Moz Pro | | loopyal0 -
Why is the crawler following form action links?
I have an issue with one of my sites where the SEOMoz crawler is following some form action links. It is my understanding that the crawler will ignore these links. Why would it not be ignoring them in certain cases. If you need more detail, please ask. Thanks.
Moz Pro | | AmberHanson0