Crawling Website
-
How often does Moz crawl the website? I have a lot of high priority crawl issues but my web development team says that it fixed them. However, Moz still shows me that none of the errors are fixed.
-
The crawl test will show the results in a CSV file, but not update on your dashboard. It is a way to spot-check and see if the changes have been made.
When you go to your crawl diagnostics page, it will tell you the date of the last crawl. We crawl on the same day each week.
-
So will it update automatically in my overview as well? When will it be crawled on a weekly basis, in the beginning of the week or end of the week? How do I know if my website has been crawled yet?
-
We crawl on a weekly basis. You can force a crawl for up to 3000 URLs at the crawl test at https://moz.com/researchtools/crawl-test.
-
Please see this discussion to help you out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz can't crawl my site.
Moz cannot carry out the site crawl on my online shop. Not really sure what the issue is, it has no problem getting onto my site when you use www. before the address, but it needs to be able to access bluerinsevintage.co.uk Stuck as what to do, we are a shopify store. Anyone else had this problem, or know what i need to change so they can crawl the site? thjis is the page they are getting when trying to get on bluerinsevintage.co.uk but if they use www.bluerinsevintage.co.uk the site comes up. Adam
Getting Started | | bluerinsevintage0 -
Crawl rate
How often does Moz crawl my website ? (I have a number of issues I believe I have fixed, and wondered if there was a manual request to re-crawl ?) Thanks. Austin.
Getting Started | | FuelDump0 -
How to locate page with the duplicate title? (Crawl Diagnostics - Duplicate Titles Warning)
I am looking through my crawl diagnostics and one of my errors states that a page has a duplicate title. My problem is that I do not know how to find the duplicate. Any advice here?
Getting Started | | bearpaw0 -
Creating a daily to-do list for SEO my website
Hi all, I'm very new here, and I'm quite slow in terms of learning things. Now I've had MOZ for about one and half month, but still only just understand to use on-page grade to SEO my site. I've been spending quite a bit money (for a small business) for online marketing: facebook adverts $50 a day; google adwords $10 a day; MOZ $99 a month... However, the conversion rate isn't ideal... only less than 2% Now what I'm trying to do is, while keep doing the cash burning online adverts, do more SEO on a daily basis. Can I please ask: other than optimise 5 pages' on-page grade, what else should I include into my daily SEO to-do list? Many thanks!
Getting Started | | alexlovesmoz0 -
Multiple websites -- one for each brand
I am a MOZ newb, and didn't find answer to the following: We are an enterprise with four distinct brands, each acting as a (reasonably) independent subsidiary. I want to track their performance and report on keyword rankings for the enterprise. Is Moz Analytics capable of this? In Google Analytics, for instance, we have an account for each brand, and multiple properties within each account, and so on. I use 3rd-party dashboard tools to combine multiple accounts into a single report. Can I do something similar here. Thanks in advance.
Getting Started | | afhiWebMaster0 -
High Number of Crawl Errors for Blog
Hello All, We have been having an issue with very high crawl errors on websites that contain blogs. Here is a screenshot of one of the sites we are dealing with: http://cl.ly/image/0i2Q2O100p2v . Looking through the links that are turning up in the crawl errors, the majority of them (roughly 90%) are auto-generated by the blog's system. This includes category/tag links, archived links, etc. A few examples being: http://www.mysite.com/2004/10/ http://www.mysite.com/2004/10/17/ http://www.mysite.com/tagname As far as I know (please correct me if I'm wrong!), search engines will not penalize you for things like this that appear on auto-generated pages. Also, even if search engines did penalize you, I do not believe we can make a unique meta tag for auto-generate pages. Regardless, our client is very concerned seeing these high number of errors in the reports, even though we have explained the situation to him. Would anyone have any suggestions on how to either 1) tell Moz to ignore these types of errors or 2) adjust the website so that these errors now longer appear in the reports? Thanks so much! Rebecca
Getting Started | | Level2Designs0 -
Is there a way to force re-crawl?
1- Can we force the system to recrawl a website without having to wait for the scheduled date? 2- Please forgive me as I am new to this, but is "Moz Analytics" supposed to replace the "Pro"? Thank you!
Getting Started | | BlackTreeIT0 -
Cant download my crawl csv
When I click on the [download csv] in my crawl campaign section nothing happens.
Getting Started | | digitalmedialounge0