Crawl test from tools
-
Hi,
I notice that the crawl test which is from the Research Tools doesn't really get a new crawl even though there is 2 crawl per day. It will only provide the data which was acquire from the crawl diagnostics in my pro account. There is no point for me to get the data which I get from my crawl diagnostic isn't it? Even seomoz provided with more than 2 crawl per day also useless in this case.
This whole thing doesn't make sense as the crawl diagnostics will only perform a full crawl test once every week. but even the crawl test also not helping any thing out for me.
-
Everyone, thanks for helping... Greatly appreciated.
-
Hi there Hanzo,
Thanks for reaching out! it looks like you are having some trouble with the crawl test. Keri is correct that our crawl test uses "Cached" information to construct your crawl, which can take up to 2 days to clear. While I am not sure why the sites are still reporting 404s, there's a few things that could explain why you might be getting a stale crawl:
- Your server simply did not update the changes you've made on the backend.
- There might an issue with our crawler, which in my last status there isn't a outage that I know of.
I looked into your crawl data and the 404s seems to be confirmed by the report, if you download your latest report and do a sort you should see that
Hope I was able to provide some insight, please let me know if you have questions!
Best,
-
I believe the crawl test has a cache of a couple of days. I'm asking someone from the help desk to come in and confirm, but it may be Monday before they have a chance to do so.
-
I know there is third party app which I can use to crawl,
but then isn't the crawl test is there for us to do testing instead of using other app rather than seomoz's tool?
-
You may want to give screaming frog or xenu's link sleuth. They should be able to crawl you site and give you the info you need.
-
From what I do with the crawl test, it doesn't show me the latest crawl. I need to check whether all the error 404 or related error was being solved. It only show me the old data which was crawl before but not the latest one.
-
Hanzo, what is it that you need to do that the crawl test isn't helping you with?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Could my Crawl Error Report be wrong?
HI there, I am using Yoast SEO plugin on a wordpress website. I am currently showing 70 Med priority crawl errors 'missing meta description' on my Moz pro account. This number of missing meta descriptions has increased over the last 6 weeks. But every single page / post / tag / category has both the title and meta description completed via Yoast. I requested a google bot to crawl the site a few weeks ago as thought it perhaps wasn't been crawled and updated. Any idea what the issue might be? Could the Moz report be incorrect for some reason? Or could something be blocking Moz / Google from seeing the Yoast plugin?
Moz Pro | | skehoe0 -
Anyone had an Extensive Look at Hubspot SEO Features and Tools ?
Mozz, Some co-workers are looking into some of the of tools that Hubspot offer as part of there marketing package, I understand it is part of a bigger picture for the hubspot software and moz is much more focused on technical seo etc, But i am just wondering if anyone had spent any extensive time using the hubspot software and has any feedback on the tools? Is there anything (Just the seo section) that moz cannot ? there are alot of really great moz tools i use that i haven't seen in the hubspot software, like the crawl test, the onpage grading doesn't look at as many aspects as moz, link information and opportunities looked limited compared to moz/majestic seo , no landing page tool like the one built into moz analytics campaigns and hubspot just generally seemed to provide less information ? Any advice or feedback would be grealy appreciated ? Thanks James
Moz Pro | | Antony_Towle0 -
Good tool to track external links from the website
I am in search of a tool that provides me links generating from my site to another site. Is there a software or tool that can scan the whole site and provide me what are the links of other sites in my site.
Moz Pro | | csfarnsworth0 -
Tools for Monitoring Hundreds to Thousands of Keywords and Rankings
Hi All, I am in process of doing and SEO overhaul for our five global sites in: US, UK, Canada, Sweden, France I'd like to track hundreds of keywords and rankings per site - I'm talking at least 300-400 keywords each site. Each site has its own country domain with both www and www2 domains. So, I need a keyword tool that will let me track massive amounts of keywords. I know that the Moz Pro tool helps, but we only have 350 keywords on this account. I think on this. Any suggestions on something reliable that will provide good data? I'm sure I can get some budget to purchase something, but I also can't spend too too much money. I'm not looking for a massive analytics package. Right now, I'm concerned mainly with our keyword rankings Thanks in advance!
Moz Pro | | CSawatzky0 -
I am experimenting with Raven Tools and have a question
Since I am an in-house SEO I'm a little slower to adopt some of the paid tools out there, but I am thinking of jumping in and opting for the paid version of Raven Tools. Can anyone who's currently using Raven Tools suggest how and where is the best place to export backlinks info for import into the Link Manager? I want something that will give me as much of the data as possible already filled in (i.e. PR, domain authority, anchor text, follow/no follow information etc.). I don't mind if I have to rearrange it, I just don't want to have to be inputting things like PR and anchor text one at a time by hand. I do have SEOMoz pro and I see that I can pull a CSV of backlinks and do two separate reports, one for followed and one for not followed links, etc. I'm just wondering if anyone has any additional suggestions. How about Ahrefs.com? Is anyone using the paid version of that tool in conjunction with Raven Tools for Link Management? Is there a better tool out there for Link Management than Raven Tools? (I just found this link in a Q & A thread for an open source SEO panel to try...http://www.seopanel.in/ - anyone using that one?) Thanks all!
Moz Pro | | danatanseo0 -
Crawl Diagnostics Error Spike
With the last crawl update to one of my sites there was a huge spike in errors reported. The errors jumped by 16,659 -- majority of which are under the duplicate title and duplicate content category. When I look at the specific issues it seems that the crawler is crawling a ton of blank pages on the sites blog through pagination. The odd thing is that the site has not been updated in a while and prior to this crawl on Jun 4th there were no reports of these blank pages. Is this something that can be an error on the crawler side of things? Any suggestions on next steps would be greatly appreciated. I'm adding an image of the error spike Xovep.jpg?1 Xovep.jpg?1
Moz Pro | | VanadiumInteractive1 -
How can i get seomoz to crawl a campaign on demand
hi how can i get seomoz to crawl a campaign on demand instead of on a weekly basis? For example i have corrected some error warnings and on page elements and would like it to re crawl the site sooner to see how the corrections have worked? thanks
Moz Pro | | Bristolweb0 -
Is the keyword difficulty tool the most helpful in all situations?
I understand that the scores it generates are essentially based on the difficulty of appearing on the first SERP for the keyword in question. That said, I am having a lot of difficulty finding keywords in my niche which return a score that would make this easily achievable for a site of my size.... The reason I'm pointing this out is because theoretically, a keyword could have a HIGHLY competitive first SERP, with a significant drop-off on the second SERP, which would make achieving a top ranking on that page substantially easier. So my question really is, is the importance of appearing on the first SERP so unequivocally important that it is a pointless activity to attempt deliberately to rank for keywords on the second SERP, which is ignored by the keyword difficulty tool? I know the breakdown of clicks goes something like 40% for top spot, 12% for second and downwards from there, but if a certain query has over a million searches per month, for example, it would still be possible to get considerable amounts of traffic by trying to rank highly on the second SERP, which the keyword difficulty tool cannot help with. So is this really a useless activity?
Moz Pro | | ZakGottlieb710