902 Error and Page Size Limit
-
Hello,
I am getting a 902 error when attempting to crawl one of my websites that was recently upgraded to a modern platform to be mobile friendly, https, etc.
After doing some research it appears this is related to the page size. On Moz's 902 error description it states: "Pages larger than 2MB will not be crawled. For best practices, keep your page sizes to be 75k or less."
It appears all pages on my site are over 2MB because Rogbot is no longer doing any crawling and not reporting issues besides the 902. This is terrible for us because we purchased MOZ to track and crawl this site specifically.
There are many articles which show the average page size on the web is well over 2MB now: http://www.wired.com/2016/04/average-webpage-now-size-original-doom/
Due to that I would imagine other users have come up against this as well and I'm wondering how they handled it. I hope Moz is planning to increase the size limit on Rogbot as it seems we are on a course towards sites becoming larger and larger.
Any insight or help is much appreciated!
-
Hi there! Tawny from Moz's Help Team here.
I'm so sorry that you're running into persistent 902 errors from your site crawl! Unfortunately, this is a technical limitation of our tools. We're in the process of building a bigger, badder, better crawler, but it isn't quite ready yet.
You're certainly not the first person to write in about this issue, so I'll be sure to take your feedback to our product team! The ability to crawl larger pages is going to be more and more important as time moves forward.
Thanks for writing in to let us know you were having issues! If you have follow-up questions or need any other help with the tools, feel free to write in to help@moz.com and we'll get things sorted out for you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Cant figture out why On-Page Grader is not recognizing the heading tag text
Hi all, I'm using the On-Page Grader tool to optimize some of our pages: https://analytics.moz.com/pro/research/page-grader?keyword=Russian Immigration Lawyers in Sacramento&url=https%3A%2F%2Fwww.russianlawyers.us%2Fus%2Fcalifornia%2Fsacramento-immigration One of the suggestions is "Optimal Use of Keywords in Header Tags". They keyword is "Russian Immigration Lawyers in Sacramento" and we do have it in an tag on the page (located in the right sidebar). The tool doesnt seem to catch that though. Can someone help me please? Thanks
Moz Bar | | heritageweb1 -
How to export to csv all top pages links in link explorer?
When i press export to csv it does not offer to choose how many rows but exports to file only 500 rows. But there is more than 3000. How to export all rows? Thanks
Moz Bar | | AndrisZigurs0 -
Page Authority drop to Zero with new Moz crawl
My Site having page authority about 15 to 17 and i added many new pages in end of November. Everything is fine in December Moz Crawl update (Page Authority - Domain Authority) but in recent update my DA id drop 3 points and all my pages authority vanished drop to 1. Can anyone help me out to understand whats going on there!!!
Moz Bar | | signsny0 -
After adding a new page to optimization section it shows up as dashes
After adding a page to the optimization section, it doesn't put the score, title, or ranking on. It shows up as dashes After adding a new page to the optimization section it shows me my score. I go back to the home page of the optimization section to see it's ranking and all the details are dashes. Like the moz score, the ranking, and the title. It has only been happening recently. What am I doing wrong?
Moz Bar | | leannegoff0 -
How Does the On-Page Grader Know about my Target Keyword? Or how can I tell it?
So we've optimized a home page for a particular keyword, "blue widgets tx". But the on-page grader is giving information about "tx blue widgets", "blue widgets in texas", etc. and telling us we are an "F" for these keywords. Is it possible to tell the on-page grader, "Hey, just worry about 'blue widgets tx', and forget about those other ones"? I know I can do it by hand, but it will take forever.
Moz Bar | | Titan5520 -
Duplicate content reported for totally different pages
Hi, The Moz report is showing just over 21,500 duplicate page issues on our site. This is more or less every page we have. However when I look at the pages it says are duplicates they are totally different (it could for example report that a news page for 2009 is the same as a product page just added which has no relation when you read the content or view the page). What sort of thing could it be picking up as duplicate content? I assume it must be something in the HTML for the site rather than the actual page content as there is no cross over at all on the pages highlighted. The only issue I can currently identify is that the menu for the mobile version of the site has a huge number of internal links which I will cut down. If the tools purely look at HTML content this could be seen as duplicate but shouldn't it be clever enough to realise what is content and what is site structure? Thanks,
Moz Bar | | TW-Steve0 -
Duplicate content errors
Hi I am getting some errors for duplicate content errors in my crawl report for some of our products www.....com/brand/productname1.html www.....com/section/productname1.html www.....com/productname1.html we have canonical in the header for all three pages rel="canonical" href="www....com/productname1.html" />
Moz Bar | | phes0 -
My 301 Error and Duplicate Title Content Issue is Growing !
When i redirect some of my page - it shows error. not redirecting and i made this 3-4 months before, no effect. All Errors under each category make me gone sick.
Moz Bar | | Esaky0