What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My home page has an "A" rank in Moz and not ranking in Google.
My home page recently dropped from page one in Google to not being ranked for my top keyword. The page has an "A" ranking in MOZ for the keyword. Is there a way to find out the cause. I did have what looked like a duplicate page for a while 3 months ago when a domain was forwarding to my home page incorrectly. Appeared with second domain name instead of primary. Our business has been 95% through internet leads so quite an issue. Is there anyway to find out what is going on.
Moz Pro | | FredRoven0 -
What's my best strategy for Duplicate Content if only www pages are indexed?
The MOZ crawl report for my site shows duplicate content with both www and non-www pages on the site. (Only the www are indexed by Google, however.) Do I still need to use a 301 redirect - even if the non-www are not indexed? Is rel=canonical less preferable, as usual? Facts: the site is built using asp.net the homepage has multiple versions which use 'meta refresh' tags to point to 'default.asp'. most links already point to www Current Strategy: set the preferred domain to 'www' in Google's Webmaster Tools. set the Wordpress blog (which sits in a /blog subdirectory) with rel="canonical" to point to the www version. Ask programmer to add 301 redirects from the non-www pages to the www pages. Ask programmer to use 301 redirects as opposed to meta refresh tags & point all homepage versions to www.site.org. Does this strategy make the most sense? (Especially considering the non-indexed but existent non-www pages.) Thanks!!
Moz Pro | | kimmiedawn0 -
I have double-checked the rel canonical is properly employed on our page but the On Page Grader says it's not working?
I have double-checked the rel canonical is properly employed on our page but the On Page Grader says it's not working Here is the URL - http://www.solidconcepts.com/industries/aerospace-parts-manufacturing/ What is wrong with how we are doing things?
Moz Pro | | StratasysDirectManufacturing0 -
Optimization report card says "F" but I'm ranking #1
Using the SEOMoz pro tools and several of the terms I have listed get a report card of "F" but they also show Google rank of #1, #2. Is this unusual? And should I spend a lot of time "fixing" this if we're already ranking high?
Moz Pro | | schof0 -
Okay how do I fix this cannibalization error?
Okay so check it out. I already have the problem where my main page shows 3 of my posts of 3 popular posts and the SEOmoz optimization tool tells my there are like 60 keyword occurances? for the damn page so I had to make it look damn ugly for my views and lowered it to 1 post and 2 popular posts. Can I fix that? Another thing - The SEOMOZ tool keeps telling me I have this cannibalization crap on all my posts because my homepage in the upper left corner has the keyword in it and in the footer of the template, my homepage title shows up (Keyword). What the heck do I do to fix this? And also another question - I have my main page AND 1 of my posts ranking for the same keyword. Is there a problem?
Moz Pro | | 6786486312640 -
Moz's customer service. Is this normal?
A few days ago I noticed an issue with the on-page optimizer, when I couldn't find a solution, or even any mention of the problem in the forum I posted a ticket at the help desk.
Moz Pro | | Abidos
After ten hours I got a generic reply that didn't answer my ticket so I posted saying so.
After two days of not getting any reaction I looked through the forums again, this time searching for reactions about Moz's help desk. I found the post saying that sometimes the mail with a reply isn't received correctly so I went to the page of the ticket.
Here I saw that not only didn't I get an answer but my issue had been marked as solved, even though my last message clearly stated that it wasn't. This was three days ago. So my question is, is this how Moz usually treats it's clients? I'm new and my free trial is about to expire. At this point Moz should be impressing me with their great tools and services, convincing me it's worth my money. If I can barely get their attention now I don't expect it to change once I have payed them.0 -
OSE Advanced Reports best practices
I'm curious how people use the OpenSiteExplorer Advanced Reports tab. It seems very powerful. What do you use it for? In particular, I see that it has choices for 'same C block' and 'different C block'. Those seem useful to find C blocks that my competitors have links from that I do not, but I'm not totally clear on how to construct the query. Any help or best practices would be appreciated. Thanks!
Moz Pro | | scanlin0 -
Company Name in Page Title creating thousands of "Duplicate Page Title" errors
I am new, and I just got back my crawl results (after a week or more). The first thing I noticed is that the "duplicate page title" is in the thousands, my urls and page titles are different. The only thing I can see is that our company name is at appended to the name of every title. I did search and found one other person with this problem, but no answer was given. Can anyone offer some advice? This doesn't seem right... Thanks,
Moz Pro | | AoyamaJPN0