What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best blog practices for website
For my Insurance website blog, I use MOZ to help me find high DA authoritative sites, then either generate ideas from them, or rewrite the copy. If I rewrite the copy, I tend to pull from 2 - 3 top authoritative sites. Just so I don't get in trouble, but still offer the most concision information. _My question is, Is this ok to do? _ Secondly, I just read that on some .Gov sites the information is public, and that you can use it as long as you give credit. _My questions is, how do I tell which information is public? _ Thank you in advance 🙂
Moz Pro | | MissThumann0 -
Why would I be ranking for a term when it's not anywhere in my page or pages linking to it?
Hi, I have a keyword that's not anywhere in my page in page-grade tool nor are any pages linking to it with that term, why would that be ranking. The term is "dynamic card solution" This is a company we used to have 5 years ago and merged with our company. This term is ranking #1 for this page- http://www.datacard.com/instant-issuance-solutions . I can't figure why that would be even as this is a newer page in the past year. Thanks for any insight as I'm newer to moz. Laura
Moz Pro | | lauramrobinson320 -
I'm New To Moz What To Focus On First
Hello, Recently signed up to MOZ for the sites we operate in the UK. I wondered what folks would recommend I focus on first when starting to use MOZ for the first time for sites SEO? Cheers Stuart
Moz Pro | | Urban331 -
Can't log into Firefox MozBar
I just downloaded and installed the MozBar for Firefox, but it will not let me login to my account. The Log In button is gray and none of the buttons do anything when I click on them. Please help! Thank you,
Moz Pro | | Instabill
Meghan0 -
What should I put in 'Define Branded Keyword Rules' -Starting a Campaign
Hello, I am a new user here (this seems really interesting!), but english level is not very good (I am spanish) and I dont understand what means 'Define Branded Keyword Rules' Hope someone can explain me it in easy words so I can understand Thank you very much! 1362443047.jpeg
Moz Pro | | matiw0 -
Best Way to Include Social Media in Website?
Hi, how are you doing? I am new to MOZ, totally love it. I recently developed the social media for my page (www.aceromart.com) in Facebook, Google + and Twitter. (I am from Mexico, so the website is in Spanish) I am no expert in SEO whatsoever, but i like to engage my customers with great content both in my page and social media. My question is: **What the best way to include your social media links or icons on my page. Is there a program or a way to include the links. I want the people that visit ** **Should you include them in every page?, in a footer?, with icons or links. ** Thanks in advance for your advices, they are greatly appreciated. Best Regards, Jesus D
Moz Pro | | JesusD0 -
Did anyone else see "Rel Canonical" drop to zero after their latest SEOmoz crawl?
In the Crawl Diagnostics section of the SEOmoz reports, we get errors in red, warnings in yellow, and notices in blue. After my latest crawl, I saw the "Rel Canonical" part go from about 300 down to 0. Obviously, this isn't right, so I'm wondering if this is a bug that everyone is experiencing. U9W5I
Moz Pro | | UnderRugSwept0 -
HTTP 404 for 404-page?
Hi Mozzers! SEOmoz just finished crawling one of my websites and this crawl found 3 errors. One of these errors was the (custom) 404-page, because of the http-status 404. What's you suggestion about this? Should a 404-page have a status 404? Thanks in advance for your suggestions!
Moz Pro | | Partouter0