Google haveing problems accessing part of my site
-
hi my site is, www.in2town.co.uk and for a few weeks now google has had trouble accessing part of my site.
Today googlewebmaster tools tells me that google is having major problems it shows, 123 pages where access were denied.
i have spoken to my hosting company who could not find a problem, so not sure what to do now. can anyone please give me advice on what the problem may be.
any help would be great
-
That's correct. I don't know the specifics, however in this case, I would assume the reduced frequency/volume would mean less trouble dealing with the hosting provider's internal challenges.
-
i have read it time and time again and do not understand how it can help. if you reduce the crawl rate then that means you reduce the number of pages and the number of times google visits your site or am i wrong
-
Fascinating how some problems can come up like this - it's not anything I'd ever seen - curious to see whether reducing Google's crawl rate will help.
-
i have asked about this problem on the google webmaster forum
http://productforums.google.com/forum/#!mydiscussions/webmasters/BogC5OZqdyM
and people are saying that it is a crawler issue with my hosting company,
they are suggesting a number of things that could be wrong which includes
Many hosting providers need to have mechanisms to throttle/limit either the number of requests or bandwidth to their customers sites.
However in this instance, they are not allowing for search engine crawlers which typically make numerous requests in quick succession. Significantly, their server's response is technically incorrect (RFC 2616), it should either not respond, or respond with a 503 (service unavailable). Google is aware of such incorrect 403s and will eventually try to crawl them again.To mitigate the issue somewhat, you can reduce your site's crawl rate in Google's Webmaster Tools
-
This confirms the need to get an expert involved - one experienced in these types of issues - a programmer or systems administrator type expert, not necessarily an SEO expert.
-
the site has been built from a template but this problem has only started to happen in the past month and the site is a couple of years old
-
unfortunately I don't know exactly what else to suggest. Who built the site? Is it a custom site, or was it automatically built using a hosting company automated site builder?
If it's a custom site or you have a developer you've worked with to build it, speak with them. if it's a site automatically built using a hosting site building system, only they can help resolve it. If not, you'll need to get a different site built. The problem you are describing should not be happening.
-
hi, when i put in http://www.in2town.co.uk on the tool you told me about it brings up some pages that are fine and some that are 403, and the 14th one down is the url as above. that comes up with a 403 however, when i just put the url on its own, it comes up as a 200, which is strange.
Have you got any ideas on what i should say to the hosting company. i have spoken to them a number of times and all i get is, everything is fine but it is not
-
When I put that URL in, I get a "200" "OK" response. If you get a "403 forbidden, you need to speak with the site systems (server) administrator to find out if they can pinpoint the problem. Something is broken on the site at the server level to cause that. Even if it's only intermittently broken.
-
hi, thanks for this. i have used that tool and it has come up with as an example the following
http://www.in2town.co.uk/news/parenting/parents-need-to-act-against-child-obesity 403 forbidden
i am not sure why this has happened and this is the same result as i am being shown in google webmaster tools.
-
One step I would suggest is that I would suggest using a tool like Screaming Frog. It can crawl your site and report on problems or broken internal links on the site.
I would also go into Google Analytics and go into the Content/Speed/Page Speed data. I would then expand the date range to cover from before they started listing problems until the day before you run the report (the current day can lag in updating in Google Analytics).
From there, look to see if there are any specific days where Google shows the page speed average (in the timeline chart) as being unusually high. There may be days that are more problematic than others.
Finally, with that same date range in place, I would filter down the list of pages in the speed report down to show me one of the pages Google lists as being unreachable. Then do the same with some others. See if Google shows any odd page speed times on those.
I would also then go to URIValet.com and run a check on some of those pages as well to see if URIValet shows the pages reachable and returning a proper "200" "okay" header message.
This is not a guarantee fix, however it can help bring answers.
-
the site is www.in2town.co.uk www.in2town.co.uk can you try it with the link and without the link, not sure why you cannot get on the site but will be interested to find out.
this problem of google have access errors has been going on for some weeks now and i do not understand what is going on
-
Hi Claire,
I was able to view your robots.txt file but nothing else. I can't seem to get to the site. Is the URL correct?
Dana
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google not index main keyword on homepage in 2 countries same language, rest of pages no problem
Hello, Two the same websites, two countries, same language http://www.lavistarelatiegeschenken.nl / http://www.lavistarelatiegeschenken.be The main keyword "relatiegeschenken" in top 10 of netherlands (steady position for 2 years) and in ** belgium** not in top 15****0 the main keyword "relatiegeschenken| but other keywords good positions, thats so strange I didn't understand and now every thing turned around suddenly: Now the main keyword "relatiegeschenken suddenly " not anymore in top 10 in the netherslandsits gone and other kewyords still good positions , now **main keyword suddenly in top 10 of belgium 2 years was not **other pages still ok. It are exactly the same websites and the same language. So double content But my programmer told me in google webmaster tools settings are right, so no problem with double content ? I really dont understand first main keyword in netherland in top 10 and in belgium not, now changed, now in belgium top 10 and not findable in the netherland on the main keyword. Maybe problem in code ? Maybe problems in code because websites are identical and active in two different countries wit same language ? No message about a penalty message in WMT, no spam links week i delete two strong but according to Linkdetox a bad links. I can not find a solution but its really important keyword that my customer want back in top 10 in netherland, like it was. All other positions and visitors are the same. Befor i have had this with belgium site, also main keyword google not index homepage. But suddenly no google show in belgium in top 10 Its turned around Kind regards, Marcel
Technical SEO | | Bossie720 -
Index problems
“The website http://www.vaneyckshutters.com/nl/ does not show in the index of Google (site:vaneyckshutters.com/nl/). This must be the homepage in the Netherlands. Previously, the page www.vaneyckshutters.com was redirected to /nl/. This page is accessible now with a canonical tag to http://www.vaneyckshutters.com/nl/ in the hope to let /nl/ be indexed. When we look at the SERPS for keyword ‘shutters’, the page http://www.vaneyckshutters.com/ is shown in Google.nl on #32 and in Belgium #3. Problem & question: Why is it that /nl/ has not been indexed properly and why is it that we rank with http://www.vaneyckshutters.com on ‘shutters’ instead the/nl/ page?”
Technical SEO | | Happy-SEO1 -
Googlebot cannot access your site
Hello, I have a website http://www.fivestarstoneinc.com/ and earlier today I got an emil from webmaster tools saying "Googlebot cannot access your site" Wondering what the problem could be and how to fix it.
Technical SEO | | Rank-and-Grow0 -
Help: Google Time Spent Downloading a Page, My Site is Slow
All, My site: http://www.nationalbankruptcyforum.com shows an average time spent downloading a page of 1,489 (in milliseconds) We've had spikes of well over 3,000 and lows of around 980 (all according to WMT). I understand that this is really slow. Does anyone have some suggestions as to how I could improve load times? Constructive criticism welcomed and encouraged.
Technical SEO | | JSOC0 -
Can I noindex most of my site?
A large number of the pages on my site are pages that contain things like photos and maps that are useful to my visitors, but would make poor landing pages and have very little written content. My site is huge. Would it be benificial to noindex all of these?
Technical SEO | | mascotmike0 -
How can I get Google to crawl my site daily?
I was wndering if there was a trick to getting google to crawl my website daily?
Technical SEO | | labradoodlelocator0 -
Site just will not be reincluded in Google's Index
I asked a question about this site (www.cookinggames.com.au) some time ago http://www.seomoz.org/qa/view/38488/site-indexing-google-doesnt-like-it and had some very helpful answers which were great. However I'm still no further ahead. I have added some more content, submitted a new XML sitemap, removed the 'lorem ipsum...' Now it seems that even Bing have ditched the site too. The number 1 result in Australia for the search term 'cooking games' is now this one - http://www.cookinggames.net.au/ which surely is not so much better to deserve a #1 spot whilst my site is deindexed? I have just had another reconsideration request 'denied' and am absolutely out of ideas/. If anyone can help suggest what I need to do... or even suggest how I can get feedback from the search engines what's wring that would be fantastic. Thank you David
Technical SEO | | OzDave0