Webmaster Crawl errors caused by Joomla menu structure.
-
Webmaster Tools is reporting crawl errors for pages that do not exist due to how my Joomla menu system works. Example, I have a menu item named "Service Area" that stores 3 sub items but no actual page for Service Area. This results in a URL like domainDOTcom/service-area/service-page.html
Because the Service Area menu item is constructed in a way that shows the bot it is a link, I am getting a 404 error saying it can't find domainDOTcom/service-area/ (The link is to "javasript:;") Note, the error doesn't say domainDOTcom/service-area/javascript:; it just says /service-area/
What is the best way to handle this? Can I do something in robots.txt to tell the bot that this /service-area/ should be ignored but any page after /service-area/ is good to go? Should I just mark them as fixed as it's really not a 404 a human will encounter or is it best to somehow explain this to the bot? I was advised on google forums to try this, but I'm nervous about it.
Disallow: /service-area/*
Allow: /service-area/summerlin-pool-service.
Allow: /service-area/north-las-vegas
Allow: /service-area/centennial-hills-pool-serviceI tried a 301 redirect of /service-area to home page but then it pulls that out of the url and my landing pages become 404's.
http://www.lvpoolcleaners.com/
Thanks for any advice!
Derrick
-
No problem Derrick, my pleasure.
Tom
-
Wow,
Tom, thank you for the amazingly complete and well articulated response. You, kind sir, are a interwebs Rock Star!
-
Hi Derrick,
if you wish to use robots.txt you could simply use:
Allow: /service-area/*
Disallow: /service-area/This will allow access to any child of /service-area/ but not /service-area/.
You could redirect this page to your homepage if you wished, and to stop children of this page being redirected you could use RedirectMatch instead of the Redirect directive and use a simple regular expression to only redirect if the URI ends with /service-area/, like this:
RedirectMatch 301 /service-area/?$ http://www.lvpoolcleaners.com/
The $ sign at the end signs that the apache should only redirect if the URI is ending in that pattern, and the ? after the trailing / allows the redirect to happen with or without the trailing slash.
But perhaps the simplest solution to this problem would be making your /service-area/ link point to '#' if the Joomla menu will allow it. This will append an empty anchor to the url, it will not refresh or redirect the page and anchors in URLs are not counted as duplicate URLs.
For human usability this would be the nicest way to interact with the menu, as you don't want a visitor being interrupted mid-way through their buying cycle by being sent back to the homepage when they didn't ask for it.
Hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On our site by mistake some wrong links were entered and google crawled them. We have fixed those links. But they still show up in Not Found Errors. Should we just mark them as fixed? Or what is the best way to deal with them?
Some parameter was not sent. So the link was read as : null/city, null/country instead cityname/city
Technical SEO | | Lybrate06060 -
Client error 404 pages!
I have a number of 404 pages coming up which are left over in Google from the clients previous site. How do I get them out of Google please?
Technical SEO | | PeterC-B0 -
Webmaster Tools Links To Your Site
I logged onto webmaster tools today for my site and the section 'Links to Your Site' is showing no data. Also if I search using link:babskibaby.com it only shows 1 link. My site had been showing 500+ links previously. Does anyone know why this is?
Technical SEO | | babski0 -
Webmaster Index Status - Not Selected > Ever Crawled
Hi Mozzers, I would like to keep it short, while checking the Index Status in my Google webmaster tools, I noticed that the number of URLs "Not Selected" is more than the no. of URLs under "Ever Crawled" the difference is also a large no, a 5 no. figure. Does anyone seen a similar pattern or any insights on this would be quite helpful. Cheers,
Technical SEO | | RanjeetP0 -
406 errors
Just started seeing 406 errors on our last crawl (all jpg related). Seomoz found 670 of these on my site when there were 0 before. I have checked the MIME and everything seems to be in the right order. So could it be that Seomoz-crawler is showing errors that aren't really errors?
Technical SEO | | smines0 -
When is the last time Google crawled my site
How do I tell the last time Google crawled my site. I found out it is not the "Cache" which I had thought it was.
Technical SEO | | digitalops0 -
How to fix 404 (Client Error) errors in wordpress blog?
hey A very quick question...after analyzed my wp blog I've found "34" 404 (Client Error) Errors and I don't know how to fix it, do you know how?? *I renew html code of 404 of my wordpress blog.
Technical SEO | | akitmane1