Magento Hidden Products & Google Not Found Errors
-
We recently moved our website over to the Magento eCommerce platform. Magento has functionality to make certain items not visible individually so you can, for example, take 6 products and turn it into 1 product where a customer can choose their options. You then hide all the individual products, leaving only that one product visible on the site and reducing duplicate content issues.
We did this. It works great and the individual products don't show up in our site map, which is what we'd like. However, Google Webmaster Tools has all of these individual product URLs in its Not Found Crawl Errors.
! For example:
White t-shirt URL: /white-t-shirt
Red t-shirt URL: /red-t-shirt
Blue t-shirt URL: /blue-t-shirt
All of those are not visible on the site and the URLs do not appear in our site map. But they are all showing up in Google Webmaster Tools.
Configurable t-shirt URL: /t-shirt
This product is the only one visible on the site, does appear on the site map, and shows up in Google Webmaster Tools as a valid URL. !
Do you know how it found the individual products if it isn't in the site map and they aren't visible on the website? And how important do you think it is that we fix all of these hundreds of Not Found errors to point to the single visible product on the site? I would think it is fairly important, but don't want to spend a week of man power on it if the returns would be minimal.
Thanks so much for any input!
-
I also have this same issue.... looking for a solution.
-
Hi there,
I know this is an older question, but I wanted to follow up to see if you found a solution. We're currently experiencing the same issue with simple/not visible individually URLs appear in WMT's crawl error report.
Thanks in advance for any help!
-
The problem I am currently facing is a site where the individual 'Simple Products' were not set to be invisible. They have all been indexed!
So I have the same question really.
-
Yes, the page is modified based on selected options and it is done via Javascript.
-
OK. In a product page, the individual page is modified by a customer selecting options? If so, is that accomplished with Javascript substitution?
-
So those individual pages never actually appear on the site. They are just created to allow Magento to pull inventory on those items from that configurable product, which is why I'm not sure how Google is finding them.
For example, if I go to mysite.com/white-t-shirt, I would get a 404 (and if I searched for it, nothing would come up) because as far as the world outside of Magento admin is concerned, that URL doesn't exist.
-
How do those pages appear? In response to an AJAX call? A sitesearch? I've been hearing that Google has increased their ability to crawl pages that appear through dynamic calls.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Update
My rank has dropped quite a lot this past week and I can see from the Moz tools that there is an unconfirmed Google update responsible. Is there any information from Moz on this?
Intermediate & Advanced SEO | | moon-boots0 -
Should I use individual product pages for different formats of the same product?
Hi All -- I'm working with a publishing client who is launching a new site. They have a large product catalogue offered in a number of format types (print, ebook, online learning, packages) with each one possessing a unique ISBN code. From past experience, I know that ISBN codes can be a really important ranking factor. We are currently trying to sort out product page guidelines. The proposed methods are: A single product page for all formats. The user then has the option to select which format they wish to purchase. The page would contain all key descriptors for each format, including: individual ISBN, format, title, price, author, etc. We would then use schema mark-up just to assist search engines with understanding and crawling. BUT we worry that the single page won't rank as well as say an invidual product page with a unique ISBN in the URL (for example: http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470573325.html) Which leads to the next option... Individual URLs for each format. We understand that most e-commerce guidelines state you shouldn't dilute link equity amongst multiple pages with very similar products and descriptions. BUT we want searchers to be able to search by individual ISBN and still find that specific format within the SERPs. This seems to rule out canonicalizing, because we don't prefer one format over the other and still want say the ebook to show up as much as the print version. If anyone has any other options or considerations that we haven't thought about, it would be greatly appreciated. Thanks, U
Intermediate & Advanced SEO | | HarborOneBank0 -
Google search results
I have been doing some searches on google to see where my new site shows up, I started using the search words "graphic design firm st. louis" as a gauge, because my title is St. Louis Missouri Graphic Design Firm. I showed up on about page 5 to start , if I include the word "firm" and a few pages further back if I just search "graphic design st. louis", without the word firm. It seemed i was slowly moving up pages with both searches and then a few days ago I jumped to page 1 for search "graphic design firm st. louis" the thing is it doesnt show up at all now if i search "graphic design st. louis" without the word firm. what would cause the one search to jump so high while the other one dissapeared completely?? and what can i do? my keyword density is same for both , any ideas.
Intermediate & Advanced SEO | | eric69660 -
Google Not Displaying Rich Snippets
We implemented rich snippets for products some time ago. When viewing our site through a site:xxxx.com on Google, they don't show for every product, despite the fact that they should. I've taken some of the URLs that don't show rich snippets in the SERPs, ran them through Google's testing tool, and they display fine. Not sure what's going wrong here. Any thoughts?
Intermediate & Advanced SEO | | Kingof50 -
How to title my products?
Hi, really struggling with product titles. Or should I say keeping staff writing out titles. If I sell a Coat. I would like the product titled like: Armani Jeans Coat Green but staff are doing it like Armani Jeans Fur Hooded Coat Green. Now I think this effects our SEO efforts as how likely are people to search for the Fur Hooded coat part? Yes we might hit the numbers of the small search but is it really worth it? Would it not be best put this Fur Hooded part in the short description and long description? I am trying to make my SEO titles and meta descriptions consistant for 1 product. But find it hard writing out Fur Hooded in the Meta Title when I know numbers will be minimal? The SEO titles or the product titles are effectively links on the website for each product. So hold more weight and the product titles act as the H1 titles on the product page itself. Surely we would be best using Armani Jeans Coat Green rather than such a long, obscure title that will gain very little search? Whats best way to approach this issue? As we can have products titled like: Creative Recreation Kaplan Patent Leather/Snake Trainers Charcoal. Which to me is too long, too obscure. Surely the extra detail should go in Short Description which is visible on the catelog page and keep the product titles shorted and more to the point, eg Creative Recreation Kaplan Trainers Snake Charcoal? All this Patent Leather/ business seems pointless to me? Any ideas?
Intermediate & Advanced SEO | | YNWA0 -
Google & Bing not indexing a Joomla Site properly....
Can someone explain the following to me please. The background: I launched a new website - new domain with no history. I added the domain to my Bing webmaster tools account, verified the domain and submitted the XML sitemap at the same time. I added the domain to my Google analytics account and link webmaster tools and verified the domain - I was NOT asked to submit the sitemap or anything. The site has only 10 pages. The situation: The site shows up in bing when I search using site:www.domain.com - Pages indexed:- 1 (the home page) The site shows up in google when I search using site:www.domain.com - Pages indexed:- 30 Please note Google found 30 pages - the sitemap and site only has 10 pages - I have found out due to the way the site has been built that there are "hidden" pages i.e. A page displaying half of a page as it is made up using element in Joomla. My questions:- 1. Why does Bing find 1 page and Google find 30 - surely Bing should at least find the 10 pages of the site as it has the sitemap? (I suspect I know the answer but I want other peoples input). 2. Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening. 3. Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google. I will look forward to your input! 🙂
Intermediate & Advanced SEO | | JohnW-UK0 -
Custom Error and page not found responses
When there is a 500 Internal Server Error, is it better to return an HTTP 500 response and custom error page from the requested URL, or is it better to return a 302 redirect? The redirect would send the browser to the custom error page, which would return the HTTP 500 result. We tell Google not to index or follow our error pages, so if Google sees an error at a URL, we don't necessarily want Google to think that the URL should be ignored. That's why the alternative would be to redirect to a custom error page with it's own URL. Similarly, what's the best approach if the response is a 404? Return HTTP 404 and custom 404 page from the requested URL, or redirect? Thanks.
Intermediate & Advanced SEO | | dbuckles0 -
Google indexing flash content
Hi Would googles indexing of flash content count towards page content? for example I have over 7000 flash files, with 1 unique flash file per page followed by a short 2 paragraph snippet, would google count the flash as content towards the overall page? Because at the moment I've x-tagged the roberts with noindex, nofollow and no archive to prevent them from appearing in the search engines. I'm just wondering if the google bot visits and accesses the flash file it'll get the x-tag noindex, nofollow and then stop processing. I think this may be why the panda update also had an effect. thanks
Intermediate & Advanced SEO | | Flapjack0