Magento E-Commerce Crawl Issues
-
Hi Guys,
First post here! I am responsible for a Magento e-commerce store and there are a few crawl issues and potential solutions that I am working and would like to get some advice to see if you agree with my approach.
Old Product Pages - The majority of our stock is seasonal, therefore when a product sells out, it is not usually going to come back into stock. However the approach for Magento websites is to leave the page present but take the product off the category pages, so users can still find these pages from the search engines and they are orphaned pages as not linked to from elsewhere and not totally clear products are out of stock (just doesn't show the size pulldown or 'Add to Basket' button). There is no process in place to 301 redirect these pages either.
My solution to this problem is to:
1. Change design of these pages so a clear message is shown to users that the product is out of stock and suggest related products to reduce bounce rates. I was also planning on having a link from an 'Out of Stock' page on the site to these products so they are orphaned but is this required do you think?
2. When I know for sure (e.g. over a month) that the product will not be returned (e.g. refund) by the user, then 301 redirect the product pages back to category page. How do other users 301 redirect their pages in Magento, I would like an easy to use system.
Crawl Errors Identified in Google Webmaster Tools
It seems in the last 2 weeks there has been a sharp increase in the number of soft 404 pages identified on the website. When I inspect these pages they seem to be categories and sub categories that no longer have any products in them.
However, I don't want to delete these pages as new products might come in and go onto these category pages, therefore how should I approach this? A suggestion I have thought of is to put related products on to these pages?
Any better ideas?
Thanks,
Graeme
-
Hi Graeme,
For old product pages - your solution is good regarding showing users alternatives to the out of stock products. No need for an "out of stock page" as there's no value in that for crawlers or users. Regarding point 2 - if you redirect discontinued product pages to category pages that should be fine although Google may regard that as a soft 404. If there are loads of products like this and you 301 them in one go then the chances are it will flag up in Google WMT. If there are a small number and you introduce them gradually then you'll probably be fine.
For the crawl errors question, adding value to the pages in terms of related products is a good solution if that's viable and the pages will be different enough from each other (i.e. no duplicate content). One thing that isn't clear at the moment is if you're redirecting empty category pages all to the homepage - or if it's possible to redirect or canonical them to their parent category.
e.g. For home -> clothing -> men's clothing -> shoes
If all the men's shoes are discontinued, then redirect that page to men's clothing rather than to the homepage. This reduces your chances of getting a soft 404, and is also arguably a better user experience.
Hope that helps,
George
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Post migration issues - #11 + configuration issue
Hello Moz community. I'm keen to find out your experiences on the following: Have you ever experienced a migration whereby a large % of keywords are stuck in position #11 - post migration? The keywords do not move up or down (whilst competitors jump from 13 to 9 and vice versa) over a three month period. Please see the % difference in the attached e-mail. (sample 1,000 keyword terms) Question: Has anyone ever experienced this type of phenomenon before? If so - what was the root cause of this and did this happen post migration? What solution did you use to rectify this? Have you ever seen a cross-indexing issue between two domains (each domain serves a different purpose) post migration, which impacts the performance of the main brand domain? I will explain a little further - say you have www.example.com (brand site) and www.example-help.com (customer service site) and the day the brand website is migrated (same domain - just different file structure), www.example-help.com points to the same server that www.example.com is on (with a different file structure) and starts to inherit the legacy file structure. For example, the following is implemented on migration day: I will explain a little further - say you have www.example.com (brand site) and www.example-help.com (customer service site) and the day the brand website is migrated (same domain - just different file structure), www.example-help.com points to the same server that www.example.com is on (with a different file structure) and starts to inherit the legacy file structure. For example, the following is implemented on migration day: For example, the following is implemented on migration day: www.example.com/fr/widgets-purple => 301s to www.example.com/fr/widgets/purple But www.example-help.com now points to the same server where the customer service content is now hosted. So although the following is rendered: So although the following is rendered correctly: www.example-help.com/how-can-we-help We also have the following indexed in Google.fr - competing for the same keyword terms and the main brand website has dropped in rankings: www.example-help.com/fr/widgets-purple [legacy content from main brand website] Even when legacy content is 301 redirected from www.example-help.com to www.example.com, the authority isn't passed across and we now have www.example.com (as per Q1) a lot lower in Google than pre-migration. Question: Have you ever experienced a cross-indexing issue like above whereby Google potentially isn't passing authority across from legacy to the new setup? I'm very keen to hear your experiences on these two subjects and whether you have had similar problems on some of your domains. E0hbb
Intermediate & Advanced SEO | | SMVSEO0 -
E-Commerce Mobile Pagination Dillema
Hi Everybody, I'm managing the SEO for an E-commerce site with different desktop and mobile sites (meaning, not responsive). We're changing the way reviews on mobile product pages will be displayed from 'view all' to pagination (due to server load). Basically the above the fold part of the page will always display the product, and below the fold will have x numbers of reviews on each page. But here is where it gets tricky: 1 - A different number of review pages will exist on mobile vs desktop (due to different no. of reviews per page on each device) - so I'm wondering what's the solution regarding canonicals. Usually every mobile page points to its desktop parallel, but now we'll have non-matching pages. 2 - The users will be able to change the no. of reviews displayed on each page. So the number of paginated pages will change accordingly. I was thinking about a solution where all the reviews will be in the first page's html (and only X of them will be displayed on screen), and all the other paginated pages will be created dynamically (with # and won't be indexed, so basically no pagination in mobile). Does anyone think this can be seen as cloaking or has any other thoughts? Thanks, Sarah
Intermediate & Advanced SEO | | Don340 -
Use Nonindex or Canonical on product tags of a e-commerce site
I run a e-commerce site and we have many product tags. These product tags come up as "Duplicate Page Content" when Moz does it's crawl. I was wondering if I should use Nonindex or Canonical? The tags all go to the same product when used so I figure I would just nonindex them but was wondering what's the best for SEO?
Intermediate & Advanced SEO | | EmmettButler1 -
What is the best practice for URLs for E-commerce products in multiple categories?
Hello all! I have always worked successfully with SEO on E-commerce sites, however we are currently revamping an older site for a client and so I thought I'd turn to the community to ask what the best practices that you guys are experiencing for url structures at the moment. Obviously we do not wish to create duplicate content and so the big question is, what would you guys do for the very best structure for URLs on an E-commerce site that has products in multiple categories? Let's imagine we are selling toy cars. I have a sports car for sale, so naturally it can go in the sports cars category and it could also go in to the convertibles category too. What is the best way you have found recently that works and increases rankings, but does not create duplicate content? Thanks in advance! 🙂 Kind Regards, JDM
Intermediate & Advanced SEO | | Hatfish0 -
Recommended e-commerce site search for Magento?
Does anyone have recommendations for any particular site searches for large e-commerce sites based on Magento? Some (hopeful) requirements: Possibility to segment product pages and blog content on results page Doesn't cause any major SEO or technical issues Understands semantic search Ability to filter results Ability to sort (e.g. by price, popularity, new in stock) It'd be really useful to see examples and know if there are any particular issues to be aware of. Thanks. 🙂
Intermediate & Advanced SEO | | Alex-Harford0 -
How to fix issues from 301s
Case: We are currently in the middle of a site migration from .asp to .net and Endeca PageBuilder, and from a homebrewed search provider to Endeca Search. We have migrated most of our primary landing pages and our entire e-commerce site to the new platforms. During the transition approximately 100 of our primary landing pages were inadvertently 302ed to the new version. Once this was caught they were immediately changed to 301s and submitted to the Google’s index through webmaster tools. We initially saw increases in visits to the new pages, but currently (approximately 3 weeks after the change from 301 to 302) are experiencing a significant decline in visits. Issue: My assumption is many of the internal links (from pages which are now 301ed as well) to these primary landing pages are still pointing to the old version of the primary landing page in Google’s cache, and thus have not passed the importance and internal juice to the new versions. There are no navigational links or entry points to the old supporting pages left, and I believe this is what is driving the decline. Proposed resolution: I intend to create a series of HTML sitemaps of the old version (.asp) of all pages which have recently been 301ed. I will then submit these pages to Google’s index (not as sitemaps, just normal pages) with the selection to index all linked pages. My intention is to force Google to pick up all of the 301s, thus enforcing the authority channels we have set up. Question 1: Is the assumption that the decline could be because of missed authority signals reasonable? Question 2: Could the proposed solution be harmful? Question 3: Will the proposed solution be adequate to resolve the issue? Any help would be sincerely appreciated. Thank you in advance, David
Intermediate & Advanced SEO | | FireMountainGems0 -
HTML5 Nav Tag Issue - Be Aware
In checking my internal links with GWT, it is apparent that links within the nav tag in HTML5 are discounted by Google as "internal links" This could have major repercussions for designing your internal link structure for SEO purposes. I was surprised to see this result, as I have never seen it discussed. Anyone else notice this, or have any alternative views?
Intermediate & Advanced SEO | | veezer0 -
MOZ crawl report says category pages blocked by meta robots but theyr'e not?
I've just run a SEOMOZ crawl report and it tells me that the category pages on my site such as http://www.top-10-dating-reviews.com/category/online-dating/ are blocked by meta robots and have the meta robots tag noindex,follow. This was the case a couple of days ago as I run wordpress and am using the SEO Category updater plugin. By default it appears it makes categories noindex, follow. Therefore I edited the plugin so that the default was index, follow as I want google to index the category pages so that I can build links to them. When I open the page in a browser and view source the tags show as index, follow which adds up. Why then is the SEOMOZ report telling me they are still noindex,follow? Presumably the crawl is in real time and should pick up the new follow tag or is it perhaps because its using data from an old crawl? As yet these pages aren't indexed by google. Any help is much appreciated! Thanks Sam.
Intermediate & Advanced SEO | | SamCUK0