Extreme high number of pages found on webshop
-
Hi,
Im working for the first time on a magento webshop. But i run into a problem where crawlers find then thousands of pages while there are a few hunderd products.
I expect is has something to do with filters that generate dynamic URL's. I can't find any setting in Magento to prevent this and i think this will hurt SEO performance because of duplicate content and high amount of pages that need to be crawled while the site has no authority.
What would my approach be to solve this? Do i need to ad certain tags to the pages or are these settings in my robots file.
-
Answers above is spot on,
A comprehensive robots file should help with this, if your in a hurry to clean the indexation up, you a can remove URLs witihin the search console.
One other thing to consider is, your xml sitemap. look at this in detail & what your asking the search engine to crawl. End of life products, legacy catalogues should not be included.
-
This could possibly be because of the different filters and combinations you have you your category and sub-category pages. I would recommend doing the following :
-
Block access to unwanted pages through robots.txt
-
Setup the canonicals properly so that all different types of filtering combinations will always point to the main category/sub-category pages
-
Make sure your Pagination is properly setup with each page having its canonical pointing to itself and the "rel=next/prev" included (although Google has mentioned that it has stopped supporting those, I would still recommend implementing them)
Here's a good guide on Magento SEO:
https://paulnrogers.com/magento-technical-seo-issues-to-look-out-for/
Hope that helps!
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What to do with old conversion pages
Hey folks! I have a ton of old conversion pages from past trade shows, old webinars, etc that are either getting no traffic or very little. Wondering if I should just 404 them out? Here's an example: http://marketing.avidxchange.com/rent-manager-user-conference-demo-request-2015 For the pages getting traffic (from PPC, referral links, organic) my presumption is to keep those. The only problem is we have multiple instances of the same asset (prior marketers would just clone them for different campaigns), so in those cases should I 301 them to one version? Looking for advice on best practices here for future instances. Such as future trade shows, after we use the conversion pages at an event, should I just delete/404 them? Cleaning up old pages should I just delete/404? They don't have any value really and they're annoying to have hanging around. Thanks!
Technical SEO | | Bill_King0 -
How do I fix issue regarding near duplicate pages on website associated to city OR local pages?
I am working on one e-commerce website where we have added 300+ pages to target different local cities in USA. We have added quite different paragraphs on 100+ pages to remove internal duplicate issue and save our website from Panda penalty. You can visit following page to know more about it. And, We have added unique paragraphs on few pages. But, I have big concerns with other elements which are available on page like Banner Gallery, Front Banner, Tool and few other attributes which are commonly available on each pages exclude 4 to 5 sentence paragraph. I have compiled one XML sitemap with all local pages and submitted to Google webmaster tools since 1st June 2013. But, I can see only 1 indexed page by Google on Google webmaster tools. http://www.bannerbuzz.com/local http://www.bannerbuzz.com/local/US/Alabama/Vinyl-Banners http://www.bannerbuzz.com/local/MO/Kansas-City/Vinyl-Banners and so on... Can anyone suggest me best solution for it?
Technical SEO | | CommercePundit0 -
Help! Pages not being indexed
Hi Mozzers, I need your help.
Technical SEO | | bshanahan
Our website (www.barnettcapitaladvisors.com) stopped being indexed in search engines following a round of major changes to URLs and content. There were a number of dead links for a few days before 301 redirects were properly put in place. And now, only 3 pages show up in bing when I do the search "site:barnettcapitaladvisors.com". A bunch of pages show up in Google for that search, but they're not any of the pages we want to show up. Our home page and most important services pages are nowhere in search results. What's going on here?
Our sitemap is at http://www.barnettcapitaladvisors.com/sites/default/files/users/AndrewCarrillo/sitemap/sitemap.xml
Robots.txt is at: http://www.barnettcapitaladvisors.com/robots.txt Thanks!0 -
Page not indexed but still has a PageRank, how?
http://www.optiproerp.com/products.aspx page is not indexed in Google but still has a PageRank of 1. How? Regards
Technical SEO | | IM_Learner0 -
SEOMoz Crawl Diagnostic indicates duplicate page content for home page?
My first SEOMoz Crawl Diagnostic report for my website indicates duplicate page content for my home page. It lists the home page URL Page Title and URL twice. How do I go about diagnosing this? Is the problem related to the following code that is in my .htaccess file? (The purpose of the code was to redirect any non "www" backlink referrals to the "www" version of the domain.) RewriteCond %{HTTP_HOST} ^whatever.com [NC]
Technical SEO | | Linesides
RewriteRule ^(.*)$ http://www.whatever.com/$1 [L,R=301] Should I get rid of the "http" reference in the second line? Related to this is a notice in the "Crawl Notices Found" -- "301 Permanent redirect" which shows my home page title as "http://whatever.com" and shows the redirect address as http://http://www.whatever.com/ I'm guessing this problem is again related to the redirect code I'm using. Also... The report indicates duplicate content for those links that have different parameters added to the URL i.e. http://www.whatever.com?marker=Blah Blah&markerzoom=13 If I set up a canonical reference for the page, will this fix this? Thank you.0 -
High number of Duplicate Page titles and Content related to index.php
It appears that every page on our site (www.bridgewinners.com) also creates a version of itself with a suffix. This results in Seomoz indicating that there are thousands of duplicate titles and content. 1. Does this matter? If so, how much? 2. How do I eliminate this (we are using joomla)? Thanks.
Technical SEO | | jfeld2220 -
Can I use canonical tags to merge property map pages and availability pages to their counterpart overview pages?
I have a property website, for each property are 4-5 tabs each with their own URL, these pages include the overview page which is content rich, and auxilliary pages such as maps, availability, can I use a canonical tag to merge the tabs with very little content to their corresponding overview page which is content rich? I.e. www.mywebsite.co.uk/property-1/overview This page has tabs for map, town info, availability which all have their own url i.e. www.mywebsite.co.uk/property-1/map
Technical SEO | | assertive-media
www.mywebsite.co.uk/property-1/availability
www.mywebsite.co.uk/property-1/towninfo Because these auxilary pages do not contain much content can I place a canonical tag in them pointing back to the content rich overview page at www.mywebsite.co.uk/property-1/overview?0 -
What Are The Page Linking Options?
Okay, so I'm working with a site that has pretty good domain authority, but the interior pages are not well linked or optimized. So, it ranks for some terms, but everything goes to the home page. So, I'd like to increase the authority of the interior pages. The client is not wild about spreading targeted juice via something like a footer. They also don't like a "Popular Searches" style link list. The objection is that it's not attractive. They currently use cute euphemisms as the linking text, so like "cool stuff" instead of "sheepskin seat covers." In that made up example, they'd like to rank for the latter, but insist on using the former. What about a slide show with alt text/links? Would that increase the authority of those pages in a term-targeted kinda way? Are there other options? Does it matter how prominent those links are, like footers not as good as something higher up the page? They currently use a pull-down kind of thing that still results in some pages having no authority. Do bots use that equally well? Thanks!
Technical SEO | | 945010