Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Have Your Thoughts Changed Regarding Canonical Tag Best Practice for Pagination? - Google Ignoring rel= Next/Prev Tagging
Hi there, We have a good-sized eCommerce client that is gearing up for a relaunch. At this point, the staging site follows the previous best practice for pagination (self-referencing canonical tags on each page; rel=next & prev tags referencing the last and next page within the category). Knowing that Google does not support rel=next/prev tags, does that change your thoughts for how to set up canonical tags within a paginated product category? We have some categories that have 500-600 products so creating and canonicalizing to a 'view all' page is not ideal for us. That leaves us with the following options (feel it is worth noting that we are leaving rel=next / prev tags in place): Leave canonical tags as-is, page 2 of the product category will have a canonical tag referencing ?page=2 URL Reference Page 1 of product category on all pages within the category series, page 2 of product category would have canonical tag referencing page 1 (/category/) - this is admittedly what I am leaning toward. Any and all thoughts are appreciated! If this were in relation to an existing website that is not experiencing indexing issues, I wouldn't worry about these. Given we are launching a new site, now is the time to make such a change. Thank you! Joe
Web Design | | Joe_Stoffel1 -
When rel canonical tag used, which page does Google considers for ranking and indexing? A/B test scenario!
Hi Moz community, We have redesigned our website and launched for A/B testing using canonical tags from old website to new website pages, so there will be no duplicate content issues and new website will be shown to the half of the website visitors successfully to calculate the metrics. However I wonder how actually Google considers it? Which pages Google will crawl and index to consider for ranking? Please share your views on this for better optimisation. Thanks
Web Design | | vtmoz0 -
Regarding rel=canonical on duplicate pages on a shopping site... some direction, please.
Good morning, Moz community: My name is David, and I'm currently doing internet marketing for an online retailer of marine accessories. While many product pages and descriptions are unique, there are some that have the descriptions duplicated across many products. The advice commonly given is to leave one page as is / crawlable (probably best for one that is already ranking/indexed), and use rel=canonical on all duplicates. Any idea for direction on this? Do you think it is necessary? It will be a massive task. (also, one of the products that we rank highest for, we have tons of duplicate descriptions.... so... that is sort of like evidence against the idea?) Thanks!
Web Design | | DavidCiti0 -
Post vs Pages
Does Google make any distinction between a web page and a blog post? Assuming all else is equal...any reason why a page would rank higher than a post? And that includes a page in WordPress vs a WordPress blog post.
Web Design | | Pinlaser1 -
One big page vs. multi-step pages
Hi mozers! Brand new to SEO and LOVING it! Having several key questions that I don't see answered yet, but I'll start with one we've been very curious about. Consider this guide we have for Forming a Delaware Corp.
Web Design | | Mase
https://www.upcounsel.com/Free-Legal/Guide/17/Form-A-Delaware-Corporation This is our overview page, giving you a breakdown of what this process involves. We love this page, but (Question1:) does it lack better real "content" rather than lots of links to the guide process itself? Then, you can start to walk through the guide beginning with step one, where each step has crowd sourced answers to it. But as you see, the step pages are all very similar, except for the answers and step info. (Question 2) Would it be better to put all our answers into the one overview page and skip having separate pages for each step? We like the process and simplicity of seeing one step at a time, but then these pages don't seem to have enough unique content on them. Related, at what point (if any) is a page too big with too much content and considered bad for SEO? We're recovering from a big hit from Google, and slowly recovering by nailing down various SEO mistakes. We DO have great, unique and valueable content - now we just need it to rank!0 -
3rd party commenting systems vs native?
I'm curious what you all think about using a 3rd party commenting system (like Disqus) vs the native wordpress commenting system? I've read so many reviews online it makes my head spin, so I wanted to see if any of you have any experience, or perhaps some trusted case studies. I was using the native comment system for a while, and then tried out Disqus; which seems to be good, but I'm not sure if people know how to easily get notified of new comments. With the native system there was a check box that said "subscribe," plus I used a plugin to redirect a first time commenter to a welcome page, as well as sent an email to them. I feel like Disqus makes it harder for people to get notified on new comments. However, I like giving people the ability to log in via different 3rd party channels (facebook/twitter/disqus, etc.) I know there are some 3rd party tools that allow you to do this on the native comment system as well. Any way, I'm just curious if anyone had any experience. Also I'm assuming the audience makes a big difference. My target readership is mommy's and parents, and not necessarily a 'tech" one; so I want to make the ability to comment very simple and easy for them. That's key. I'm sort of leaning on moving back over to the Wordpress comment system.
Web Design | | NoahsDad0 -
Are slimmed down mobile versions of a canonical page considered cloaking?
We are developing our mobile site right now and we are using a user agent sniffer to figure out what kind of device the visitor is using. Once the server knows whether it is a desktop or mobile browser it will deliver the appropriate template. We decided to use the same URL for both versions of the page rather than using m.websiteurl.com or www.websiteurl.mobi so that traffic to either version of these pages would register as a visit to the page. Will search engines consider this cloaking or is mobile "versioning" an acceptable practice? The pages in essence are the same, the mobile version will just leave out extraneous scripts and unnecessary resources to better display on a mobile device.
Web Design | | TahoeMountain400 -
Why is site not being indexed by Google, and not showing on a crawl test??
On a site we developed of which .com is forwarded to .net domain, we quit getting crawled by google on about the 20th of Feb. Now when we try to run a crawl test on either url, we get There was an error fetching this page. Error description For some reason the page returned did not describe itself as an html page. It could be possible that the url is serving an image, rss feed, pdf, or xml file of some sort. The crawl tool does not currently report metrics on this type of data. Our other sites are fine and this was up to this date. We took out noodp, noydir today as the only thing we could think of. Site is on WP cms.
Web Design | | RobertFisher0