Handling long URLs and overly-dynamic URLs on eCommerce site
-
Hello Forum,
I've been optimizing an eCommerce site and our SEOmoz crawls are favorable for the most part, except for long URLs and overly-dynamic URLs. These issues stem from two URL types: Layered navigation (faceted search) and non-Google internal search results. I outline the issues for each below.
We use an SEO-friendly URL structure for our product category pages, but once bots start "clicking" our layered navigation options, all the parameters are appended to our SEO-friendly urls, causing the SEOmoz crawl warnings.
Layered Navigation :
SEO-Friendly Category Page: oursite.com/shop/meditation-cushions.htmlEffects of layered navigation: oursite.com/shop/meditation-cushions.html?bolster_material_quality=414&bolsters_appearance=206&color=12&dir=asc&height=291&order=name
As you can see the parameters include product attributes and page sorts. I should note that all pages generated by these parameters use the element to point back to the SEO-friendly URL We have also set up Google's Webmaster Tools to handle these parameters.
Internal Search Function:
Our URLs start off simple: oursite.com/catalogsearch/result/?q=brown. Then the bot clicks all the layered navigation options, yielding oursite.com/catalogsearch/result/index/?appearance=54&cat=67&clothing_material=83&color=12&product_color=559&q=brown. Also, all search results are set to noindex,follow.My question is: Should we worry about these overly-dynamic and long ULR warnings? We have set up canonical elements, "noindex,follow" solutions, and configured Webmaster Tools to handle our parameters. If these are a concern, how would you resolve these issues?
-
I see this thread was from last year, so I am hoping between then and now you have determined an answer and would be able to advise. I am having the same issue with our consumer sight.
-
If you make them friendly it will shorten them
x=y can become y
But having done that and they are still too long i would ignore them as they are no-index.
-
There another company handling the server side of things. All I know is that we're using PHP and MySQL for Magento.
Even if we did a friendly URL rewrite, wouldn't we still get long URLs? We would just have each parameter become words separated by slashed. i.e .
/shop/meditation-cushions.html/high quaily/patterened/green/10inches/sortedbyname/
I suppose these URLs shorter. Is something like this better?
-
Marc
The crawl warnings are those found in SEOmoz's crawl diagnostics: "Overly-Dynamic URL" and "Long URL." These are not duplicate content issues and the URLs resolve properly.
I just want to make sure we're not getting dinged for having URLs that are too long. If we are, what are some way to go about shortening them?
-Aaron
-
What kind of "crawl warnings" are we talking about here? Duplicate content? Do the URL's resolve properly when the additional parameters are appending to the SEO-friendly URL's?
"I should note that all pages generated by these parameters use the element to point back to the SEO-friendly URL We have also set up Google's Webmaster Tools to handle these parameters."
Keep in mind, using canonical tags is like setting up 301 redirects on all those pages. Some people don't now that, so I thought I'd just throw it out there. So, if any of those additional pages with the host of parameters contain unique/different content than the seo-friendly versions, using canonical tags is not a good move as they will get no attention from search engines that respect the canonical tag.
For example, do not use a canonical tag on a 'Page 2' to point back to page 1. Each page will contain different information/products/whatever, and you want search engines (SE) to see and index those pages, regardless of what the URL looks like (as long as it works and your Title/META/H1-H6 tags are all in order to reflect the different content on each page.
I'm not sure I'm following your concern 100% percent, so I hope I was on the right path with what I said. Can you please be more specific as to what you concern is with the "overly-dynamic and long ULR warnings" please, and I'll be happy to help you out some more.
- Marc
-
The easy fix is the canonical, yet Bing suggest not using the canonical on the true page, only the duplicates. Best if you can handle that in code, but not a big worry of you cant.
Facet naviagtion is a big problem, with no easy answers.
What sort of server are you using, on a windows server it is very easy to set up friendly urls for your dynamic urls.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Our crawler was not able to access the robots.txt file on your site.
Good morning, Yesterday, Moz gave me an error that is wasn't able to find our robots.txt file. However, this is a new occurrence, we've used Moz and its crawling ability many times prior; not sure why the error is happening now. I validated that the redirects and our robots page are operational and nothing is disallowing Roger in our robots.txt. Any advice or guidance would be much appreciated. https://www.agrisupply.com/robots.txt Thank you for your time. -Danny
Moz Pro | | Danny_Gallagher0 -
Block Moz (or any other robot) from crawling pages with specific URLs
Hello! Moz reports that my site has around 380 duplicate page content. Most of them come from dynamic generated URLs that have some specific parameters. I have sorted this out for Google in webmaster tools (the new Google Search Console) by blocking the pages with these parameters. However, Moz is still reporting the same amount of duplicate content pages and, to stop it, I know I must use robots.txt. The trick is that, I don't want to block every page, but just the pages with specific parameters. I want to do this because among these 380 pages there are some other pages with no parameters (or different parameters) that I need to take care of. Basically, I need to clean this list to be able to use the feature properly in the future. I have read through Moz forums and found a few topics related to this, but there is no clear answer on how to block only pages with specific URLs. Therefore, I have done my research and come up with these lines for robots.txt: User-agent: dotbot
Moz Pro | | Blacktie
Disallow: /*numberOfStars=0 User-agent: rogerbot
Disallow: /*numberOfStars=0 My questions: 1. Are the above lines correct and would block Moz (dotbot and rogerbot) from crawling only pages that have numberOfStars=0 parameter in their URLs, leaving other pages intact? 2. Do I need to have an empty line between the two groups? (I mean between "Disallow: /*numberOfStars=0" and "User-agent: rogerbot")? (or does it even matter?) I think this would help many people as there is no clear answer on how to block crawling only pages with specific URLs. Moreover, this should be valid for any robot out there. Thank you for your help!0 -
Links from Dofollow sites are Nofollow ?
Hii i have made few links from dofollow site through blog commenting method and when i checked my site on Open Site Explorer it shows Nofollow . So my question is links from Dofollow blogs will be nofollow ? There is any benefit of getting links from dofollow site through comments ? Thanks
Moz Pro | | sumit40 -
Why would only 1 pg of an 18 pg site be crawled?
I signed up yesterday. Added 4 sites. The one I really need data on now has only had 1 page crawled. The other sites have had almost all pages crawled (over 40 each) in one day. What is wrong? The main domain has a 301 redirect to another domain name is that the problem? Is there something wrong at Google analytics? I dont manage this site and I'm picking up behind another firm..where should I start my discovery? Thanks so much!
Moz Pro | | moreidea0 -
How to Link build for our site
Hello, A client of mine has a site that gives advice to people. It's moderately competitive. It's a content site. Using Open Site explorer, I looked up competitor's top pages and backlinks, and there were things such as sites that were heavy on directories, sites heavily on editorial mension, sites on juicy topics that I don't want to cover, and the rest was pretty much junk. Just so that you know, I only link build with quality websites that look strong, long term, have a nice design, are high quality, and are not a directory (with a few exceptions) What's the standard way to link build with a site like this? We've got 7 articles of good content that is from me spending hours learning about the subject and using my skills as a life coach. Right now I'm just adding quality content and social media. I am not a top expert in the field, but I'm learning as much as possible. How do we link build?
Moz Pro | | BobGW0 -
Site explore reporting error over week
unable to dispaly anchor text error Doh! Roger is still working out the kinks with the new index and is having issues untangling anchor text data. We're currently showing anchor text data from the previous index, but we will update as soon as we can.
Moz Pro | | 1step2heaven120 -
HTTPS site in Open Site Explorer
I'm looking at a site for which the https URL currently ranks in Google. Using a header checker on the http URL I see that it is being 302 redirected to the https version (I have no control or input on this site). In OSE there's no option to specify an https URL as the http part is pre-populated and uneditable. My question is: does OSE treat the https and http version as the same URL? I'm guessing so as the http URL has a lot of domain authority despite not being the "default" URL.
Moz Pro | | Equatorites0 -
How to get seomoz to re-crawl a site?
I had a lot of duplicate content issues and have fixed all the other warnings. I want to check the site again.
Moz Pro | | adamzski0