URL Parameters
-
Hi Moz Community,
I'm working on a website that has URL parameters. After crawling the site, I've implemented canonical tags to all these URLs to prevent them from getting indexed by Google. However, today I've found out that Google has indexed plenty of URL parameters..
1-Some of these URLs has canonical tags yet they are still indexed and live.
2- Some can't be discovered through site crawling and they are result in 5xx server error.
Is there anything else that I can do (other than adding canonical tags) + how can I discover URL parameters indexed but not visible through site crawling?
Thanks in advance!
-
I'm also facing the same problem with my website pages. My Blackpods pro website pages don't show the exact permalink urls.
-
Hi there,
Thanks very much for your response. I checked the sitemap and there are no URL parameters listed - only the canonical URL listed on the sitemap.
If you have any other suggestions it'll be much appreciated.
Thank you!
-
Hi Rajesh,
Thank you for your response. I cannot share the website due to client's confidentiality but basically when I search to find a stockist {brand name}, Google lists similar URLs below on the first page. The pages are showing a list of stockists depending on the product availability:
1-website.com/find-stockist?model=10 (5xx status code)
2-website.com/find-stockist?model=11 (200 status code)
3-website.com/find-stockist?model=10 (5xx status code)
4-website.com/find-stockist?model=11 (200 status code)Thank you!
-
Hi Gaston,
Thanks very much for your time. The canonicals have implemented around a month ago and the pages are almost identical. I discovered all URL parameters without performing an advanced search.
Also, I come across the 5xx errors when I clicked indexed URL parameters on Google SERP and I cannot discover them when I crawl the site with Screaming Frog.
I'd appreciate if you have any other suggestions based on your experience!
Many thanks
-
Just so you know, if a URL results in a 5XX server error then it usually won't render your canonical tag to begin with! You might want to check your sitemap XML, to check that it's not 'undoing' your canonical tags by feeding these URLs to Google. Indexation tags must be perfectly aligned with your sitemap XML, or you are sending Google mixed messages (e.g: a URL is in sitemap XML so Google should index it, but when it is crawled it contains a canonical tag citing itself as non-canonical, which is the opposite signal)
Everything which Gaston said is right on the money
-
I think you need to show some examples.
-
Hi there,
Its important to note that canonicals are a signal. Google can obey them if its algorithm considers that those pages are actually canonicals between each other.
In my experience, this does not happen immediately, it usually takes Google some time to figure out if the canonicalization is correct. Keep in mind that pages being canonicalized HAVE TO be nearly identical and refer to the same topic.
And on the indexation part, pages can be indexed and be shown only when you search for that specific URL or using any advanced search parameter (such as site:).
More information about canonicals
- Consolidate duplicate URLs - Google Search supportRegarding the second issue, if you refer to "site crawling" as what you do with an external tool, such as Screaming Frog or Moz, you are getting 5xx errors because that tool is making to many requests, try lowering its crawl frequency. I know for a fact that Screaming Frog allows you to do that.
But, unfortunately, I don't know any other way of discovering URL parameters in bulk but using an external tool.Hope it helps,
Best luck.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Submitting URLs After New Search Console
Hi Everyone I wanted to see how people submit their urls to Google and ensure they are all being indexed. I currently have an ecommerce site with 18,000 products. I have sitemaps setup, but noticed that the various product pages haven't started ranking yet. If I submit the individual url through the new Google Search Console I see the page ranking in a matter of minutes. Before the new Google Search Console you could just ask Google to Fetch/Render an XML sitemap and ask it to crawl all the links. I don't see the same functionality working today on Google Search Console and was wondering if there are any new techniques people could share. Thanks,
Intermediate & Advanced SEO | | abiondo
Anthony1 -
How to switch from URL based navigation to Ajax, 1000's of URLs gone
Hi everyone, We have thousands of urls generated by numerous products filters on our ecommerce site, eg./category1/category11/brand/color-red/size-xl+xxl/price-cheap/in-stock/. We are thinking of moving these filters to ajax in order to offer a better user experience and get rid of these useless urls. In your opinion, what is the best way to deal with this huge move ? leave the existing URLs respond as before : as they will disappear from our sitemap (they won't be linked anymore), I imagine robots will someday consider them as obsolete ? redirect permanent (301) to the closest existing url mark them as gone (4xx) I'd vote for option 2. Bots will suddenly see thousands of 301, but this is reflecting what is really happening, right ? Do you think this could result in some penalty ? Thank you very much for your help. Jeremy
Intermediate & Advanced SEO | | JeremyICC0 -
Image URLs - best practice
Hi - I'm assuming image URL best practice follows same principles as non image URLs (not too many files and so on) - I notice alot of web devs putting photos in subdomains, so wonder if I'm missing something (I usually avoid subdomains like the plague)!
Intermediate & Advanced SEO | | McTaggart1 -
Changing the spellings of titles and URl changes
Hi, Changing the spellings of titles and URl changes We identifies 500+ titles with some issues like spellings and punctuations and short or too long. We want to change them, but the titles are connected with the URL's when we change the titles the URl's change as well. My questions are 1. Is it a good way to change them all in one shot or do few daily 2. As the URl's change will Google index drop the old pages as they would be 404 and index new ones? 3. Will we have chances to have drop in traffic due to this? 4. Any way to redirect? as we have a Drupal website Thanks
Intermediate & Advanced SEO | | mtthompsons0 -
Correct URL Parameters for GWT?
Hi, I am just double checking to see if these parameters are ok - I have added an attachment to this post. We are using an e-commerce store and dealing with faceted navigation so I excluded a lot of parameters from being crawled as I didnt want them indexed. (they got indexed anyway!). Advice and recommendations on the use of GWT would be very helpful - please check my screenshot. thanks, B0gSmRu
Intermediate & Advanced SEO | | bjs20100 -
Query parameters for normalization in Bing
Almost everyday I get this- Query parameters for normalization found on www.sitename.com Site: www.sitename.comDate: 3/26/2013Priority: LowBing has detected new parameters in your URLsAnyone know why? We aren't changing anything. I have read it has to do with internal urls but I can find out what internal urls this is a problem with.
Intermediate & Advanced SEO | | EcommerceSite0 -
Linking to Short vs Long URL
Suppose I have a long url on an established site and created a shorter version of it so it is easier for people to enter directly and click. I 301 the short version to the long. I don't think there is much concern for people linking to the long version pages, but will there be a tendency for people to link to the short url instead of the long for the domain links? Will I not benefit as much from links to the short vs the long? Thanks.
Intermediate & Advanced SEO | | AWCthreads0 -
How to get better URL description when ranking #3
OK, last question of the week. I promise. I'm doing something right. I distribute for manufacturers. For some of my less known manufactuers I am #3 or #2 rank. For my main product the manufacturer is #1,2 with some numerical code as the discription. The manufacture will sell direct. To get the customers attention I need to PPC, and luckily it's inexpensive. Is there any way to control what the content is of the listing?. I would like to state in my #3 rank " 10-10-PP, In stock, same day shipping, best pricing." Does Google choose what to display and if so is there some where on my site I can influece this? For you experts, perhaps this is the joke of the week.? Please do not have a heart attack when laughing.
Intermediate & Advanced SEO | | Wales0