Removing duplicate &var=1 etc var name urls from google
-
Hi I had a huge drop in traffic around the 11th of july over 50% down with no recovery as yet... ~5000 organic visits per day down to barley over 2500.
I fixed up a problem that one script was introducing that had caused high bounce rates.
Now i have identified that google has indexed the entire news section 4 times, same content but with var=0 var=1 2 3 etc around 40,000 urls in total.
Now this would have to be causing problems.
I have fixed the problem and those url's 404 now, no need for 301's as they are not linked to from anywhere.
How can I get them out of the index? I cant do it one by one with the url removal request.. I cant remove a directory from url removal tool as the reuglar content is still there..
If I ban it in robots.txt those urls, wont it never try to index them again and thus not ever discover they are 404ing?
These urls are no longer linked to from anywhere, so how can google ever reach them by crawling to find them 404ing?
-
yes
-
Hi thanks, so if it cant find a page and finds no more links to a page. does that mean that it should drop out of the index within a month?
-
The definition of a 404 page is a page which cannot be found. So in that sense, no Google can't find the page.
Google's crawlers follow links. If there is not a link to the page, then there is no issue. If Google locates a link, they will attempt to follow that link.
-
Hi Thanks, so if a page is 404'ing but not linked to from anywhere google will still find it?
-
Hi Adam.
The preferred method to handle this issue would have been to only offer one version of the URL. Once you realized the other versions were active, you have a couple options to deal with the problem:
Use a 301 to redirect all the versions of the page to the main URL. This method would have allowed your existing Google links to work. Users would still find the correct page. Google would have noticed the 301 and adjusted their links.
Another option to consider IF the pages were helpful would be to keep them and use the canonical tag to indicate the URL of the primary page. This method would offer the same advantages mentioned above.
By removing the pages and allowing them to 404, everyone loses for the next month. Users who click on a search result will be taken to a 404 page rather then finding the content they seek. Google wont be offering the search results users are seeking. You will experience a high bounce rate as many users do not like 404 pages, and it will take a month for an average site to be fully crawled and the issue corrected.
If you block the pages in robots.txt, then Google wont attempt to crawl the links. In general, your robots.txt should not be used in this manner.
My recommendation is to fix this issue either with the proper 301s. If that is not an option, be sure your 404 page is helpful and as user friendly as possible. Include a site search option along with your main navigation. Google will crawl a small percent of your site each day. You will notice the number of 404 links diminish over time.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to do A/B testing without creating two separate url like google analytic experiment?
Hello Experts, I want to do A/B testing for my page. In google analytic experiment we have to create two pages 1) Original Page 2) Variant 1 but I don't want to go in this method that is I donot want to create two pages is it possible only via one page but two different events or something else ? If yes then which is the best tool? Thanks! Wrigths!
Technical SEO | | wright3350 -
Does Google think My main keyword is my Company name due to over use in anchor text?
OK, i have recently noticed google is adding a keyword to my titles... I have attached an image which shows the results from a google site:mywebsite search... 8 out of the 10 results on the screen shot clearly have the keyword ( - CCTV Cameras) added to the end of the title. Now a lot of those pages are support pages i.e "Help & Support", "returns" etc etc.. nothing really to do with cctv cameras. The only thing i can think of, is the most common anchor text used on backlinks to my website (according to siteexplorer) is the keyword cctv cameras... (i am currently working on removing a load of the backlinks to my site due to what i think is some kind of google penalty as the anchor text is all keywords etc...) But it seems strange if google has started to discount these backlinks, that it then uses that keyword to add to my titles - unless of course, google thinks that it is my brand name... Any body have any thoughts on this? FqwooQY.gif
Technical SEO | | isntworkdull1 -
Should we use & or and in our url's?
Example: /Zambia/kasanka-&-bangweulu or /Zambia/kasanka-and-bangweulu which is the better url from the search engines point of view?
Technical SEO | | tribes0 -
Google Places & Multiple Accounts?
As an agency that manages multiple accounts should I have all my Google Place accounts under one account or should I create a separate account for each client with a unique username and password for every client? Thanks,
Technical SEO | | fun52dig
Gary Downey0 -
For Google + purposes, should the author's name appear in the Meta description or title tag of my web site just as you would your key search phrase?
Relative to Cyrus Shepard's article on January 4th regarding Google's Superior SEO strategy, if I'm the primary author of all blog articles and web site content, and I have a link showing authorship going back to Google Plus, is a site wide link from the home page enough or should that show up on all blog posts etc and editorial comment pages etc? Conversely, should the author's name appear in the Meta description or title tag of my web site just as you would your key search phrase since Google appears to be trying to make a solid connection with my name, and all content?
Technical SEO | | lwnickens0 -
UK website ranking higher in Google.com than Google.co.uk
Hi, I have a UK website which was formerly ranked 1<sup>st</sup> in Google.co.uk and .com for my keyword phrase and has recently slipped to 6<sup>th</sup> in .co.uk but is higher in position 4 in Google.com. I have conducted a little research and can’t say for certain but I wonder if it is possible that too many of my backlinks are US based and therefore Google thinks my website is also US based. Checked Google WmT and we the geo-targeted to the UK. Our server is also UK based. Does anyone have an opinion on this? Thanks
Technical SEO | | tdsnet0 -
Almost Duplicate Product names (READ almost!)
Hi Mozzers, Here is a dilemma I thought I'd put forward and see how you guys would handle the situations. I'm working on a comparison website on which many of of the product names are very similar. There are circa 2000 products, of which many have similar names. Here are a few of the product names for example; Airsprung Enigma 5ft Mattress Airsprung Double Echo 6ft Mattress Silentnight Double Nova 6ft Mattress The Product name has been used in the page title of the product pages as show below; Compare Prices on the Airsprung Double Echo 6ft Mattress SEOmoz is highlighting these pages as having Duplicate Page Titles (and meta descriptions as a similar approach has been used). How would you handle this situation? Cheers Aran
Technical SEO | | Entrusteddev0 -
Google Webmaster Tools reports 404s for plain text. Should I create those URLs and 301 them to actual pages?
IA few years back I noticed that Google Webmaster Tools returns 404s from regular text containing a URL, but no anchor tag. I came accross this again today. Is it worthwhile to create those URLs and 301 redirect them to proper pages.
Technical SEO | | Svetoslav0