How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google ranking content for phrases that don't exist on-page
I am experiencing an issue with negative keywords, but the “negative” keyword in question isn’t truly negative and is required within the content – the problem is that Google is ranking pages for inaccurate phrases that don’t exist on the page. To explain, this product page (as one of many examples) - https://www.scamblermusic.com/albums/royalty-free-rock-music/ - is optimised for “Royalty free rock music” and it gets a Moz grade of 100. “Royalty free” is the most accurate description of the music (I optimised for “royalty free” instead of “royalty-free” (including a hyphen) because of improved search volume), and there is just one reference to the term “copyrighted” towards the foot of the page – this term is relevant because I need to make the point that the music is licensed, not sold, and the licensee pays for the right to use the music but does not own it (as it remains copyrighted). It turns out however that I appear to need to treat “copyrighted” almost as a negative term because Google isn’t accurately ranking the content. Despite excellent optimisation for “Royalty free rock music” and only one single reference of “copyrighted” within the copy, I am seeing this page (and other album genres) wrongly rank for the following search terms: “free rock music”
On-Page Optimization | | JCN-SBWD
“Copyright free rock music"
“Uncopyrighted rock music”
“Non copyrighted rock music” I understand that pages might rank for “free rock music” because it is part of the “Royalty free rock music” optimisation, what I can’t get my head around is why the page (and similar product pages) are ranking for “Copyright free”, “Uncopyrighted music” and “Non copyrighted music”. “Uncopyrighted” and “Non copyrighted” don’t exist anywhere within the copy or source code – why would Google consider it helpful to rank a page for a search term that doesn’t exist as a complete phrase within the content? By the same logic the page should also wrongly rank for “Skylark rock music” or “Pretzel rock music” as the words “Skylark” and “Pretzel” also feature just once within the content and therefore should generate completely inaccurate results too. To me this demonstrates just how poor Google is when it comes to understanding relevant content and optimization - it's taking part of an optimized term and combining it with just one other single-use word and then inappropriately ranking the page for that completely made up phrase. It’s one thing to misinterpret one reference of the term “copyrighted” and something else entirely to rank a page for completely made up terms such as “Uncopyrighted” and “Non copyrighted”. It almost makes me think that I’ve got a better chance of accurately ranking content if I buy a goat, shove a cigar up its backside, and sacrifice it in the name of the great god Google! Any advice (about wrongly attributed negative keywords, not goat sacrifice ) would be most welcome.0 -
How to optimize my website with new Google Feature snippet fuctionality?
Recently I have interacted with Google's new Feature Snippet Functionality. Please refer to the link shared. https://www.google.com/search?q=family+health+insurance&rlz=1C1CHBD_enIN893IN893&oq=family+health+insurance&aqs=chrome.0.35i39l2j0l3j69i60l3.4692j0j7&sourceid=chrome&ie=UTF-8 I'am looking for the way to optimize my website for the same? If anyone has any clue then please help me with this. search?q=family+health+insurance&rlz=1C1CHBD_enIN893IN893&oq=family+health+insurance&aqs=chrome.0.35i39l2j0l3j69i60l3.4692j0j7&sourceid=chrome&ie=UTF-8
On-Page Optimization | | adlift0 -
How do you find out all the keywords Google is ranking you for?
Hello, Is there anyway of finding out all the keywords that Google is currently ranking our website for so that we can then build on those keyword positions? Many thanks,
On-Page Optimization | | mblsolutions0 -
Will Google penalise me for duplicating my own website on a new domain?
Hi guys, Currently, we are using Magento Enterprise to run two separate ecommerce websites, for arguments sake lets say these two websites are ElectronicsX.com and electronicsY.co.uk. ElectronicsX.com isn't doing too well, it has little traffic, makes next to nothing each month and is just a waste of resources. ElectronicsY.co.uk on the other hand is getting 30,000 unique visits every month and is over exceeding every month in terms of revenue in sales. At the moment, the only traffic we get from Europe tends to be the odd "word of mouth" customer and English customers who have emigrated to Europe but still like to use English companies. We can't rank on Google FR or ES or anything like that because the website we have "electronicsY" is on a .co.uk TLD (a GeoTLD). So what we want to do is take ALL the content from electronicsY.co.uk and place it on electronicsX.com so that we can start targeting Europe and ranking in the serps internationally. What sort of effect will that have in terms of penalisation from Google? Or because the websites are on the same C block, would it have any effect at all? Thanks guys Tom
On-Page Optimization | | tomhall900 -
Google Drop
I started using SEOMOZ due to a sudden and huge drop in Google for two main keywords (hair bows and baby headbands). Our site (BloomingBows.com) has held a top three spot for years with these words and then in the last few months has dropped down on the first page and now they are completely off the charts. Is there any insight as to why? Also, we have been very active using the data from here in the last week or so to clean up and improve anything listed, but I am still seeing keywords drop into the 40 - 60 position and our traffic is drying up. Starting to panic and wondering if I am missing something or going about this in the wrong way. ANY insight is appreciated at this point!! Thank you!!
On-Page Optimization | | bloomingB0 -
What´s the penalization extent applied by Google?
Hi! I still don´t get this web site penalization applied by Google due to duplicate content. My site has many of pages that were among the first positions for top keywords (A Photoshop web site). Those pages were linked by sites like LifeHacker, BoingBoing, Microsiervos, SmashingMagazine, John Nack, and many other well known blogs. After mid February 2012 everything went down the drain. I lost half of my traffic and my well ranked pages are now almost nowhere to be found. I have plenty of ads in some pages of my site, and duplicate content (amazon product description only) in other pages of my site. So, the good quality pages my site has, are no longer considered as good quality just because I have some duplicate content or ad filled pages? I´m not complaining. I´m trying to understand this. Google needs to serve good information to their visitors. But since they found some trash in my site, they decide to remove both the trash and the good information from the search engine? That doesn´t sound logical to me. Why don´t they just remove the trash and leave the good content? Of course, I understand that information is added everyday and some may come up with something better than mine, but dropping 40 or more places in the ranking sounds more like a penalty to me. Again, I´m not complaining (although it sounds like I am!), just want to understand the reasons behind this. Thanks, Enrique
On-Page Optimization | | enriquef0 -
How long does it take to seen Title in SERP
A friend of mine asked me how long it takes before Google shows the right Title in the SERPs. He changed the title of his homepage some weeks ago but Google still shows the old Title in the rankings... I'm a Wordpress user and don't have this problem, when I change something in the title it doesn't take that long to see the changes showing up in SERPs
On-Page Optimization | | nvs.nim0 -
Does google treat all urls equal?
Sorry for the lame title, i couldn't think of a better one. I want to know if google treats this: http://www.domain.com/products/some-product-name the same as it would treat: http://www.domain.com/?products=some-product-name if not, could you tell me the differences?
On-Page Optimization | | adriandg0