Google indexing site content that I did not wish to be indexed
-
Hi is it pretty standard for Google to index content that you have not specifically asked them to index i.e. provided them notification of a page's existence.
I have just been alerted by 'Mention' about some new content that they have discovered, the page is on our site yes and may be I should have set it to NO INDEX but the page only went up a couple of days ago and I was making it live so that someone could look at it and see how the page was going to look in its final iteration. Normally we go through the usual process of notifying Google via GWMT, adding it to our site map.xml file, publishing it via our G+ stream and so on.
Reviewing our Analytics it looks like there has been no traffic to this page yet and I know for a fact there are no links to this page. I am surprised at the speed of the indexation, is it a example of brand mention? Where an actual link is now no longer required?
Cheers
David
-
Thanks Candyman, yes this is not a question about to prevent Google for not indexing my content, I know this very well. It is more about how quick they have done this with the least amount of effort on our part to inform them.
Plus it is quite an interesting situation you found yourself in, never heard of this before.
Many thanks
David
-
Hi David-
We had a similar situation recently where we had a dev site and forgot to no-index it and actually started to appear in the SERPS. After a bit of puzzling it LOOKS like Google found (or at least indexed) the pages as a function of us being logged into our Google accounts when viewing them. We did not do extensive testing on this, its mostly anecdotal but ti did look like it was true. Maybe we'll do the experiment one day to be sure!
Ken
-
Google is constantly indexing and viewing your website. Why go through the other steps? To ensure that your new page isn't overlooked. While you don't necessarily need to tell Google to index in GWT - your site map should automatically update, and if referenced in the robots.txt file than the new page will be found without issue.
Now, again if you don't want a page indexed and it has links than you need to do the noindex / no follow on the page, as the robots.txt can be over-ruled.
-
Hi Samuel,
Thanks for replying but no I'm not asking that, this I know how to do. The question is about whether this could be seen as an example of page indexation where on my part there has been no explicit activity to inform Google of the content's existence and there are no links to it yet Google is still managing to index it. Why bother informing Google vIA some of the activities mentioned earlier when they will just index it anyway you know.
Thanks
David
-
Are you asking how to prevent certain pages from appearing in search results? If so, I'd review Moz's guide to robots.
Specifically, I'd recommend the use of both the noindex meta tag and the robots.txt file. Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google's Presentation Yesterday
We hired a new website/marketing company that is a Preferred Google Partner (one of two in Charlotte according to them) and they hosted a presentation by Google at the Google Fiber office in Charlotte yesterday. As expected, there were lots of self-promotion by Google, accompanied with a plethora of data they created to support their PPC Marketing. It was an impressive performance with Molly Dince and Celena Fergusson, presenting Google Marketing Solutions: "Making the Web Work For You" and the keynote speaker Tim Reis, Director of Performance Agencies at Google: speaking on "Mobile Micromoments: Why Your Biggest Opportunities Are In The Smallest Moments" They ended with 15 minutes of Q&A and my question was answered with "I don't know" which I found surprising. So, here it is Thursday morning and I'm asking the same question to my Moz Family for some feedback: "Since the removal of Ads from the right column of a SERP, what percentage of Google traffic comes from Ads vs. the Organics?" I look forward to your comments. TY,
Algorithm Updates | | KevnJr
KJr0 -
Recent Google algorithm update?
Two of our clients have experienced a huge dip in organic rankings during the past week or so and we haven't done anything that would cause this. Have there been any major Google changes reported lately? I'm not seeing anything reported here: https://moz.com/google-algorithm-change. Thanks for your input. Eric
Algorithm Updates | | EricFish0 -
Duplicate Content
I was just using a program (copyscpape) to see if the content on a clients website has been copied. I was surprised that the content on the site was displaying 70% duplicated and it's showing the same content on a few sites with different % duplicated (ranging from 35%-80%) I have been informed that the content on the clients site is original and was written by the client. My question is, does Google know or understand that the clients website's content was created as original and that the other sites have copied it word-for-word and placed it on their site? Does he need to re-write the content to make it original? I just want to make sure before I told him to re-write all the content on the site? I'm well aware that duplicate content is bad, but i'm just curious if it's hurting the clients site because they originally created the content. Thanks for your input.
Algorithm Updates | | Kdruckenbrod0 -
Why does my site dissappeare from the top 50?
Hellow I am having some problems with my site www.kondomanija.si. It was ranked on the first page for my main KW kondomi (in www.google.si, Slovenia) but now it is not in the top 10 pages. And this has happened before, it drops out of the top 10 pages and in a cople of moths it is back for a short time (till it drops out again). It think the site has a week link profile... Could this be the reason? Does anybody know what is going on?
Algorithm Updates | | Spletnafuzija0 -
What is the point of XML site maps?
Given how Google uses Page Rank to pass link juice from one page to the next if Google can only find a page in an XML site map it will have no link juice and appear very low in search results if at all. The priority in XML sitemaps field also seems pretty much irrelevant to me. Google determines the priority of a page based on the number of inbound links to it. If your site is designed properly the most important pages will have the most links. The changefreq field could maybe be useful if you have existing pages that are updated regularly. Though it seems to me Google tends to crawl sites often enough that it isn't useful. Plus for most of the web the significant content of an existing page doesn't change regularly, instead new pages are added with new content. This leaves the lastmod field as being potentially useful. If Google starts each crawl of your site by grabbing the sitemap and then crawls the pages whose lastmod date is newer than its last crawl of the site their crawling could be much more efficient. The site map would not need to contain every single page of the site, just the ones that have changed recently. From what I've seen most site map generation tools don't do a great job with the fields other than loc. If Google can't trust the priority, changefreq, or lastmod fields they won't put any weight on them. It seems to me the best way to rank well in Google is by making a good, content-rich site that is easily navigable by real people (and that's just the way Google wants it). So, what's the point of XML site maps? Does the benefit (if any) outweigh the cost of developing and maintaining them?
Algorithm Updates | | pasware0 -
Do links count in syndicated content?
If I write a press release that goes viral and is syndicated all over do each of those links to my site in the syndications of the press release count and pass page rank with Google? Or does Google only count the link in the original press release? I heard that Google counts all the links for a time then eventually counts only one link from the original content and discounting all the other links as duplicate content. Any truth to this? Thanks mozzers! Ron10
Algorithm Updates | | Ron100 -
Did The Last Google Algorithm Update, Hit sites with poor anchor text?
My content is quite strong within my niche, so I ranked well, but last month my rankings plummeted. On closer examination and scrutiny I discovered my anchor text needed updating. Has anyone else seen this happening in the last four weeks?
Algorithm Updates | | simonberenyi0 -
Did Google just give away how Penguin works?
At SMX during the You&A with Matt Cutts, Danny asked why the algo update was called Penguin. Matt said: "We thought the codename actually might give too much info about how it works so the lead engineer got to choose." Last night Google released their 39 updates for the month of May. Among them was this: "Improvements to Penguin. [launch codename "twref2", project codename "Page Quality"] This month we rolled out a couple minor tweaks to improve signals and refresh the data used by the penguin algorithm." Whoa, codename twref2 for Penguin improvement? Is this giving us an insight about how it works? I would guess the ref2 means second refresh perhaps. But tw I am not sure about. What do you think? Is there a hidden insight here?
Algorithm Updates | | DanDeceuster1