Google indexing site content that I did not wish to be indexed
-
Hi is it pretty standard for Google to index content that you have not specifically asked them to index i.e. provided them notification of a page's existence.
I have just been alerted by 'Mention' about some new content that they have discovered, the page is on our site yes and may be I should have set it to NO INDEX but the page only went up a couple of days ago and I was making it live so that someone could look at it and see how the page was going to look in its final iteration. Normally we go through the usual process of notifying Google via GWMT, adding it to our site map.xml file, publishing it via our G+ stream and so on.
Reviewing our Analytics it looks like there has been no traffic to this page yet and I know for a fact there are no links to this page. I am surprised at the speed of the indexation, is it a example of brand mention? Where an actual link is now no longer required?
Cheers
David
-
Thanks Candyman, yes this is not a question about to prevent Google for not indexing my content, I know this very well. It is more about how quick they have done this with the least amount of effort on our part to inform them.
Plus it is quite an interesting situation you found yourself in, never heard of this before.
Many thanks
David
-
Hi David-
We had a similar situation recently where we had a dev site and forgot to no-index it and actually started to appear in the SERPS. After a bit of puzzling it LOOKS like Google found (or at least indexed) the pages as a function of us being logged into our Google accounts when viewing them. We did not do extensive testing on this, its mostly anecdotal but ti did look like it was true. Maybe we'll do the experiment one day to be sure!
Ken
-
Google is constantly indexing and viewing your website. Why go through the other steps? To ensure that your new page isn't overlooked. While you don't necessarily need to tell Google to index in GWT - your site map should automatically update, and if referenced in the robots.txt file than the new page will be found without issue.
Now, again if you don't want a page indexed and it has links than you need to do the noindex / no follow on the page, as the robots.txt can be over-ruled.
-
Hi Samuel,
Thanks for replying but no I'm not asking that, this I know how to do. The question is about whether this could be seen as an example of page indexation where on my part there has been no explicit activity to inform Google of the content's existence and there are no links to it yet Google is still managing to index it. Why bother informing Google vIA some of the activities mentioned earlier when they will just index it anyway you know.
Thanks
David
-
Are you asking how to prevent certain pages from appearing in search results? If so, I'd review Moz's guide to robots.
Specifically, I'd recommend the use of both the noindex meta tag and the robots.txt file. Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google & Site Architecture
Hi I've been reading the following article about Google's quality signals here: https://searchenginewatch.com/2016/10/10/guide-to-google-ranking-signals-part-6-trust-authority-and-expertise/?utm_source=Search+Engine+Watch&utm_campaign=464594db7c-11_10_2016_NL&utm_medium=email&utm_term=0_e118661359-464594db7c-17828341 They mention - 3) All your categories should be accessible from the main menu. All your web pages should be labelled with the relevant categories. Is this every category? We have some say 3 levels deep, and they aren't all in the menu. I'd like them to be, so would be good to make a case for it. Thank you
Algorithm Updates | | BeckyKey1 -
Are SEO Friendly URLS Less Important Now That Google Is Indexing Breadcrumb Markup?
Hi Moz Community and staffers, Would appreciate your thoughts on the following question: **Are SEO friendly URLS less important now that Google is indexing breadcrumb markup in both desktop and mobile search? ** Background that inspired the question: Our ecommerce platform's out of the box functionality has very limited "friendly url" settings and would need some development work to setup an alias for more friendly URLS. Meanwhile, the breadcrumb markup is implemented correctly and indexed so it seems there's no longer an argument for improved CTR with SEO friendly URLS . With that said I'm having a hard time justifying the URL investment, as well as the 301 redirect mapping we would need to setup, and am wondering if more friendly URLs would lead to a significant increase in rankings for level of effort? Sidenote: We already rank well for non-brand and branded searches since we are brand manufacturer with an ecommerce presence. Our breadcrumbs are much cleaner & concise than our URL structure. Here are a couple examples. Category URL: http://www.mysite.com/browse/category1/subcat2/subcat3/_/N-7th
Algorithm Updates | | jessekanman
Breadcrumb: www.mysite.com > category1 > subcat2 > subcat3 Product URL: http://www.mysite.com/product/product-name/_/R-133456E112
Breadcrumb: www.mysite.com > category1 > subcat2 > subcat3 > product name The "categories" contain actual keywords just hiding them here in the example. According to my devs they can't get rid of the "_" but could possible replace it with a letter. Also they said it's an easier fix to make the URLs always lower case. Lastly some of our product URLS contain non-standard characters in the product name like "." and "," which is also a simpler fix according to my developers. Looking forward to your thoughts on the topic! Jesse0 -
Duplicate content on a sub domain
I have two domains www.hairremoval.com and a sub domain www.us.hairromoval.com both sites have virtual the same content apart from around 8 pages and the sub domain is more focused to US customers so the spelling are different, it is also hosted in the states. Would this be classed as duplicate content ? (The url’s are made up for the question but the format is correct)
Algorithm Updates | | Nettitude0 -
Am I the only one experiencing this Google SERP problem?
I perform Google searches every single day, sometimes several times in a day. These searches have nothing to do with being a marketer--they're simply as a consumer, researcher, person who needs a question answered, or in other words: a typical person. For about the past month or so, I have been unsuccessful at finding what I'm looking for on the first try EVERY SINGLE TIME. Yes, I mean it--every single time. I'm left either going all the way to the third page, clicking dozens of results and retuning to the SERPs, or having to start over with a differently worded query. This is far too often to be a coincidence. Has this been happening to anymore else? I know there was a recent significant algorithm update, right? I always look at algorithm updates through the eyes of an SEO, but I'm currently looking at it through the eyes for an average searcher, and I'm frustrated! It's been like trying to find something on Bing!
Algorithm Updates | | UnderRugSwept0 -
Youtube, Video SEO, & my site
For our business we are building a collection of videos ranging including product info, how-to's, and some funny content. My understanding is that if you embed these onto my site from youtube you don't get any credit for these videos on the web site even if submitting a video sitemap. My thinking is to post these videos to youtube and to host them on our own site and submit a video sitemap including the videos on our site. We would change the name, description, etc. on youtube vs. what's o our web site. Question is - is this the best strategy? Do I get penalized for duplicate content? They are important for both the social aspects of youtube and the content vaue of our web site.
Algorithm Updates | | uwaim20120 -
Google Cached Pages
I made some on-site changes to a site last week, in particular their page titles. This was all done on the same day at the same time. Now, one of those pages, got re-indexed on August 8th and has my updated changes, which also helped with my ranking. The other page I made changes to still shows a cached version from July 27th, which is before I made the changes. Why wouldn't google have an updated page from August 8th for both pages, not just one?
Algorithm Updates | | MichaelWeisbaum0 -
Question relates to mobile site & duplicate content.
We are working on the mobile version of a large site (migraine.com) and will be using a separate theme for it (directing visitors to m.migraine.com)- what are the necessary code or other important step we should take so that we do get penalized for having duplicate content? Thank you in advance for your responses
Algorithm Updates | | OlivierChateau0 -
High bounce rates from content articles influencing our rankings for rest of site
We have a large content article section on our e-commerce site that receives a lot of visits but also have very high bounce rates. We are wondering if this is hurting the rest of our site's rankings. **When I say bounce rates I mean what ever metrics Google is using to determine quality content (specifically after the Panda update). ** We are trying to determine if having the content articles on our domain hurts us. We only have the content articles for link building.
Algorithm Updates | | seozachz0