Why is our noindex tag not working?
-
Hi,
I have the following page where we've implemented a no index tag. But when we run this page in screaming frog or this tool here to verify the noidex is present and functioning, it shows that it's not.
But if you view the source of the page, the code is present in the head tag. And unfortunately we've seen instances where Google is indexing pages we've noindexed. Any thoughts on the example above or why this is happening in Google?
Eddy
-
Hi Eddy,
Edit: this was already answered before I could post my reply. But I've left the example.
The issue with the meta robots tag is that you are using curly quotation marks around robots and noindex:
You have:
“robots**” content=“noindex”/>
Instead of:
name="robots" content="noindex"**/>This will fix your issue.
Cheers,
David
-
That SF response is from the robots.txt block, not a noindex tag though. SF is also ignoring the incorrectly formatted tag (as it should).
Paul
-
The example page does have a noindex tag in place, but it's not formatted correctly, so it's being ignored. Very subtle issue, but your tag is using "smart quotes" around the elements instead of the plain quotation marks that are required for code. If you look very carefully at the page source code, you'll see that they are quotation marks like you'd see in a Word document; the ones at the beginning of robots and noindex curl a different way than the ones at the end.) This usually occurs when the content was written in a word processor instead of a plain-text editor.
Because the tag's not formatted correctly, it's ignored by both the crawling tools and the search engines.
In addition, the site also has all pages blocked from crawling by the sitewide robots.txt file. This and noindex are conflicting instructions to search engines.
If a page is blocked in robots.txt, then the search engine will not crawl the page and so is not able to discover the noindex tag, even if it were formatted correctly. Therefore if the search engine becomes aware of the page in any other way than straight crawling (and there are a number of ways this can happen), then the page will still get indexed.
If it's a dev site, the proper way to keep it from being indexed is to either noindex all pages, or to put the site behind a password so the search engines and public visitors can't access it. If using noindex, the site must not be blocked with a robots.txt directive.
Does that all make sense?
Paul
-
I ran that page thru screaming frog and it came back with a "blocked by robots" status.
The second tool you suggested is not finding the noindex tag and I don't have an explanation for that, nor am I familiar with the tool.
A site command does not return any results.
Are you sure you have a problem? Is there another example you can provide?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Description tag in code is different from what is shown in SERPS...
Hi there: We have a client whose website we built in WP, using Yoast Pro as our SEO plugin. I was reading some reports (actually coming out of SEMrush but we use Moz as well) and I am getting really varying results in the description are of the SERPS. Even though I'm seeing the copy we wrote in Yoast in the description tag code, the SERP is showing an excerpt from the copywriting on the site. What's even weirder is that SEMrush is pulling an entirely DIFFERENT description. I'm obviously missing out on the finer points of description tags, as Google clearly does not always choose to feature what is actually written in the description tag itself. Can someone explain to me what might be going on here? Thanks in advance,
Intermediate & Advanced SEO | | Daaveey1 -
Mobile Canonical Tag Issue
Hey so, For our site
Intermediate & Advanced SEO | | ggpaul562
we have the desktop version: www.site.com/product-name/product-code/ The mobile version www.site.com/mobile/product-name/product-code So...on the desktop version we'd have the following.. | | Now my question is, what do we do as far as canonicals on the actual mobile URL? Would it be this? | |
| | OR are we NOT supposed to have mobile canonical tags whatsoever since we've already added "rel alternate" ? Would like some clarificaiton. | | |0 -
Should I nofollow my Wordpress tags?
I have a website that have a strong root domain (ranking on many terms) but the subpages (articles) doesn't rank well. My feeling is that the linkjuice is not flowing to them (not enough anyway). When I run site:http://mydomain.com I have my root as the first result and the next many results are tagpages on my sites. I have arund 180 index pages, and I need to go to down to result #50 give or take before I see any subpage using the site command. My website theme have the tags on every page possible. The tags are useful for my viewers, but not SEO useful, but I fear that they are dilluting my linkjuice. Should I nofollow and noindex them? Noindex makes sense (the tags are just duplicate content featuring snippets of text from the articles). But Nofollow would make sense too since I wouldn't send any linkjuice through the tags. What would you guys do? Bests regards
Intermediate & Advanced SEO | | claus101 -
Does alt tag optimization benefit search rankings (not image search) at all?
The benefits of alt tag optimization for traditional SEO has always been a "yo yo" subject for me. Way back in the day (2004 to 2007) I believed there was some benefit to alt tag SEO. However as time went on I saw evidence that the major search engines were no longer considering alt tag SEO as a ranking signal. However I later had the pleasure to work on a joint project with a high end SEO firm in 2011/2012. My colleagues fully believed that alt tag optimization was still a very important strategy for traditional SEO at that time. Is there any evidence available that alt tags still help with traditional SEO nowadays? I'm fully aware of the benefits of optimized alt tags and image search. However could optimized alt tags be one of those ranking factors that Google removed due to abuse and later quietly resurrected?
Intermediate & Advanced SEO | | RosemaryB0 -
Crawl Issue Found: No rel="canonical" Tags
Given that google have stated that duplicate content is not penalised is this really something that will give sufficient benefits for the time involved?Also, reading some of the articles on moz.com they seem very ambivalent about its use – for example http://moz.com/blog/rel-confused-answers-to-your-rel-canonical-questionsWill any page with a canonical link normally NOT be indexed by google?Thanks.
Intermediate & Advanced SEO | | fdmgroup0 -
What tags/coding are not good for SEO?
what tags/coding are not good for SEO? and also what tags not to include while creating website. For example - I read some where to avoid Span tag.
Intermediate & Advanced SEO | | JordanBrown0 -
ALT Tag Labels that Use Near Duplicate Text-SEO No, No???
Greetings Moz Community: About 280 pages of my 650 page commercial real estate website are listing pages. Each listing page contains between two and five photos, each with a corresponding ALT tag. My developer has set up the labeling of the ALT tags in the following manner. I can create a label for the first photo, but each subsequent photo automatically gets the same label plus a number tagged to the ALT. Like this: alt="Flatiron Loft for Rent"
Intermediate & Advanced SEO | | Kingalan1
alt="Flatiron Loft for Rent - Photo 0"
alt="Flatiron Loft for Rent - Photo 1"
alt="Flatiron Loft for Rent - Photo 2"
alt="Flatiron Loft for Rent - Photo 3" Is this method neutral, positive or negative for SEO? I am concerned that this manner of labeling ALT tags might risk triggering a duplicate content penalty. In early July I migrated the site from Drupal to Wordpress. We changed the URL structure (adding a sub-directory) for the listings at that time. Google is refusing to index about 100 listing pages. Any chance the ALT tags are contributing to Google's reluctance to index the URLs? I might also add that images are hosted on Amazon's CDN. A sample listing URL is http://www.nyc-officespace-leader.com/listings/278-21st-street-flatiron-loft-for-rent
Note: (/listings/278) were added to the URL in July, representing the listing sub directory plus the listing number. I Look forward to hearing the opinion of the MOZ community!!! THANKS!!!
Alan1 -
Canonical Tag Uses Source Title and Meta Data?
When optimising a regional same language micro site within a sub folder of a .com it dawned on me that our use of the hreflang and canonical meta elements will render individual elements such as H1 and title obsolete. As a canonical tag takes the canonical source title and meta right? It would still have value in optimising localised headings though? Appreciate any thoughts, suggestions (o:
Intermediate & Advanced SEO | | 3wh0