Blocked by Meta Robots.
-
Hi,
I get this warning on my reporting.
- Blocked by Meta Robots - This page is being kept out of the search engine indexes by meta-robots.
what does that means ? and how to solve that, if i using wordpress as my website engine.
and about rel=canonical , in which page I should put this tag, in original page, or in copy page ?
thanks for all of your answer, it will be means a lot
-
There are wordpress plugins you can use to modify your robots.txt, wordpress makes it difficult.
http://yoast.com/example-robots-txt-wordpress/
Also, make sure it is an important page for your blog- Google is just being proactive on your behalf, it might be an irrelevant page to your overall plan.
-
Actually it would not be in the meta robots noindex. The meta tag does not prevent Google from crawling the page it is on. If it did that, then Google would not be able to crawl the page and then it would not be able to read the tag :-). The meta robots will tell Google to remove the page from the index and so it is very effective for that application.
That said, the GWT warning, is probably related to you robots.txt file located at
http://www.yourdomain.ext/robots.txt
Put that in your browser and see if you have any of your files, pages Disallowed in that file. If that is the case, then Google will not be able to spider a page to start with, let alone read the meta tags. Do some searching on Google on how robots.txt works Moz obviously has one
http://moz.com/learn/seo/robotstxt
Here is a video on how to use Wordpress and robot.txt - it may or may not relate to your config, but will show a plugin that you can use to adjust
http://www.youtube.com/watch?v=JY9A5OqHTvw
You can figure out how to understand it and then what you need to update it. Get with your IT person or whoever admins your site
-
It means there is a meta tag on the page that is blocking the page. Look in the head section of the page for a tag. Remove this and you should be good to go. Check your WordPress settings, sometimes these tags are automatically assigned to pages as a default. You could also download a SEO plug in to help manage the meta robots tags.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Same linking c-blocks trend as competitor
I noticed in our competitive link report that our number of linking c-blocks has risen and fallen in the exact same pattern as one of our competitors. Is there a reason why this would be happening?
Moz Pro | | ZoomInformation0 -
Meta Tag Descriptions not being found in Moz Crawls
Hey guys, I have been managing a few websites and have input them into Moz for crawl reports, etc. For a while I have noticed that we were getting a gratuitous amount of errors when it came to the number of missing meta tags. It was numbering in the 200's. The sites were in place before I got here and a lot of the older posts no one had even attempted to include tags, links of the page or anything. As they are all Wordpress Sites and they all already had the Yoast/Wordpress SEO plug-in installed on them, I decided I would go through each post and media file one at a time and update their meta tags via the plug in. I personally did this so I know that I added and saved each one, however the Moz crawl reports continue to show that we are missing roughly 200 meta tags. I've seen a huge drop off in 404 errors and stuff since I went through and double checked everything on the sites, however the meta tag errors persist. Is this the case that Moz is not recognizing the tags when it crawls because I used the Yoast Plugin? Or would you say that the plugin is the issue and I should find another way to add meta tags to the pages and posts on the site? My main concern is that if Moz is having issues crawling the sites, is Google also seeing the same thing? The URLS include:
Moz Pro | | MOZ.info
sundancevacationsblog.com
sundancevacationsnews.com
sundancevacationscharities.com Any help would be appreciated!0 -
The pages that add robots as noindex will Crawl and marked as duplicate page content on seo moz ?
When we marked a page as noindex with robots like {<meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noindex</a>" />} will crawl and marked as duplicate page content(Its already a duplicate page content within the site. ie, Two links pointing to the same page).So we are mentioning both the links no need to index on SE.But after we made this and crawl reports have no change like it tooks the duplicate with noindex marked pages too. Please help to solve this problem.
Moz Pro | | trixmediainc0 -
Fix Missing Meta-Description?
We have a Wordpress site. SEOMoz crawl diagnostics are showing 144 missing meta-description tags. The problem is that about 100 are blog pages: http://www.portlandpersonalinjuryaccidentlawyer.com/blog/page/15/ http://www.portlandpersonalinjuryaccidentlawyer.com/blog/page/16/ http://www.portlandpersonalinjuryaccidentlawyer.com/blog/page/17/ Etc. We have a meta-description for http://www.portlandpersonalinjuryaccidentlawyer.com/blog/ but not for every page.... is this a problem? How can we fix it? Thanks!
Moz Pro | | DuBois0 -
Does the SEOMoz weekly crawl that highlights no meta description tag, take into account if there is a meta robots noindex,follow tag on the pages it indicates the missing meta descriptions?
The weekly crawl website report is telling me that there are pages that have missing meta description tags, yet I've implemented meta robots tags to 'noindex, follow' those pages which are visible in those page source files. As far as Google Is concerned, surely this then won't be a problem since it is being instructed NOT to consider these specific pages for indexing. I am assuming that the weekly SEOmoz website crawl is simply throwing the missing meta description crawl findings into its report without itself observing that the particluar URL references contain the meta robots 'noindex,follow' tag ???? Appreciate if you can clairfy if this is the case. It would help me understand that (at least in terms of my efforts towards Google) your own crawl doesn't observe the meta robots tag instruction, hence the resultant report's flagging the discrepancy.
Moz Pro | | callassist0 -
Blocking all robots except rogerbot
I'm in the process of working with a site under development and wish to run the SEOmoz crawl test before we launch it publicly. Unfortunately rogerbot is reluctant to crawl the site. I've set my robots.txt to disallow all bots besides rogerbot. Currently looks like this: User-agent: * Disallow: / User-agent: rogerbot Disallow: All pages within the site are meta tagged index,follow. Crawl report says: Search Engine blocked by robots.txt Yes Am I missing something here?
Moz Pro | | ignician0 -
Crawl Diagnostics and missing meta tags on noindex blog pages
Hi Guys/Gals We do love the Crawl Diagnostics, but do find the missing meta tags ("Missing Meta Description" Tag in this case) somewhat spammy. We use the "All in One SEO Pack" for our blog and it does stick in noindex,follow (as it should) on the pages that is of no use to us. "2008/04/page/2/" and the likes. Maybe I'm wrong but should the Diagnostics tool not respect the noindex tag and just ignore any warnings, since it should really mean that these pages are NOT included in the search index. Meaning that the other meta tags are really useless. Any thoughts?
Moz Pro | | sfseo0 -
What's name of SEOmoz and Open Site Explorer robots?!
I would like to exclude in robots.txt SEOmoz and Open Site Explorer bots to don't let them index my sites… what's their names?
Moz Pro | | cezarylech0