Some web 2.0 sites blocked by robot
-
I had created some web2.0 sites, google also indexed those sites but those sites robots.txt are disallow, this mean Google will not give any credit on ranking ? I have seen in google, those sites meta description was written - A description of this site is not available due to the robot.txt file. I checked that file, it is written disallow. Can google still count those sites for the ranking purpose ?
-
Hi,
Pages blocked by robots.txt file will not be considered for ranking. You still see those pages in Google's index is because of their mentions on other websites. Google has been doing link discovery from websites other than these Web2.0 sites of yours. So to stop Google completely from indexing these pages, you should go in for page-levelrobots meta tag. Here you go for more: https://support.google.com/webmasters/answer/93710?hl=en
Once you implement the page-level robots meta tag to block those pages, Google will remove them from the index after the next crawl. You can expedite this process by using the link removal procedure in webmaster tools account.
Here you go for more: https://support.google.com/websearch/troubleshooter/3111061?rd=1
Good Luck to you my friend.
Best regards,
Devanur Rafi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
[help] I launch my site 2 month ago
Hi all i launch my site 2 month ago here is site link seotoolscloud.com my site is index in google but not article index now even my article also written by self?? any one tell me how can index fast and what is best solution to rank my site??? help help help
Link Building | | 63bxbbzhsj0 -
Backlinks from missing sites
While combing though spammy backlinks, I'm finding a lot of supposedly "active" links that lead to 404 errors, pages that contain no html, domains for sale, and sites that all have a uniform design which I can't tell if they're web host default pages, or pages using a spammy third-party link aggregator (or both).
Link Building | | rickmic
For example: http://easyreveal.us/ So, do I disavow missing sites? Lost sites? Is there a drawback to disavowing sites that have been removed? Any help would be appreciated.0 -
Is ListofDomains.org a bad linking site?
I am trying to clean up/disavow bad linking domains and ListofDomains.org appears numerous times. Any idea what this site is about and should it make the disavow list?
Link Building | | khull0 -
Best On-Site Internal Linking Practices?
Hi everyone, Our company works extensively with ecommerce website, and we're beginning to wonder with all of the impending Google updates what our best bets for internal linking practices will be, in particular when it comes to menu options. We typically set up our navigation with top-level categories in the main menu, then drop downs to the sub-category level pages. Our question is, should the links in the drop-down menus be followed, and if so, should they be followed links across all pages of the website, or just on the homepage menu list, or not at all? We're trying to figure out what will pass the most internal linking power without being too much and overly "spammy," I guess. Thanks for your input!
Link Building | | MountainMedia0 -
Reassessing site achitecture
Firstly I'm only a recently new member but am already a big fan and rely on the tools a lot. For the sake of context, our site is www.cleancruising.com.au which we've steadily built into one of Australia's top cruise sites. We've focussed on building unique content including a mapping engine to show over 10,000 cruise routes, 3D ship models for virtual tour simulations and over 350,000 words of unique descriptive content. So we think we've got some good content which we currently rank quite well for, but think there's a lot of room for improvement with the links, both internal and inbound. Looking at the internal links, I thought the site architecture was pretty good but now I'm not so sure. The site hierarchy is like this: home > cruiseline (20) > ship (200) > itinerary page (12000) > enquiry form and home > region (25) > port (1400) > itinerary page (12000) > enquiry form My question is, am I wasting link juice by doing the following? A sample itinerary page shows lots of links through to the enquiry form (with different parameters). However years ago we added the enquiry form to be blocked via robots.txt and none of these page variations are indexed by google. Is this stuffing up the link flow, and should I allow the forms to be indexed so their links can add value back through the site? Are there any tools for visualizing a site link structure? Lots of questions and hope this is appropriate. 1am in Australia so may take a few hours before any follow up posts, cheers
Link Building | | cruiserDan0 -
New Site Checklist - Links and Social Aspects
Hi I have a new site which launched around 6 weeks ago, new domain name with a partial keyword match on it, so its going to be a long process but one which I want to tred along the right path for long term results What I am looking for is a checklist if possible on sites that the site (which is a UK based home improvments company) should be listed on? The difficulty is because it can seem a boring subject (windows and doors) it makes it difficult to find authentic quality links, especially with no big budgets or pools of article writers I have registered against the likes of Brownbook, Yelp, Yell.com, Qype, Google Places covering the local aspect but what about the general external links Is it advisable to stay away from directory listings, blog comments, forum posts etc which is what a lot of the competition are doing (weatherseal .co.uk anglianhome .co.uk etc
Link Building | | ocelot0 -
For a new site, does it matter which sites I get back links from first? I.e., are the first back links more important than later ones?
I've heard that when a site is new, the sources from which it gets its first back links are very important -- that the early links are more important for ranking than later links. Is this true. If so, what would be some good places to get these very first links? Thanks.
Link Building | | sigma3x0 -
New PDF Guide. On My site or linked by blogs?
Hi experts, I created a new worth guide about food. Its purpose it to improve backlink by adding website's URL on PDF's footer (spider crawls that one, is it true?). I want send my guide to trusted blogs so users can download new pdf guide on the web. Is it the best way to gain popularity and let my backlinks grow? I've seen that seomoz published is guide such as html pages.
Link Building | | Greenman
Could it be the worth way for me? Thanks a bunch in advance! G.0