Does Disallowing a directory also tell search engines to unindex?
-
I have a bunch of duplicate pages/duplicate title issues because of Joomla's item/category/menu structures.
I want to tell search engines not to crawl, and also to unindex anything in those directories in order to solve the duplicate issues.
I thought of disallowing in robots.txt, but then I realized that might not remove the URLs if they've already been indexed.
Please help me figure this out.
-
Yes, this will remove them, but if you want to speed up the process you can use the google URL removal tool http://support.google.com/webmasters/bin/answer.py?hl=en&answer=164734
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to be included in related questions (People also ask) in Google SERP?
Just wondered if anyone knows how to be included in SERP if it comes to related questions (People also ask). Do you have to fill some requirements or is it featured snippets kind of thing.
Intermediate & Advanced SEO | | Optimal_Strategies1 -
Organic search traffic dropped 40% - what am I missing?
Have a client (ecommerce site with 1,000+ pages) who recently switched to OpenCart from another cart. Their organic search traffic (from Google, Yahoo, and Bing) dropped roughly 40%. Unfortunately, we weren't involved with the site before, so we can only rely on the wayback machine to compare previous to present. I've checked all the common causes of traffic drops and so far I mostly know what's probably not causing the issue. Any suggestions? Some URLs are the same and the rest 301 redirect (note that many of the pages were 404 until a couple weeks after the switch when the client implemented more 301 redirects) They've got an XML sitemap and are well-indexed. The traffic drops hit pretty much across the site, they are not specific to a few pages. The traffic drops are not specific to any one country or language. Traffic drops hit mobile, tablet, and desktop I've done a full site crawl, only 1 404 page and no other significant issues. Site crawl didn't find any pages blocked by nofollow, no index, robots.txt Canonical URLs are good Site has about 20K pages indexed They have some bad backlinks, but I don't think it's backlink-related because Google, Yahoo, and Bing have all dropped. I'm comparing on-page optimization for select pages before and after, and not finding a lot of differences. It does appear that they implemented Schema.org when they launched the new site. Page load speed is good I feel there must be a pretty basic issue here for Google, Yahoo, and Bing to all drop off, but so far I haven't found it. What am I missing?
Intermediate & Advanced SEO | | AdamThompson0 -
Rich Snippets - Aggregated Review in Organic Search
Hi All, Let’s say you are a service provider such a Garden Landscaper, and over time your customers have placed reviews on Google Local Places. As part your sites redesign your looking to implement "Rich Snippets Aggregated Reviews". Your organic natural search results should include the reviews; aggregated star picture, aggregated review value and total reviews. Question #1 Is this possible?
Intermediate & Advanced SEO | | Mark_Ch
#2 What is the best way of achieving this via Wordpress as a plugin?
#3 Do you need to file for consideration via Google? Any other useful advice will be appreciated. Thanks Mark0 -
Meta NoIndex tag and Robots Disallow
Hi all, I hope you can spend some time to answer my first of a few questions 🙂 We are running a Magento site - layered/faceted navigation nightmare has created thousands of duplicate URLS! Anyway, during my process to tackle the issue, I disallowed in Robots.txt anything in the querystring that was not a p (allowed this for pagination). After checking some pages in Google, I did a site:www.mydomain.com/specificpage.html and a few duplicates came up along with the original with
Intermediate & Advanced SEO | | bjs2010
"There is no information about this page because it is blocked by robots.txt" So I had added in Meta Noindex, follow on all these duplicates also but I guess it wasnt being read because of Robots.txt. So coming to my question. Did robots.txt block access to these pages? If so, were these already in the index and after disallowing it with robots, Googlebot could not read Meta No index? Does Meta Noindex Follow on pages actually help Googlebot decide to remove these pages from index? I thought Robots would stop and prevent indexation? But I've read this:
"Noindex is a funny thing, it actually doesn’t mean “You can’t index this”, it means “You can’t show this in search results”. Robots.txt disallow means “You can’t index this” but it doesn’t mean “You can’t show it in the search results”. I'm a bit confused about how to use these in both preventing duplicate content in the first place and then helping to address dupe content once it's already in the index. Thanks! B0 -
Rich Snippets not appearing in Search Results
Hi Everyone, Just a few questions on rich snippets please: We have now integrated microdata (data-vocabulary.org) on all our product pages like http://www.homeshop18.com/samsung-galaxy-tab-2-310-tablet/computer-peripherals/ipads-tablets/product:30409470/cid:8937/ which I have tested on the rich snippets testing tool which Google provides and all is working fine and is rendering properly in the tool but it is not coming in search results in google. Are we doing everything right or is there any issue in our implementation? How long does it take usually for rich snippets to appear in google organic search results?
Intermediate & Advanced SEO | | h1seo0 -
Original Site content was used for submission to article directories
I had a communication problem with my writer and she used original unspun content and posted it to Unique Article Wizard. So all UAW does is take each paragraph and mix them up. So I searched a sentence on my site where the content came from and got back a bunch of returns for that sentence. My site wasn't the first result returned. I"m wondering how bad that is going to be for me. The links from UAW are going back to an anchor layer that then links back to this site. Can anyone tell me if I need to rewrite the content on the original site? That is the only way I can think to make that not an issue. Thanks
Intermediate & Advanced SEO | | mtking.us_gmail.com0 -
If google ignores links from "spammy" link directories ...
Then why does SEO moz have this list: http://www.seomoz.org/dp/seo-directory ?? Included in that list are some pretty spammy looking sites such as: <colgroup><col width="345"></colgroup>
Intermediate & Advanced SEO | | adriandg
| http://www.site-sift.com/ |
| http://www.2yi.net/ |
| http://www.sevenseek.com/ |
| http://greenstalk.com/ |
| http://anthonyparsons.com/ |
| http://www.rakcha.com/ |
| http://www.goguides.org/ |
| http://gosearchbusiness.com/ |
| http://funender.com/free_link_directory/ |
| http://www.joeant.com/ |
| http://www.browse8.com/ |
| http://linkopedia.com/ |
| http://kwika.org/ |
| http://tygo.com/ |
| http://netzoning.com/ |
| http://goongee.com/ |
| http://bigall.com/ |
| http://www.incrawler.com/ |
| http://rubberstamped.org/ |
| http://lookforth.com/ |
| http://worldsiteindex.com/ |
| http://linksgiving.com/ |
| http://azoos.com/ |
| http://www.uncoverthenet.com/ |
| http://ewilla.com/ |0 -
How do I link a profile picture to my search results?
How do you get an icon which links to a google profile to display in google search results? Link this.... https://skitch.com/edwardrobertshaw/f12f8/central.ly-google-search
Intermediate & Advanced SEO | | ed1234560