Does Disallowing a directory also tell search engines to unindex?
-
I have a bunch of duplicate pages/duplicate title issues because of Joomla's item/category/menu structures.
I want to tell search engines not to crawl, and also to unindex anything in those directories in order to solve the duplicate issues.
I thought of disallowing in robots.txt, but then I realized that might not remove the URLs if they've already been indexed.
Please help me figure this out.
-
Yes, this will remove them, but if you want to speed up the process you can use the google URL removal tool http://support.google.com/webmasters/bin/answer.py?hl=en&answer=164734
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search visibility consistently low. No apparent cause.
I'm trying to nail down why a site would be consistently showing a search visibility of 1-2% whilst similar competitors are in 20-30s. There are no site errors reported. Sitemap, robot.txt, meta titles & descriptions, key word presence, alt attributes, load times, canonicals, etc all check out as fine. Backlinks profile is healthier than competitors'. Yet even searching for our main product the result links to an obscure blog page rather than our main site, despite the presence of identical and similar keywords on our homepage, in our title, h1 tags, web address... Site content and design seem subjectively good and at the very least matches better performing competitor site's. Does anyone know of any less visible reason as to why a site would be tanking so badly in search rankings? Have checked using other SEO tools and they all report the same as Moz.
Intermediate & Advanced SEO | | SimonZM1 -
How Google organic search results differ in Local Searches?
We all know Google displays nearby results by locating our ip address. My question is how does these results differ? For eg 1. If someone from Newyork search for "chinese Restaurant in Newyork" 2. Someone from California search for "chinese Restaurant in Newyork" 3. Someone from California changes his location to Newyork and search for "chinese Restaurant in Newyork" What are the factors the Google SERP looks into to display the result in local terms?
Intermediate & Advanced SEO | | rajeevEDU0 -
Customer Experience vs Search Result Optimisation
Yes, I know customer experience is king, however, I have a dilema, my site has been live since June 2013 & we get good feedback on site design & easy to follow navigation, however, our rankings arent as good as they could be? For example, the following 2 pages share v similar URLs, but the pages do 2 different jobs & when you get to the site that is easy to see, but my largest Keyword "Over 50 Life Insurance" becomes difficult to target as google sees both pages and splits the results, so I think i must be losing ranking positions? http://www.over50choices.co.uk/Funeral-Planning/Over-50-Life-Insurance.aspx http://www.over50choices.co.uk/Funeral-Planning/Over-50-Life-Insurance/Compare-Over-50s-Life-Insurance.aspx The first page explains the product(s) and the 2nd is the Quote & Compare page, which generates the income. I am currently playing with meta tags, but as yet havent found the right combination! Originally the 2nd page meta tags were focussing on "compare over 50s life insurance" but google still sees "over 50 life insurance" in this phrase, so the results get split. I also had internal anchor text supporting this. What do you think is the best strategy for optimising both pages? Thanks Ash
Intermediate & Advanced SEO | | AshShep10 -
Rich Snippets - Aggregated Review in Organic Search
Hi All, Let’s say you are a service provider such a Garden Landscaper, and over time your customers have placed reviews on Google Local Places. As part your sites redesign your looking to implement "Rich Snippets Aggregated Reviews". Your organic natural search results should include the reviews; aggregated star picture, aggregated review value and total reviews. Question #1 Is this possible?
Intermediate & Advanced SEO | | Mark_Ch
#2 What is the best way of achieving this via Wordpress as a plugin?
#3 Do you need to file for consideration via Google? Any other useful advice will be appreciated. Thanks Mark0 -
My website (non-adult) is not appearing in Google search results when i have safe search settings on. How can i fix this?
Hi, I have this issue where my website does not appear in Google search results when i have the safe search settings on. If i turn the safe search settings off, my site appears no problem. I'm guessing Google is categorizing my website as adult, which it definitely is not. Has anyone had this issue before? Or does anyone know how to resolve this issue? Any help would be much appreciated. Thanks
Intermediate & Advanced SEO | | CupidTeam0 -
Blog Not Ranking Well at All in Search Engines, Need Help!
Hi Mozers, Need some help on a CMS I've been working with over the last year. The CMS is built by a team of guys here in Washington State. Basically, I'm having issues with clients content on the blog system not getting ranking correctly at all. Here's a few problems I've noticed: Could you confirm and scale these problems based upon being, "not a problem" "a problem" and "critical must fix" 1. The title tag is pulling from the title of the article which is also automatically generating a URL with underscores instead of dashes. Is having a duplicate URL, Title, and Title tag spammy looking to search engines? Are underscores on long URL's confusing google? Where shorter one's are fine (i.e. domain/i_pad/
Intermediate & Advanced SEO | | Keith-Eneix
(i.e.http://www.ductvacnw.com/blog/archives/2013/05/20/5_reasons_to_hire_a_professional_to_clean_your_air_ducts_and_vents), 2. The CMS is resolving all URL's with a canonical instead of a 301 redirect (I've told webmaster tools which preferred url should be indexed). Does using a canonical over a 301 redirect cause any confusion with Google? Is one better practice then the other? 3. The H1 tags on the blog pull from "blog category" instead of the title of the blog post. Is this is a problem? 4. The URl's are quite long with the added "archives/2013/05/20/5". Does this cause problems by pushing the main target keyword further away from the domain name? 5. I'm also noticing the blog post is actually not part of the breadcrumbs where we normally would expect that to populate after the blog category name, Problem? These are some of the things I've noticed and need clarification on. If you see anything else please let me know?0 -
Google cached pages and search terms
Here's something I noticed. We have a rank A page and it's ranking 10 on Google search results. When I hover my mouse over our search result, Google gives us a preview, but Google also highlights in red where the search keyword is present on the page. Reviewing our page, even though we have it as the h1 header and intro paragraph, Google is highlighting it half way down the page. Any ideas why? I review rank 1 - 5 and Google highlights the keyword on the intro paragraph and h1 header Have you guys experienced anything like this? It makes me think..Google could be crawling my site and thinking I haven't got it in the h1 or intro paragraph etc.. Thoughts?
Intermediate & Advanced SEO | | Bio-RadAbs0 -
Confirmation Needed: Do any search engine crawlers accept cookies?
I'm looking for confirmation here. Do any search engine crawlers accept cookies? I thought that the answer was always no, but we're looking in our weblogs and seeing some odd behavior.
Intermediate & Advanced SEO | | SharieBags0