Domain.com and domain.com/index.html duplicate content in reports even with rewrite on
-
I have a site that was recently hit by the Google penguin update and dropped a page back. When running the site through seomoz tools, I keep getting duplicate content in the reports for domain.com and domain.com/index.html, even though I have a 301 rewrite condition. When I test the site, domain.com/index.html redirects to domain.com for all directories and root. I don't understand how my index page can still get flagged as duplicate content.
I also have a redirect from domain.com to www.domain.com.
Is there anything else I need to do or add to my htaccess file?
Appreciate any clarification on this.
-
Hello Anthony,
Saw this still open.
If your index.html "Rewrite" code is accurate, could the issue be WWW, i.e. http://www.domain.com vs. http://domain.com?
RewriteCond %{HTTP_HOST} ^domain.com
RewriteRule ^(.*)$ http://www.domain.com/$1 [R=permanent,L] -
I checked one of your campaigns, and it does seem like the 301-redirect is working properly. I'm also not seeing any evidence of links to the "index.htm" version or other issues. I don't see evidence of both version sin Google's index. Not sure exactly what's going on here, but I'll run it by the support team. I don't think you have cause for concern.
-
Thank you for the feedback and help.
I have looked up url removal in webmaster tools and it states that the page must be removed from the site. If I remove index.html I wont have a home page. Am I understanding you correctly? Heres what google states on url removal.
To remove a page or image, you must do one of the following:
- Make sure the content is no longer live on the web. Requests for the page must return an HTTP 404 (not found) or 410 status code.
- Block the content using a robots.txt file.
- Block the content using a meta noindex tag.
Please clarify when you get a moment.
I would have thought the htaccess 301 redirects from www.domain.com/index.html to www.domain.com would be enough.
Thank you in advance.
-
a) request removal of the /index.html URL in webmaster tools and it will go away in Google's index quickly.
b) make sure that when you link to your homepage on your site you are not linking to the /index.html URL - I bet you are somewhere
do a sitewide search in dreamweaver to find all instances and do a global replace.
-
It could take a little time. I did some redirects myself earlier this year, but the old pages are still in Google's index.
Maybe someone else can confirm that it can take a little time before the old pages are dropped from Google's index?
-
HTTP/1.1 301 Moved Permanently => Date => Tue, 08 May 2012 13:44:26 GMT Server => Apache/2.0.52 (CentOS) Location => http://www.domain.com/ Content-Length => 330 Connection => close Content-Type => text/html; charset=iso-8859-1
-
Did you verify with a tool like http://www.webconfs.com/http-header-check.php that you get a 301 redirect?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Parked Domains and Local SEO
Hi, I wanted to know what your thoughts were on **Parked Domains **and Local SEO. I came across a company that has a lot of parked domains that were created for different countries. i.e. xxxxx.co.uk, xxxxx.ch, xxxxx.com etc. The content they load are from the main domain xxxxx.com I went to a country specific Google search, for instance google.co.uk and google.ch Searched for the company name and xxxxx.com showed up, not the parked domains that existed for each "country" No analytics exist as the tracking code is on xxxxx.com so I can't see if any specific traffic goes to xxxxx.ch, or other domains. From my understand, parked domains lead to unnecessary duplicate content. So why have them? Simply setup a redirect to xxxxx.com from the parked domain. But are there benefits for Local SEO? Search engines like google will still see the domain as a separate entity if no redirect is in place. If so, I realise content has to be unique..but it raises a question... Why have parked domains at all apart from stopping your competitors or users typing other variance of your URL? What do you guys think?
Moz Pro | | Diamond.Dog0 -
Duplicate content across two websites
Hi. I'm looking at ways to compare duplicate content across two different websites instead of one, as with the Moz crawler. Instead it will flag up up duplicates present on both site A and B.
Moz Pro | | Blink-SEO0 -
Duplicate page content and title
hi,
Moz Pro | | solutionforweb
i have a serious issue with my site. my website contains 21 pages. but during my weekly report, moz found 84 errors. i.e 42 errors in duplicate page content and 42 errors in duplicate page title... when i see the error in details.. all my 21 links are displaying twice. for example http://domain.com/
http://domain.com/page1.html
http://domain.com/page2.html
and
http://www.domain.com/
http://www.domain.com/page1.html
http://www.domain.com/page2.html so, the same link is repeating twice with www and without www. how do i resolve this error? so please kindly anyone help me....0 -
Duplicate Page Titles & Content
We have just launched a new version of a website and after running it through SEOMOZ we have over 6000 duplicate title & content errors. (awesome) 😕 We have products that show up multiple times under different URLs however we "thought" we had implemented the rel=canonical correctly. My question is - do these errors still show up in SEOMOZ despite the canonical tags being there OR if they were "correct" would we be getting "zero" errors?
Moz Pro | | ZaddleMarketing0 -
Getting rid of duplicate content
Hi everyone, I'm a newbie and at the moment don't know very much about SEO. I have a problem with some of my campaigns where i keep getting a report with either Duplicate Page and/or Duplicate Content errors. I have no idea how to rectify this error, remove it or fix it on the relevant websites. Can anyone please help explain how to do this, maybe step by step? I really appreciate your views and opinions! Regards, Hugh
Moz Pro | | DigitalAcademyZA0 -
Wrong duplicated page content
I found out that some errors on my website are considered as "duplicated page content" while they are not, the content is different on each page. I wonder why ? Is it an issue from Seomoz ?
Moz Pro | | Amadeus_eBC0 -
Frequency of the Campaign reports
Hello there, I would like to know the frequency of the New Rankings and On-page reports for the campaigns that we set up. We received our last campaign report on 3rd September for all the 5 campaigns. We also received 2 reports again on 9th September. Can you please let me know how often do we get these reports and can we manage to receive them every week or fortnight. Best Regards Sandeep
Moz Pro | | CalistonDigital0