Domain.com and domain.com/index.html duplicate content in reports even with rewrite on
-
I have a site that was recently hit by the Google penguin update and dropped a page back. When running the site through seomoz tools, I keep getting duplicate content in the reports for domain.com and domain.com/index.html, even though I have a 301 rewrite condition. When I test the site, domain.com/index.html redirects to domain.com for all directories and root. I don't understand how my index page can still get flagged as duplicate content.
I also have a redirect from domain.com to www.domain.com.
Is there anything else I need to do or add to my htaccess file?
Appreciate any clarification on this.
-
Hello Anthony,
Saw this still open.
If your index.html "Rewrite" code is accurate, could the issue be WWW, i.e. http://www.domain.com vs. http://domain.com?
RewriteCond %{HTTP_HOST} ^domain.com
RewriteRule ^(.*)$ http://www.domain.com/$1 [R=permanent,L] -
I checked one of your campaigns, and it does seem like the 301-redirect is working properly. I'm also not seeing any evidence of links to the "index.htm" version or other issues. I don't see evidence of both version sin Google's index. Not sure exactly what's going on here, but I'll run it by the support team. I don't think you have cause for concern.
-
Thank you for the feedback and help.
I have looked up url removal in webmaster tools and it states that the page must be removed from the site. If I remove index.html I wont have a home page. Am I understanding you correctly? Heres what google states on url removal.
To remove a page or image, you must do one of the following:
- Make sure the content is no longer live on the web. Requests for the page must return an HTTP 404 (not found) or 410 status code.
- Block the content using a robots.txt file.
- Block the content using a meta noindex tag.
Please clarify when you get a moment.
I would have thought the htaccess 301 redirects from www.domain.com/index.html to www.domain.com would be enough.
Thank you in advance.
-
a) request removal of the /index.html URL in webmaster tools and it will go away in Google's index quickly.
b) make sure that when you link to your homepage on your site you are not linking to the /index.html URL - I bet you are somewhere do a sitewide search in dreamweaver to find all instances and do a global replace.
-
It could take a little time. I did some redirects myself earlier this year, but the old pages are still in Google's index.
Maybe someone else can confirm that it can take a little time before the old pages are dropped from Google's index?
-
HTTP/1.1 301 Moved Permanently => Date => Tue, 08 May 2012 13:44:26 GMT Server => Apache/2.0.52 (CentOS) Location => http://www.domain.com/ Content-Length => 330 Connection => close Content-Type => text/html; charset=iso-8859-1
-
Did you verify with a tool like http://www.webconfs.com/http-header-check.php that you get a 301 redirect?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
OSE - advanced reports.
Trying to run advanced reports, but am being told I've hit my daily report limited - and points me to signing up for a Pro membership - although I already have one. Been like this the last couple of days. Anyone else experiencing this? Thanks.
Moz Pro | | newstd1000 -
Domain authority drop
Hi SEOmoz gurus, The last 2 months I am seeing a drop in the Domain authority for some of my clients. Some of them are old sites. Some are new (less than 6 months old) For the old sites I thought the reason for the drop might be caused by discounting the value of low quality backlinks. But this cannot be the case for the new sites - they have high quality backlinks from newspapers and I can't make any sense of the drop. Has anyone else experienced the same issue? Might this be caused by a change in SEOmoz algorithm? Thanks!
Moz Pro | | ParisChildress0 -
Duplicate conent
I found some duplicate content on my site.... but in my SEOMOZ campaigns account. It says that I do not have duplicate content... what is wrong? is the tool working ?
Moz Pro | | sandyallain0 -
Duplicate Content Issue because of root domain and index.html
SEOMoz crawl diagnostics is suggesting that my root domain and the rootdomain/index.html are duplicate content. What can be done to ensure that both are considered as a single age only?
Moz Pro | | h1seo1 -
More complete campaign reports
SEOMoz campaigns include a lot more data than can be sent via email automatically with the custom reporting feature. So what do people do to send that data to clients? Do you not include it? Or export reports manually and send them on? Or something else?
Moz Pro | | antdesign0 -
Last Linkscape index update: 05/30/2012
When will the next Linkscape index update occur? I've been waiting to run our backlink profile numbers, but the Last Linkscape index update was 05/30/2012? Thanks!
Moz Pro | | larahill0 -
Duplicate Page Content
i getting crewl errors on Duplicate Page Title and content for the same page. www.breeze-air.com www.breeze-air.com/ www.breeze-air.com/index-html what am i doing worng? please help thank you
Moz Pro | | eoberlender0 -
Frequency of the Campaign reports
Hello there, I would like to know the frequency of the New Rankings and On-page reports for the campaigns that we set up. We received our last campaign report on 3rd September for all the 5 campaigns. We also received 2 reports again on 9th September. Can you please let me know how often do we get these reports and can we manage to receive them every week or fortnight. Best Regards Sandeep
Moz Pro | | CalistonDigital0