Duplicate Content for index.html
-
In the Crawl Diagnostics Summary, it says that I have two pages with duplicate content which are:
I read in a Dream Weaver tutorial that you should name your home page "index.html" and then you can let www.mywebsite.com automatically direct the user to index.html. Is this a bug in SEOMoz's crawler or is it a real problem with my site?
Thank you,
Dan
-
The code should definitely go into the websites root directory's .htaccess, however .htaccess can be weird, a few days ago I ran into a similar issue with a client's website, and I was able to remedy the issue with a variation of the code.
index Redirect RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.(php|html|htm|asp)\ HTTP/ RewriteRule ^(([^/]+/))index.(php|html|htm|asp)$ http://yoursite.com/$1 [R=301,L]
If you give me the URL for the site I will take a look at it and let you know what would be feasible.
-
Hi Daniel, can you share with us the URL of your site? We can take a look at it and give you a more precise answer that way. Thanks!
-
I eventually figured out that your method was a 301 redirect and I definitely broke my site trying to use the code you posted. .. haha. Its ok though. I just removed the code and it went back to normal. At first, I was editing the .htaccess file in the public_html folder which wasnt working. Then I tried the root folder for the site (I created the .htaccess file since it did not exist.) Neither of those worked. (I am using Bluehost so I do not think that I have root access and I am not sure if it is a Linux server or not.)
If there is an easy way to explain what I am doing wrong, please do so. Otherwise, I will use canonical.
Thanks for everything!
-
@Dan
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
sorry about the delay of this response, i didn't realize the that you were asking me a question right away. When placing the code I provided in my previous answer this will cause a 301 perminant redirect to the original URL. That's actually what the
[R=301,L]
portion of the code is stating (R) redirect (301) status is referring to. After reviewing the Matt Cutts video, I realize that I should have asked you if you were operating on a Linux server that you had root access to. We actually utilize both redirects and canonical tags since it was recommended by the on-page optimization reports. Heck Google uses them, I would assume because it's easier for the user to be referred to a single page URL. Obviously though if you don't have server header access, and are not familiar with .htaccess (you can accidentally break your site) then the canonical solution is appropriate
-
Josh,
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
Thanks,
Dan -
use the link rel tag for all my homepages for the http://www.yoursite.com
-
Odd enough I just recently answered this question. The SEOmoz crawler is correct, because without a redirect you will be able to access both versions of the page in your browser.
To resolve this issue simply rewrite the index.html to the root url by placing the following code into your .htaccess file into your root directory.
Options +FollowSymlinks RewriteEngine on
Index Rewrite RewriteRule ^index.(htm|html|php) http://www.yoursite.com/ [R=301,L] RewriteRule ^(.*)/index.(htm|html|php) http://www.yoursite.com/$1/ [R=301,L]
You can also do the same with the index file in any subdirectories that you might create, by simply placing a .htaccess into those sub directories and using variations of the above code. This is how you create nice tight URLs without the duplicate content issue that look like - http://www.semclix.com/design/business/
-
It is a problem which you need to fix. You need to canonicalize your pages.
Those are all various URLs which most likely lead to the same web page. I say "most likely" because these URLs can actually lead to different pages.
You need to tell crawlers and search engines how you organize your site. There are several ways to achieve canonicalization. The method I prefer is to add the following line of code to each page:
The URL provided should be the preferred URL for your page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why did Google Index a weird version of my blog post?
i wrote a page - https://domain.com/how-to-do-xyz/ but when doing an inurl search, i see that it is indexed by google as - https://secureservercdn.net/58584.883848.9834983/myftpupload/how-to-do-xyz/ (not actual url) and when i view that page, it is a weirdly formatted version of the page with many design elements missing. this is a wordpress site. Why would this be? thanks, Ryan
Web Design | | RyanMeighan0 -
We added hundreds of pages to our website & restructured the layout to include 3 additional locations within the sub-pages, same brand/domain name. How long could Google take to crawl/index the new pages and rank the keywords used within those pages?
We added hundreds of pages to our website & restructured the layout to include 3 additional locations within the sub-pages, same brand/domain name. The 3 locations old domains were redirected to their sites within our main brand domain. How long could Google take to crawl/index the new pages and rank the keywords used within those pages? And possibly increase our domain authority hopefully? We didn't want our brand spread out over multiple websites/domains on the internet. This also allowed for more content to be written on pages, per each of our locations service's, as well.
Web Design | | BurgSimpson0 -
Copy partial content to other pages ?
One of our clients looking to redesign their website since we're redesigning the whole website we thought it would be good idea to separate services into individual pages so every service will have it's own page (currently there is 1 page that describes all of the services). what we're planing to do is to write unique content for each service page (about 300-400 keywords), but we also want to use some of the existing content which is kind of explains the process of provided services. so here i need your help! what would be the best practice to use same part of existing content on every service page without getting penalized for duplicated content? here is how we want to structure the page with h1 and h2 <main> Service name (same as page title) Subline new and unique content about 300-400 keywords Part of old content which is going to be placed on every service page </main> any help would be much appreciated!
Web Design | | MozPro30 -
Does anyone know of an easy way to create jump links in WordPress without having to hardcode it into the HTML?
I am using Genesis Framework with the New Pro child theme on my site called Learn Internet Grow and when I create long articles I like to have a menu so readers can jump to a specific section. I wonder if there is a best practice for how to create jump links quickly without having to switch between the code editor and the post editor. There are a bunch of plugins but I am wary of adding more. Maybe if someone can tell me about one they had a good experience with. Ideally, a resource that shows you how to create a custom plugin for this or a non-plugin way to alter the framework to enable easy jump linking.
Web Design | | LearnInternetGrow0 -
Privacy Policy: index it/? And where to place it?
Hi Everyone, Two questions, first: should you allow google to index your privacy policy? Second: for a service based site (not e-commerce, not selling anything) should you put the policy in the footer so it's site wide or just on the "contact us" form page? Best, Ruben
Web Design | | KempRugeLawGroup0 -
Can multiple domains compete with one another if they have the same/similar content?
Could an ecommerce site with a .co.nz nd .com.au domain compete with one another and hard organic rankings if the content of the pages are the same? All the links would be identical (apart from .co.nz or .com.au) the product descriptions, pages titles etc... would all be identical or similar (our page titles are ever so slightly different). Could this be hurting us? Thanks in advance ^ Paul
Web Design | | kevinliao0 -
Will a .com and .co.uk site (with exact same content) hurt seo
hello, i am sure this question has been asked before, but while i tried to search i could not find the right answer. my question is i have a .com and .co.uk site. both sites have exact same product, exact same product descriptions, and everything is the same. the reason for 2 sites is that .com site shows all the details for US customers and in $, and .co.uk site shows all the details to UK customers and with Pound signs. the only difference in the 2 sites might be the privacy policy (different for US and UK) and different membership groups the site belongs to (US site belong to a list of US trade groups, UK belongs to a list of UK trade groups). my question is other than the minor difference above, all the content of the site is exactly the same, so will this hurt seo for either one or both the site. Our US site much more popular and indexed already in google for 4 years, while our UK site was just started 1 month ago. (also both the sites are hosted by same hosting company, with one site as main domain and the other site as domain addon (i thought i include this information also, if it makes sense to readers)) i would appreciate a reply to the question above thanks
Web Design | | kannu10 -
How can i write content rich descriptions?
we have recently started using seomoz. how can i make descriptions more content rich?
Web Design | | WCGAdmin0