undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • MozCon
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Digital Marketers
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    Let your business shine with Listings AI
    Moz Local

    Let your business shine with Listings AI

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • MozCon

      Save on Early Bird tickets and join us in London or New York City

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Digital Marketers

      Simplify SEO tasks to save time and grow your traffic.

    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. Digital Marketing
  3. Web Design
  4. Duplicate Content for index.html

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Duplicate Content for index.html

Web Design
5
9
4.9k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • superTallDan
    superTallDan last edited by May 29, 2011, 6:01 PM

    In the Crawl Diagnostics Summary, it says that I have two pages with duplicate content which are:

    www.mywebsite.com/

    www.mywebsite.com/index.html

    I read in a Dream Weaver tutorial that you should name your home page "index.html" and then you can let www.mywebsite.com automatically direct the user to index.html.  Is this a bug in SEOMoz's crawler or is it a real problem with my site?

    Thank you,

    Dan

    1 Reply Last reply Reply Quote 0
    • SEMCLIX
      SEMCLIX @superTallDan last edited by Jul 1, 2011, 7:41 PM Jul 1, 2011, 7:41 PM

      The code should definitely go into the websites root directory's .htaccess, however .htaccess can be weird, a few days ago I ran into a similar issue with a client's website, and I was able to remedy the issue with a variation of the code.

      index Redirect RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.(php|html|htm|asp)\ HTTP/ RewriteRule ^(([^/]+/))index.(php|html|htm|asp)$ http://yoursite.com/$1 [R=301,L]

      If you give me the URL for the site I will take a look at it and let you know what would be feasible.

      1 Reply Last reply Reply Quote 0
      • KeriMorgret
        KeriMorgret @superTallDan last edited by Jun 30, 2011, 8:47 PM Jun 30, 2011, 8:47 PM

        Hi Daniel, can you share with us the URL of your site? We can take a look at it and give you a more precise answer that way. Thanks!

        1 Reply Last reply Reply Quote 0
        • superTallDan
          superTallDan @SEMCLIX last edited by Jun 30, 2011, 4:28 PM Jun 30, 2011, 4:28 PM

          I eventually figured out that your method was a 301 redirect and I definitely broke my site trying to use the code you posted. .. haha.  Its ok though.  I just removed the code and it went back to normal.  At first, I was editing the .htaccess file in the public_html folder which wasnt working.  Then I tried the root folder for the site (I created the .htaccess file since it did not exist.)  Neither of those worked.  (I am using Bluehost so I do not think that I have root access and I am not sure if it is a Linux server or not.)

          If there is an easy way to explain what I am doing wrong, please do so.  Otherwise, I will use canonical.

          Thanks for everything!

          KeriMorgret SEMCLIX 2 Replies Last reply Jul 1, 2011, 7:41 PM Reply Quote 0
          • SEMCLIX
            SEMCLIX last edited by Jun 28, 2011, 12:43 AM Jun 28, 2011, 12:43 AM

            @Dan

            Thanks for your reply.  It seems like there are lots of different ways to solve this problem.  I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.

            Where would you say your solution fits in?

            sorry about the delay of this response, i didn't realize the that you were asking me a question right away. When placing the code I provided in my previous answer this will cause a 301 perminant redirect to the original URL. That's actually what the

            [R=301,L]

            portion of the code is stating (R) redirect (301) status is referring to. After reviewing the Matt Cutts video, I realize that I should have asked you if you were operating on a Linux server  that you had root access to. We actually utilize both redirects and canonical tags since it was recommended by the on-page optimization reports. Heck Google uses them, I would assume because it's easier for the user to be referred to a single page URL. Obviously though if you don't have server header access, and are not familiar with .htaccess (you can accidentally break your site) then the canonical solution is appropriate

            superTallDan 1 Reply Last reply Jun 30, 2011, 4:28 PM Reply Quote 0
            • topic:timeago_earlier,28 days
            • superTallDan
              superTallDan @SEMCLIX last edited by May 31, 2011, 8:59 AM May 31, 2011, 8:59 AM

              Josh,

              Thanks for your reply.  It seems like there are lots of different ways to solve this problem.  I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.

              Where would you say your solution fits in?

              Thanks,
              Dan

              watch?v=zW5UL3lzBOA

              1 Reply Last reply Reply Quote 0
              • usaccess608
                usaccess608 last edited by May 29, 2011, 7:57 PM May 29, 2011, 7:57 PM

                use the link rel tag for all my homepages for the http://www.yoursite.com

                1 Reply Last reply Reply Quote 0
                • SEMCLIX
                  SEMCLIX last edited by Jun 25, 2011, 11:36 AM May 29, 2011, 6:48 PM

                  Odd enough I just recently answered this question. The SEOmoz crawler is correct, because without a redirect you will be able to access both versions of the page in your browser.

                  www.yoursite.com/

                  www.yoursite.com/index.html

                  To resolve this issue simply rewrite the index.html to the root url by placing the following code into your .htaccess file into your root directory.

                  Options +FollowSymlinks                                                                                                            RewriteEngine on

                  Index Rewrite                                                                                                                                RewriteRule ^index.(htm|html|php) http://www.yoursite.com/ [R=301,L]                                      RewriteRule ^(.*)/index.(htm|html|php) http://www.yoursite.com/$1/ [R=301,L]

                  You can also do the same with the index file in any subdirectories that you might create, by simply placing a .htaccess into those sub directories and using variations of the above code. This is how you create nice tight URLs without the duplicate content issue that look like - http://www.semclix.com/design/business/

                  superTallDan 1 Reply Last reply May 31, 2011, 8:59 AM Reply Quote 2
                  • RyanKent
                    RyanKent last edited by May 29, 2011, 6:32 PM May 29, 2011, 6:32 PM

                    It is a problem which you need to fix. You need to canonicalize your pages.

                    www.mywebsite.com

                    www.mywebsite.com/

                    www.mywebsite.com/index.html

                    mywebsite.com/

                    mywebsite.com/index.html

                    Those are all various URLs which most likely lead to the same web page. I say "most likely" because these URLs can actually lead to different pages.

                    You need to tell crawlers and search engines how you organize your site. There are several ways to achieve canonicalization. The method I prefer is to add the following line of code to each page:

                    The URL provided should be the preferred URL for your page.

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    1 out of 9
                    • First post
                      1/9
                      Last post

                    Got a burning SEO question?

                    Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                    Start my free trial


                    Browse Questions

                    Explore more categories

                    • Moz Tools

                      Chat with the community about the Moz tools.

                    • SEO Tactics

                      Discuss the SEO process with fellow marketers

                    • Community

                      Discuss industry events, jobs, and news!

                    • Digital Marketing

                      Chat about tactics outside of SEO

                    • Research & Trends

                      Dive into research and trends in the search industry.

                    • Support

                      Connect on product support and feature requests.

                    • See all categories

                    Related Questions

                    • vtmoz

                      Any risks involved in removing a sub-domain from search index or completely taking down? Ranking impact?

                      Hi all, One of our sub-domains has thousands of indexed pages but traffic is very less and irrelevant. There are links between this sub-domain to other sub domains of ours. We are planning  to take this subdomain completely. What happens if so? Google responds for this with a ranking change? Thanks

                      Web Design | Apr 8, 2018, 8:21 AM | vtmoz
                      0
                    • Sam.at.Moz

                      Hiding content until user scrolls - Will Google penalize me?

                      I've used: "opacity:0;" to hide sections of my content, which are triggered to show (using Javascript) once the user scrolls over these sections. I remember reading a while back that Google essentially ignores content which is hidden from your page (it mentioned they don't index it, so it's close to impossible to rank for it). Is this still the case? Thanks, Sam

                      Web Design | Nov 29, 2017, 5:11 PM | Sam.at.Moz
                      0
                    • Dino64

                      Duplicate Content Issue: Mobile vs. Desktop View

                      Setting aside my personal issue with Google's favoritism for Responsive websites, which I believe doesn't always provide the best user experience, I have a question regarding duplicate content... I created a section of a Wordpress web page (using Visual Composer) that shows differently on mobile than it does on desktop view. This section has the same content for both views, but is formatted differently to give a better user experience on mobile devices. I did this by creating two different text elements, formatted differently, but containing the same content. The problem is that both sections appear in the source code of the page. According to Google, does that mean I have duplicate content on this page?

                      Web Design | Sep 8, 2015, 11:49 PM | Dino64
                      0
                    • J_Sinclair

                      Above the Fold Content - Use of large images

                      Hi All, Our designers have come to the SEO team to ask if have a large image across the top of the page taking up a large majority of the above the fold real estate will impact our SEO. Our initial thoughts are no as long as we have an optimised H1 visibal to the user landing there which informs them what the page is about. Any thoughts would be appreciated.

                      Web Design | Aug 1, 2014, 2:08 PM | J_Sinclair
                      1
                    • Kingalan1

                      Lots of Listing Pages with Thin Content on Real Estate Web Site-Best to Set them to No-Index?

                      Greetings Moz Community: As a commercial real estate broker in Manhattan I run a web site with over 600 pages. Basically the pages are organized in the following categories: 1. Neighborhoods (Example:http://www.nyc-officespace-leader.com/neighborhoods/midtown-manhattan)  25 PAGES Low bounce rate 2. Types of Space (Example:http://www.nyc-officespace-leader.com/commercial-space/loft-space) 
                      15 PAGES Low bounce rate. 3. Blog (Example:http://www.nyc-officespace-leader.com/blog/how-long-does-leasing-process-take 
                      30 PAGES Medium/high bounce rate 4. Services (Example:http://www.nyc-officespace-leader.com/brokerage-services/relocate-to-new-office-space)  High bounce rate
                      3 PAGES 5. About Us (Example:http://www.nyc-officespace-leader.com/about-us/what-we-do
                      4 PAGES High bounce rate 6. Listings (Example:http://www.nyc-officespace-leader.com/listings/305-fifth-avenue-office-suite-1340sf)
                      300 PAGES High bounce rate (65%), thin content 7. Buildings (Example:http://www.nyc-officespace-leader.com/928-broadway
                      300 PAGES  Very high bounce rate (exceeding 75%) Most of the listing pages do not have more than 100 words.  My SEO firm is advising me to set them "No-Index, Follow". They believe the thin content could be hurting me. Is this an acceptable strategy? I am concerned that when Google detects 300 pages set to "No-Follow" they could interpret this as the site seeking to hide something and penalize us. Also, the building pages have a low click thru rate. Would it make sense to set them to "No-Follow" as well? Basically, would it increase authority in Google's eyes if we set pages that have thin content and/or low click thru rates to "No-Follow"? Any harm in doing this for about half the pages on the site? I might add that while I don't suffer from any manual penalty volume has gone down substantially in the last month. We upgraded the site in early June and somehow 175 pages were submitted to Google  that should not have been indexed. A removal request has been made for those pages. Prior to that we were hit by Panda in April 2012 with search volume dropping from about 7,000 per month to 3,000 per month. Volume had increased back to 4,500 by April this year only to start tanking again. It was down to 3,600 in June. About 30 toxic links were removed in late April and a disavow file was submitted with Google in late April for removal of links from 80 toxic domains. Thanks in advance for your responses!! Alan

                      Web Design | Jul 3, 2014, 5:09 PM | Kingalan1
                      0
                    • Atlanta-SMO

                      ECWID How to fix Duplicate page content and external link issue

                      I am working on a site that has a HUGE number of duplicate pages due to ECWID ecommerce platform.  The site is built with Joomla!  How can I rectify this situation?  The pages also show up as "external " links on crawls...  Is it the ECWID platform?  I have never worked on a site that uses this. Here is an example of a page with the issue (there are 6280 issues) URL: http://www.metroboltmi.com/shop-spare-parts?Itemid=218&option=com_rokecwid&view=ecwid&ecwid_category_id=3560081

                      Web Design | Jul 19, 2013, 3:56 PM | Atlanta-SMO
                      0
                    • BobGW

                      How to put 'Link to this article' HTML code at bottom of article & is it helpful?

                      Hello, I was thinking about putting a box down at the bottom of my client's main articles that let's the reader easily copy the html code it takes to link to the article they're reading.  Maybe I'd put it after the author bio. Do any of you do this? If so, what format do you use? It has to look nice of course. This is a non-techie industry. Thanks.

                      Web Design | Jan 11, 2013, 4:26 AM | BobGW
                      0
                    • sftravel

                      Custom 404 Page Indexing

                      Hi - We created a custom 404 page based on SEOMoz recommendations.  But.... the page seems to be receiving traffic via organic search.  Does it make more sense to set this page as "noindex" by its metatag?

                      Web Design | May 18, 2012, 7:20 PM | sftravel
                      0

                    Get started with Moz Pro!

                    Unlock the power of advanced SEO tools and data-driven insights.

                    Start my free trial
                    Products
                    • Moz Pro
                    • Moz Local
                    • Moz API
                    • Moz Data
                    • STAT
                    • Product Updates
                    Moz Solutions
                    • SMB Solutions
                    • Agency Solutions
                    • Enterprise Solutions
                    Free SEO Tools
                    • Domain Authority Checker
                    • Link Explorer
                    • Keyword Explorer
                    • Competitive Research
                    • Brand Authority Checker
                    • Local Citation Checker
                    • MozBar Extension
                    • MozCast
                    Resources
                    • Blog
                    • SEO Learning Center
                    • Help Hub
                    • Beginner's Guide to SEO
                    • How-to Guides
                    • Moz Academy
                    • API Docs
                    About Moz
                    • About
                    • Team
                    • Careers
                    • Contact
                    Why Moz
                    • Case Studies
                    • Testimonials
                    Get Involved
                    • Become an Affiliate
                    • MozCon
                    • Webinars
                    • Practical Marketer Series
                    • MozPod
                    Connect with us

                    Contact the Help team

                    Join our newsletter
                    Moz logo
                    © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                    • Accessibility
                    • Terms of Use
                    • Privacy

                    Looks like your connection to Moz was lost, please wait while we try to reconnect.