Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • SEO Q&A
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Digital Marketers
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • MozCon

        Save on Early Bird tickets and join us in London or New York City

      Unlock flexible pricing & new endpoints
      Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Digital Marketers

        Simplify SEO tasks to save time and grow your traffic.

      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. Moz Tools
    3. Moz Pro
    4. Domain.com and domain.com/index.html duplicate content in reports even with rewrite on

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Domain.com and domain.com/index.html duplicate content in reports even with rewrite on

    Moz Pro
    5
    8
    2030
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • anthonytjm
      anthonytjm last edited by

      I have a site that was recently hit by the Google penguin update and dropped a page back. When running the site through seomoz tools, I keep getting duplicate content in the reports for domain.com and domain.com/index.html, even though I have a 301 rewrite condition. When I test the site, domain.com/index.html redirects to domain.com for all directories and root. I don't understand how my index page can still get flagged as duplicate content.

      I also have a redirect from domain.com to www.domain.com.

      Is there anything else I need to do or add to my htaccess file?

      Appreciate any clarification on this.

      1 Reply Last reply Reply Quote 0
      • SeanKoenig
        SeanKoenig last edited by

        Hello Anthony,

        Saw this still open.

        If your index.html "Rewrite" code is accurate, could the issue be WWW, i.e. http://www.domain.com vs. http://domain.com?

        RewriteCond %{HTTP_HOST} ^domain.com
        RewriteRule ^(.*)$ http://www.domain.com/$1 [R=permanent,L]

        1 Reply Last reply Reply Quote 0
        • Dr-Pete
          Dr-Pete Staff last edited by

          I checked one of your campaigns, and it does seem like the 301-redirect is working properly. I'm also not seeing any evidence of links to the "index.htm" version or other issues. I don't see evidence of both version sin Google's index. Not sure exactly what's going on here, but I'll run it by the support team. I don't think you have cause for concern.

          1 Reply Last reply Reply Quote 1
          • anthonytjm
            anthonytjm @irvingw last edited by

            Thank you for the feedback and help.

            I have looked up url removal in webmaster tools and it states that the page must be removed from the site. If I remove index.html I wont have a home page. Am I understanding you correctly? Heres what google states on url removal.

            To remove a page or image, you must do one of the following:

            • Make sure the content is no longer live on the web. Requests for the page must return an HTTP 404 (not found) or 410 status code.
            • Block the content using a robots.txt file.
            • Block the content using a meta noindex tag.

            Please clarify when you get a moment.

            I would have thought the htaccess 301 redirects from www.domain.com/index.html to www.domain.com would be enough.

            Thank you in advance.

            1 Reply Last reply Reply Quote 0
            • irvingw
              irvingw last edited by

              a) request removal of the /index.html URL in webmaster tools and it will go away in Google's index quickly.

              b) make sure that when you link to your homepage on your site you are not linking to the /index.html URL -  I bet you are somewhere  😉  do a sitewide search in dreamweaver to find all instances and do a global replace.

              anthonytjm 1 Reply Last reply Reply Quote 0
              • AJPro
                AJPro @anthonytjm last edited by

                It could take a little time. I did some redirects myself earlier this year, but the old pages are still in Google's index.

                Maybe someone else can confirm that it can take a little time before the old pages are dropped from Google's index?

                1 Reply Last reply Reply Quote 0
                • anthonytjm
                  anthonytjm @AJPro last edited by

                  HTTP/1.1 301 Moved Permanently => Date => Tue, 08 May 2012 13:44:26 GMT Server => Apache/2.0.52 (CentOS) Location => http://www.domain.com/ Content-Length => 330 Connection => close Content-Type => text/html; charset=iso-8859-1

                  AJPro 1 Reply Last reply Reply Quote 0
                  • AJPro
                    AJPro last edited by

                    Did you verify with a tool like http://www.webconfs.com/http-header-check.php that you get a 301 redirect?

                    anthonytjm 1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    • First post
                      Last post

                    Got a burning SEO question?

                    Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                    Start my free trial


                    Browse Questions

                    Explore more categories

                    • Moz Tools

                      Chat with the community about the Moz tools.

                    • SEO Tactics

                      Discuss the SEO process with fellow marketers

                    • Community

                      Discuss industry events, jobs, and news!

                    • Digital Marketing

                      Chat about tactics outside of SEO

                    • Research & Trends

                      Dive into research and trends in the search industry.

                    • Support

                      Connect on product support and feature requests.

                    • See all categories

                    Related Questions

                    • gokimedia

                      Pages with Duplicate Content Error

                      duplicate content crawl errors seo audit

                      Hello, the result of renewed content appeared in the scan results in my Shopify Store. But these products are unique. Why am I getting this error? Can anyone please help to explain why? screenshot-analytics.moz.com-2021.10.28-19_53_09.png

                      Moz Pro | | gokimedia
                      0
                    • Easigrass

                      Domain still not being found in search

                      Hi guys, I've been handed a client who needs some seo work. I've tweaked one of their pages to focus on a chosen keywords about 4 months back but still the site is not even visible using the new Domain Analysis tool from moz and it still won't rank at all for the keywords. Am I missing something here? Is there something blocking the SERP from listing the website? I've ran a site: search on Google and it returns 283 results on the website. It's puzzling me as there clearly is something stopping it from being ranked. The domain name in question is: https://cloud9inecommunications.co.uk Thanks in advance.

                      Moz Pro | | Easigrass
                      1
                    • AspenFasteners

                      Is one page with long content better than multiple pages with shorter content?

                      (Note, the site links are from a sandbox site and has very low DA or PA) If you look at this page, you will see at the bottom a lengthy article detailing all of the properties of the product categories in the links above. http://www.aspensecurityfasteners.com/Screws-s/432.htm My question is, is there more SEO value in having the one long article in the general product category page, or in breaking up the content and moving the sub-topics as content to the more specific sub-category pages? e.g. http://www.aspensecurityfasteners.com/Screws-Button-Head-Socket-s/1579.htm
                      http://www.aspensecurityfasteners.com/Screws-Cap-Screws-s/331.htm
                      http://www.aspensecurityfasteners.com/Screws-Captive-Panel-Scre-s/1559.htm

                      Moz Pro | | AspenFasteners
                      0
                    • dlmilli

                      My "tag" pages are showing up as duplicate content. Is this harmful?

                      Hi. I ran a Moz sitecrawl. I see "Yes" under "Duplicate Page Content" for each of my tag pages. Is this harmful? If so, how do I fix it? This is a Wordpress site. Tags are used in both the blog and ecommerce sections of the site. Ecommerce is a very small portion. Thank you. |   |

                      Moz Pro | | dlmilli
                      1
                    • GBCweb

                      Dynamic contents causes duplicate pages

                      Technical help required - please!
                      In our Duplicate Content Pages Report I see a lot of duplicate pages that are created by one URL plus several versions of the same page with the dynamic content, for example,
                      http://www.georgebrown.ca/immigranteducation/programs
                      http://www.georgebrown.ca/school-program.aspx?id=1909&Sortid=Study
                      http://www.georgebrown.ca/school-program.aspx?id=1909&Sortid=Term
                      http://www.georgebrown.ca/school-program.aspx?id=1909&Sortid=Certification
                      http://www.georgebrown.ca/school-program.aspx?id=1909&Sortid=Title How do we solve it?

                      Moz Pro | | GBCweb
                      0
                    • thlonius

                      Should I block .ashx files from being indexed ?

                      I got a crawl issue that 82% of site pages have missing title tags
                      All this pages are ashx files (4400 pages). 
                      Should I better removed all this files from google ?

                      Moz Pro | | thlonius
                      0
                    • apo11o177

                      Comparing Domain Authority Scores

                      Since your scale (like PageRank) is a logarithmic scale, it makes it hard to judge the distance between 2 scores.  Can you give me a rule of thumb.  For PageRank,  each jump is an exponential jump - so that a PR6 is perhaps 10 times stronger than a PR5. What is the log base that SEOMoz uses.   Should I assume that a 60 is 10 times stronger than a 50? This is important when it comes to measuring progress because growth is going to get more difficult as you move up the scale and I need to communicate the distance between our current Authority score and our goal. Thank You!

                      Moz Pro | | apo11o177
                      0
                    • MarloSchneider

                      What is mT/mR good for In the keyword tool difficulty full report results?

                      So the ratio is MozTrust to MozRank, but what is this good for?  What can I deduce from this and what can I use it for?

                      Moz Pro | | MarloSchneider
                      2

                    Get started with Moz Pro!

                    Unlock the power of advanced SEO tools and data-driven insights.

                    Start my free trial
                    Products
                    • Moz Pro
                    • Moz Local
                    • Moz API
                    • Moz Data
                    • STAT
                    • Product Updates
                    Moz Solutions
                    • SMB Solutions
                    • Agency Solutions
                    • Enterprise Solutions
                    Free SEO Tools
                    • Domain Authority Checker
                    • Link Explorer
                    • Keyword Explorer
                    • Competitive Research
                    • Brand Authority Checker
                    • Local Citation Checker
                    • MozBar Extension
                    • MozCast
                    Resources
                    • Blog
                    • SEO Learning Center
                    • Help Hub
                    • Beginner's Guide to SEO
                    • How-to Guides
                    • Moz Academy
                    • API Docs
                    About Moz
                    • About
                    • Team
                    • Careers
                    • Contact
                    Why Moz
                    • Case Studies
                    • Testimonials
                    Get Involved
                    • Become an Affiliate
                    • MozCon
                    • Webinars
                    • Practical Marketer Series
                    • MozPod
                    Connect with us

                    Contact the Help team

                    Join our newsletter
                    Moz logo
                    © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                    • Accessibility
                    • Terms of Use
                    • Privacy

                    Looks like your connection to Moz was lost, please wait while we try to reconnect.