The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Forum
    2. Categories
    3. SEO Tactics
    4. Intermediate & Advanced SEO
    5. If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

    Intermediate & Advanced SEO
    3 3 821
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Gabriele_Layoutweb
      Gabriele_Layoutweb last edited by

      If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

      1 Reply Last reply Reply Quote 0
      • GastonRiera
        Gaston Riera last edited by

        Hello Jaro,

        What Andy says is right, im backing him up. Remember to not include that URL in the sitemap.

        Also is a good moment to say that with the robots.txt you just tell google bot not no follow it, that differs from indexing it. There are cases where URLs are indexed instead of being "blocked" in the robots.txt.

        The fine way stop google from indexing a certain URL would be adding the meta robots tag including a noindex atribute.
        Here there a quote from the Webmaster central help forum in Google:

        If you block a file from crawling and Google discovers a URL for that file on another site, it may still index the file using whatever information it can find, even though crawling is blocked. So robots.txt disallow does not necessarily stop something being indexed.
        (in the ets answer, a note below the Disallow part)

        Hope it's clarifying.
        Best luck.
        GR.

        1 Reply Last reply Reply Quote 2
        • Andy.Drinkwater
          Andy.Drinkwater last edited by

          Hi Jaro,

          Head into search console and use the Temporary Remove URL tool - this should work pretty quickly. The next time Google comes around to that page, they should see the NOINDEX flag and not re-index it.

          -Andy

          1 Reply Last reply Reply Quote 3
          • 1 / 1
          • First post
            Last post

          Got a burning SEO question?

          Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


          Start my free trial


          Explore more categories

          • Moz Tools

            Chat with the community about the Moz tools.

            Getting Started
            Moz Pro
            Moz Local
            Moz Bar
            API
            What's New

          • SEO Tactics

            Discuss the SEO process with fellow marketers

            Content Development
            Competitive Research
            Keyword Research
            Link Building
            On-Page Optimization
            Technical SEO
            Reporting & Analytics
            Intermediate & Advanced SEO
            Image & Video Optimization
            International SEO
            Local SEO

          • Community

            Discuss industry events, jobs, and news!

            Moz Blog
            Moz News
            Industry News
            Jobs and Opportunities
            SEO Learn Center
            Whiteboard Friday

          • Digital Marketing

            Chat about tactics outside of SEO

            Affiliate Marketing
            Branding
            Conversion Rate Optimization
            Web Design
            Paid Search Marketing
            Social Media

          • Research & Trends

            Dive into research and trends in the search industry.

            SERP Trends
            Search Behavior
            Algorithm Updates
            White Hat / Black Hat SEO
            Other SEO Tools

          • Support

            Connect on product support and feature requests.

            Product Support
            Feature Requests
            Participate in User Research

          • See all categories

          • Block session id URLs with robots.txt
            Mat_C
            Mat_C
            1
            4
            972

          • My url disappeared from Google but Search Console shows indexed. This url has been indexed for more than a year. Please help!
            Davit1985
            Davit1985
            0
            3
            1.1k

          • How to stop URLs that include query strings from being indexed by Google
            McTaggart
            McTaggart
            0
            2
            8.4k

          • If Robots.txt have blocked an Image (Image URL) but the other page which can be indexed has this image, how is the image treated?
            Malika1
            Malika1
            1
            4
            2.7k

          • Blocking Dynamic URLs with Robots.txt
            AndrewY
            AndrewY
            1
            4
            6.9k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy

          Looks like your connection to Moz was lost, please wait while we try to reconnect.