undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • MozCon

      Save on Early Bird tickets and join us in London or New York City

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. SEO Tactics
  3. How can I make a list of all URLs indexed by Google?

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

How can I make a list of all URLs indexed by Google?

SEO Tactics
redirect indexed urls google url index google broken links
6
7
4.5k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • aplusnetsolutions
    aplusnetsolutions Subscriber last edited by Feb 9, 2022, 7:48 PM

    I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.

    BlueprintMarketing SHIJOTACS 3 Replies Last reply Feb 11, 2022, 10:23 PM Reply Quote 0
    • Kaushalwpandas
      Kaushalwpandas last edited by Dec 11, 2023, 4:12 PM

      very depressed of being not indexing URL - https://www.wrightleatherworks.com/collections/gun-holsters-for-women please someone help

      1 Reply Last reply Reply Quote 0
      • topic:timeago_earlier,2 months
      • Natalie305
        Natalie305 last edited by Oct 9, 2023, 4:12 PM

        Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

        1 Reply Last reply Reply Quote 1
        • Luna204
          Luna204 last edited by Oct 6, 2023, 4:53 PM

          Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

          1 Reply Last reply Reply Quote 0
          • topic:timeago_earlier,2 years
          • SHIJOTACS
            SHIJOTACS @aplusnetsolutions last edited by Feb 11, 2022, 10:23 PM

            @aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also

            1 Reply Last reply Reply Quote 0
            • BlueprintMarketing
              BlueprintMarketing @aplusnetsolutions last edited by Feb 11, 2022, 5:13 PM

              @aplusnetsolutions
              Make sure all these things are selected in screaming frog and crawl analysis selected

              1 Reply Last reply Reply Quote 1
              • BlueprintMarketing
                BlueprintMarketing @aplusnetsolutions last edited by Feb 11, 2022, 5:10 PM

                @aplusnetsolutions

                Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want

                #1
                By far the easiest is to going to search console and select coverage and all known pages
                alt text

                number 2

                For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”

                #3

                You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
                more on how this is done

                ![ you can see the non-indexable URLs on the the right click that](77e61d47-2172-491c-8070-9a6cd5905fff-image.png image url)

                video https://youtu.be/iYeXSdUt_hg

                #4
                or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
                https://www.rankwatch.com/free-tools/google-index-checker

                https://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472

                https://seo.co/google-index/

                1 Reply Last reply Reply Quote 0
                • 1 / 1
                1 out of 7
                • First post
                  1/7
                  Last post

                Got a burning SEO question?

                Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                Start my free trial


                Browse Questions

                Explore more categories

                • Moz Tools

                  Chat with the community about the Moz tools.

                • SEO Tactics

                  Discuss the SEO process with fellow marketers

                • Community

                  Discuss industry events, jobs, and news!

                • Digital Marketing

                  Chat about tactics outside of SEO

                • Research & Trends

                  Dive into research and trends in the search industry.

                • Support

                  Connect on product support and feature requests.

                • See all categories

                Related Questions

                • worldviajar.com

                  How can I check my website is not in spam?

                  keyword ranking keywords

                  I have a blogging website where I post about famous food, home remedies, and more. When I started my website's keywords were ranking on Google But Now a single keyword is not in the ranking list. That's why I have concerns about how I can fix it.

                  SEO Tactics | Oct 29, 2024, 10:33 AM | worldviajar.com
                  0
                • TimoRossa

                  Google doesn't show my Twitter account

                  google search ranking issue

                  Hello, my full name is Timo Rossa and my Twitter (X) account is @TimoRossa. If I search for my name with "Timo Rossa" on Google, it doesn't find any results referencing my Twitter account. It is very important for me that Google does not only show Twitter results for my name but also that those results would be ranked at the top. The only reason I could come up with is that my account name has not separated words. Does this make sense? What would be a simple strategy to achieve my goal? Thank you!

                  SEO Tactics | Oct 7, 2023, 12:02 PM | TimoRossa
                  0
                • xShams

                  Google is ranking only home page not even considering to rank other pages!

                  ranking homepage sub-pages not ranking

                  Hey Community! I am facing an ranking issue, I am trying to rank on multiple quires with my sub-pages. The issue is homepage is ranking on every single query. Even the homepage's content is not relevant to that query! I have made sure to remove any sort of content relevancy from homepage which i trying to rank with my sub-pages. Business/Website Details: We are a recruitment agency based Pakistan providing manpower around the globe and other recruiting services. We are targeting each country with a sub-page. Even after I have done some necessary things but still there is not effect on rankings. Let me share some examples: Query: recruitment agency for gulf in pakistan Home Page Showing
                  1.jpg Page i want to ranking:
                  2.jpg This issue is not only with this page, it's appearing on multiple quires but I think it should clear the issue I am facing. Now: Any Possible Solutions to this technical or ranking error. Backlinks on this query don't matter because you can check the other search results don't have any backlink data on them. Please share some quick thoughts on-page content. Even I make a page that have the word recruiting/recruitment in it google will automatically rank homepage and not the page i want! Thanks in advance for help 🙂

                  SEO Tactics | May 30, 2024, 8:33 PM | xShams
                  0
                • BakeryTech

                  Unsolved Using NoIndex Tag instead of 410 Gone Code on Discontinued products?

                  ecommerce noindex shopify indexed urls

                  Hello everyone, I am very new to SEO and I wanted to get some input & second opinions on a workaround I am planning to implement on our Shopify store. Any suggestions, thoughts, or insight you have are welcome & appreciated! For those who aren't aware, Shopify as a platform doesn't allow us to send a 410 Gone Code/Error under any circumstance. When you delete or archive a product/page, it becomes unavailable on the storefront. Unfortunately, the only thing Shopify natively allows me to do is set up a 301 redirect. So when we are forced to discontinue a product, customers currently get a 404 error when trying to go to that old URL. My planned workaround is to automatically detect when a product has been discontinued and add the NoIndex meta tag to the product page. The product page will stay up but be unavailable for purchase. I am also adjusting the LD+JSON to list the products availability as Discontinued instead of InStock/OutOfStock.
                  Then I let the page sit for a few months so that crawlers have a chance to recrawl and remove the page from their indexes. I think that is how that works?
                  Once 3 or 6 months have passed, I plan on archiving the product followed by setting up a 301 redirect pointing to our internal search results page. The redirect will send the to search with a query aimed towards similar products. That should prevent people with open tabs, bookmarks and direct links to that page from receiving a 404 error. I do have Google Search Console setup and integrated with our site, but manually telling google to remove a page obviously only impacts their index. Will this work the way I think it will?
                  Will search engines remove the page from their indexes if I add the NoIndex meta tag after they have already been index?
                  Is there a better way I should implement this? P.S. For those wondering why I am not disallowing the page URL to the Robots.txt, Shopify won't allow me to call collection or product data from within the template that assembles the Robots.txt. So I can't automatically add product URLs to the list.

                  Technical SEO | Aug 2, 2023, 5:11 PM | BakeryTech
                  0
                • Leviiii

                  slug Link redirect to subdomain?

                  redirect subdomain link building

                  Hi !
                  Im Levi new here and new in the world of SEO, please dont judge if my questions are silly. Back on the days when the site was built we thought it is a good ideea to have subdomains that together with the domain name represent our main keywords.
                  ex. https://stansted.tonorwich.uk, https://heathrow.tonorwich.uk, https://luton.tonorwich.uk, https://gatwick.tonorwich.uk. There is content on this subdomains, would it make any difference from SEO perspective if we create slugs that redirect to these subdomains? for example creating https://tonorwich.uk/taxi-minibus-vip-tesla-norwich-to-stansted that redirects to https://stansted.tonorwich.uk ? Or better create these slugs with slightly different content?
                  Any ideeas would be appreciated.
                  Thanks in advance!

                  Link Building | Nov 11, 2022, 2:58 AM | Leviiii
                  0
                • BeKonstructive

                  Multiple URLs from One Domain on Page 1

                  multiple urls

                  Is it possible (and if it is possible, how difficult is it) to get multiple URLs from the same domain name to be indexed on the first page of Google for the same keyword or phrase? Assuming each URL has significantly different content of course... Context: A large real estate brand has 6 franchise offices in the same city. Each office has its own listing/landing page on the parent brand website. Each franchise owner wants their page on the website to rank for the term 'Christchurch real estate'. The homepage of the parent brand website currently ranks on the first page of Google for this search term, but none of the franchisee pages do. So my question is: with the right unique content on each franchisee page, supported by quality backlinks to each of the different offices, is it possible to get multiple franchisees listed on the first page of Google given that ultimately the 6 URLs are all attached to the same domain name? (And, if so, do you have any hot tips you can share to assist me on this uphill battle?)

                  SEO Tactics | Aug 1, 2022, 3:26 PM | BeKonstructive
                  0
                • Kate_Nadeau

                  What is the difference between "document" and "object" moved redirect errors?

                  redirect chain redirect

                  What is the difference between "document" and "object" moved redirect errors? I'm used to see "object moved" as a redirect chain issue that needs to be fixed, but this week my report contained a "document moved" redirect chain issue. And it's on our homepage. Looks like it might be a HTTP versus an HTTPS issue.

                  Reporting & Analytics | Jun 22, 2022, 8:29 PM | Kate_Nadeau
                  0
                • fJ66doneOIdDpj

                  Multilang site: Auto redirect 301 or 302?

                  redirect 301 redirect 302 international seo

                  We need to establish if 301 or 302 response code is to be used for our auto redirects based on Accept-Language header. https://domain.com
                  30x > https://domain.com/en
                  30x > https://domain.com/ru
                  30x > https://domain.com/de The site architecture is set up with proper inline HREFLANG.
                  We have read different opinions about this, Ahrefs says 302 is the correct one:
                  https://ahrefs.com/blog/301-vs-302-redirects/
                  302 redirect:
                  "You want to redirect users to the right version of the site for them (based on location/language)." You could argue that the root redirect is never permanent as it varies based on user language settings (302)
                  On the other hand, the lang specific redirects are permanent per language: IF Accept-Language header = en
                  https://domain.com > 301 > https://domain.com/en
                  IF Accept-Language header = ru
                  https://domain.com > 301 > https://domain.com/ru So each of these is 'permanent'. So which is the correct?

                  International SEO | Oct 11, 2022, 2:10 AM | fJ66doneOIdDpj
                  0

                Get started with Moz Pro!

                Unlock the power of advanced SEO tools and data-driven insights.

                Start my free trial
                Products
                • Moz Pro
                • Moz Local
                • Moz API
                • Moz Data
                • STAT
                • Product Updates
                Moz Solutions
                • SMB Solutions
                • Agency Solutions
                • Enterprise Solutions
                Free SEO Tools
                • Domain Authority Checker
                • Link Explorer
                • Keyword Explorer
                • Competitive Research
                • Brand Authority Checker
                • Local Citation Checker
                • MozBar Extension
                • MozCast
                Resources
                • Blog
                • SEO Learning Center
                • Help Hub
                • Beginner's Guide to SEO
                • How-to Guides
                • Moz Academy
                • API Docs
                About Moz
                • About
                • Team
                • Careers
                • Contact
                Why Moz
                • Case Studies
                • Testimonials
                Get Involved
                • Become an Affiliate
                • MozCon
                • Webinars
                • Practical Marketer Series
                • MozPod
                Connect with us

                Contact the Help team

                Join our newsletter
                Moz logo
                © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                • Accessibility
                • Terms of Use
                • Privacy

                Looks like your connection to Moz was lost, please wait while we try to reconnect.