undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • MozCon

      Save on Early Bird tickets and join us in London or New York City

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. SEO Tactics
  3. Intermediate & Advanced SEO
  4. E-Commerce Site Collection Pages Not Being Indexed

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

E-Commerce Site Collection Pages Not Being Indexed

Intermediate & Advanced SEO
2
4
1.5k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • Ben-R
    Ben-R Subscriber last edited by May 19, 2017, 3:28 PM

    Hello Everyone,

    So this is not really my strong suit but I’m going to do my best to explain the full scope of the issue and really hope someone has any insight. We have an e-commerce client (can't really share the domain) that uses Shopify; they have a large number of products categorized by Collections. The issue is when we do a site:search of our Collection Pages (site:Domain.com/Collections/) they don’t seem to be indexed.

    Also, not sure if it’s relevant but we also recently did an over-hall of our design. Because we haven’t been able to identify the issue here’s everything we know/have done so far:

    • Moz Crawl Check and the Collection Pages came up.
    • Checked Organic Landing Page Analytics (source/medium: Google) and the pages are getting traffic.
    • Submitted the pages to Google Search Console.
    • The URLs are listed on the sitemap.xml but when we tried to submit the Collections sitemap.xml to Google Search Console 99 were submitted but nothing came back as being indexed (like our other pages and products).
    • We tested the URL in GSC’s robots.txt tester and it came up as being “allowed” but just in case below is the language used in our robots:
      User-agent: *
      Disallow: /admin
      Disallow: /cart
      Disallow: /orders
      Disallow: /checkout
      Disallow: /9545580/checkouts
      Disallow: /carts
      Disallow: /account
      Disallow: /collections/+
      Disallow: /collections/%2B
      Disallow: /collections/%2b
      Disallow: /blogs/+
      Disallow: /blogs/%2B
      Disallow: /blogs/%2b
      Disallow: /design_theme_id
      Disallow: /preview_theme_id
      Disallow: /preview_script_id
      Disallow: /apple-app-site-association
      Sitemap: https://domain.com/sitemap.xml
    • A Google Cache:Search currently shows a collections/all page we have up that lists all of our products.

    Please let us know if there’s any other details we could provide that might help.

    Any insight or suggestions would be very much appreciated.

    Looking forward to hearing all of your thoughts!

    Thank you in advance.

    Best,

    1 Reply Last reply Reply Quote 0
    • Nigel_Carr
      Nigel_Carr last edited by May 22, 2017, 6:39 AM May 22, 2017, 6:39 AM

      It seems odd to deal with filtering here. I'd normally do that in Search Console under URL parameters but you have to be extremely careful altering stuff in there. If you email me the site I'll run a check on why Collections aren't showing.

      1 Reply Last reply Reply Quote 1
      • Ben-R
        Ben-R Subscriber @Nigel_Carr last edited by May 21, 2017, 1:52 PM May 21, 2017, 1:52 PM

        Hello Nigel,

        Thank you very much for your help. The reason we were a little hesitant to remove those lines from the robots.txt file is we were not sure if that would cause the ‘filtered’ pages we have for these collections to be indexed (Each collection can be sorted and filtered). We didn’t write the robots.txt originally and don’t really know what the syntax we would be removing is doing. While we try to find out and look for other options do you have any idea of whether it could have that type of impact?

        Also, we checked and there’s not a ‘noindex’ tag present.

        We currently only have one blog post up but oddly enough it does seem to be ‘indexed’ (even with the /blogs/ URL).

        We’re not so experienced with backend Shopify elements but is it possible there’s an issue that related to Google only caching the collections/all page we have up that lists all of our products?

        Any help or suggestions you have would be very much appreciated and thank you again!

        Please let us know if you have any questions, concerns or we could provide any further details in the meantime.

        Looking forward to hearing your thoughts!

        Best,

        1 Reply Last reply Reply Quote 0
        • Nigel_Carr
          Nigel_Carr last edited by May 20, 2017, 10:30 AM May 20, 2017, 10:27 AM

          Try deleting these from the Robots txt.

          Disallow: /collections/+
          Disallow: /collections/%2B
          Disallow: /collections/%2b

          Submit again and see what comes up. I'd hazard that this is the most logical explanation apart from having a noindex tag on the collections pages.

          Are the blogs pages coming up by the way?

          Right click 'view source' and have a look if there is one there.

          Regards

          Nigel

          Ben-R 1 Reply Last reply May 21, 2017, 1:52 PM Reply Quote 1
          • 1 / 1
          1 out of 4
          • First post
            1/4
            Last post

          Got a burning SEO question?

          Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


          Start my free trial


          Browse Questions

          Explore more categories

          • Moz Tools

            Chat with the community about the Moz tools.

          • SEO Tactics

            Discuss the SEO process with fellow marketers

          • Community

            Discuss industry events, jobs, and news!

          • Digital Marketing

            Chat about tactics outside of SEO

          • Research & Trends

            Dive into research and trends in the search industry.

          • Support

            Connect on product support and feature requests.

          • See all categories

          Related Questions

          • BobGW

            How many pages for a new site for Google to sink it's teeth into

            page authority on-page seo internal linking site structure content development

            Hello, I've got a high traffic site I'm building. Is 20 articles enough to start or should I start with 40 or 60 to not flop? I've got good internal linking. I've got: 3 10X articles
            17 cornerstones so far. Worked hard so far. Do I launch? or wait for 40? 60? I know thousands is best. Competitors have much more. Thank you, Bob

            Intermediate & Advanced SEO | Dec 3, 2024, 10:10 AM | BobGW
            0
          • aua

            Password Protected Page(s) Indexed

            Hi, I am wondering if my website can get a penalty if some password protected pages are showing up when I search on google: site:www.example.com/sub-group/pass-word-protected-page That shows that my password protected page was indexed either before or after adding the password protection. I've seen people suggest no indexing the page. Is that the best method to take care of this? What if we are planning on pushing the page live later on? All of these pages have no title tag, meta description, image alt text, etc. Should I add them for each page? I am wondering what is the best step, especially if we are planning on pushing the page(s) live. Thanks for any help!

            Intermediate & Advanced SEO | Nov 15, 2017, 1:08 PM | aua
            0
          • odihost

            Google does not want to index my page

            I have a site that is hundreds of page indexed on Google. But there is a page that I put in the footer section that Google seems does not like and are not indexing that page. I've tried submitting it to their index through google webmaster and it will appear on Google index but then after a few days it's gone again. Before that page had canonical meta to another page, but it is removed now.

            Intermediate & Advanced SEO | Feb 28, 2017, 3:58 AM | odihost
            0
          • SolveWebMedia

            My site shows 503 error to Google bot, but can see the site fine. Not indexing in Google. Help

            Hi, This site is not indexed on Google at all. http://www.thethreehorseshoespub.co.uk Looking into it, it seems to be giving a 503 error to the google bot. I can see the site I have checked source code Checked robots Did have a sitemap param. but removed it for testing GWMT is showing 'unreachable' if I submit a site map or fetch Any ideas on how to remove this error? Many thanks in advance

            Intermediate & Advanced SEO | Nov 23, 2015, 1:10 PM | SolveWebMedia
            0
          • Online-Marketing-Guy

            Substantial difference between Number of Indexed Pages and Sitemap Pages

            Hey there, I am doing a website audit at the moment. I've notices substantial differences in the number of pages indexed (search console), the number of pages in the sitemap and the number I am getting when I crawl the page with screamingfrog (see below). Would those discrepancies concern you? The website and its rankings seems fine otherwise. Total indexed: 2,360 (Search Consule)
            About 2,920 results (Google search "site:example.com")
            Sitemap: 1,229 URLs
            Screemingfrog Spider: 1,352 URLs Cheers,
            Jochen

            Intermediate & Advanced SEO | Nov 5, 2015, 6:10 PM | Online-Marketing-Guy
            0
          • mtthompsons

            Can we retrieve all 404 pages of my site?

            Hi, Can we retrieve all 404 pages of my site? is there any syntax i can use in Google search to list just pages that give 404? Tool/Site that can scan all pages in Google Index and give me this report. Thanks

            Intermediate & Advanced SEO | Aug 5, 2013, 12:04 PM | mtthompsons
            0
          • danng

            XML Sitemap Index Percentage (Large Sites)

            Hi all I'm wanting to find out from those who have experience dealing with large sites (10s/100s of millions of pages). What's a typical (or highest) percentage of indexed pages vs. submitted pages you've seen? This information can be found in webmaster tools where Google shows you the pages submitted & indexed for each of your sitemap. I'm trying to figure out whether, The average index % out there There is a ceiling (i.e. will never reach 100%) It's possible to improve the indexing percentage further Just to give you some background, sitemap index files (according to schema.org) have been implemented to improve crawl efficiency and I'm wanting to find out other ways to improve this further. I've been thinking about looking at the URL parameters to exclude as there are hundreds (e-commerce site) to help Google improve crawl efficiency and utilise the daily crawl quote more effectively to discover pages that have not been discovered yet. However, I'm not sure yet whether this is the best path to take or I'm just flogging a dead horse if there is such a ceiling or if I'm already at the average ballpark for large sites. Any suggestions/insights would be appreciated. Thanks.

            Intermediate & Advanced SEO | Aug 15, 2012, 11:34 PM | danng
            0
          • JSOC

            Should I Allow Blog Tag Pages to be Indexed?

            I have a wordpress blog with settings currently set so that Google does not index tag pages. Is this a best practice that avoids duplicate content or am I hurting the site by taking eligible pages out of the index?

            Intermediate & Advanced SEO | Jun 2, 2011, 2:05 PM | JSOC
            0

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy

          Looks like your connection to Moz was lost, please wait while we try to reconnect.