E-Commerce Site Collection Pages Not Being Indexed
-
Hello Everyone,
So this is not really my strong suit but I’m going to do my best to explain the full scope of the issue and really hope someone has any insight. We have an e-commerce client (can't really share the domain) that uses Shopify; they have a large number of products categorized by Collections. The issue is when we do a site:search of our Collection Pages (site:Domain.com/Collections/) they don’t seem to be indexed.
Also, not sure if it’s relevant but we also recently did an over-hall of our design. Because we haven’t been able to identify the issue here’s everything we know/have done so far:
- Moz Crawl Check and the Collection Pages came up.
- Checked Organic Landing Page Analytics (source/medium: Google) and the pages are getting traffic.
- Submitted the pages to Google Search Console.
- The URLs are listed on the sitemap.xml but when we tried to submit the Collections sitemap.xml to Google Search Console 99 were submitted but nothing came back as being indexed (like our other pages and products).
- We tested the URL in GSC’s robots.txt tester and it came up as being “allowed” but just in case below is the language used in our robots:
User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /9545580/checkouts
Disallow: /carts
Disallow: /account
Disallow: /collections/+
Disallow: /collections/%2B
Disallow: /collections/%2b
Disallow: /blogs/+
Disallow: /blogs/%2B
Disallow: /blogs/%2b
Disallow: /design_theme_id
Disallow: /preview_theme_id
Disallow: /preview_script_id
Disallow: /apple-app-site-association
Sitemap: https://domain.com/sitemap.xml - A Google Cache:Search currently shows a collections/all page we have up that lists all of our products.
Please let us know if there’s any other details we could provide that might help.
Any insight or suggestions would be very much appreciated.
Looking forward to hearing all of your thoughts!
Thank you in advance.
Best,
-
It seems odd to deal with filtering here. I'd normally do that in Search Console under URL parameters but you have to be extremely careful altering stuff in there. If you email me the site I'll run a check on why Collections aren't showing.
-
Hello Nigel,
Thank you very much for your help. The reason we were a little hesitant to remove those lines from the robots.txt file is we were not sure if that would cause the ‘filtered’ pages we have for these collections to be indexed (Each collection can be sorted and filtered). We didn’t write the robots.txt originally and don’t really know what the syntax we would be removing is doing. While we try to find out and look for other options do you have any idea of whether it could have that type of impact?
Also, we checked and there’s not a ‘noindex’ tag present.
We currently only have one blog post up but oddly enough it does seem to be ‘indexed’ (even with the /blogs/ URL).
We’re not so experienced with backend Shopify elements but is it possible there’s an issue that related to Google only caching the collections/all page we have up that lists all of our products?
Any help or suggestions you have would be very much appreciated and thank you again!
Please let us know if you have any questions, concerns or we could provide any further details in the meantime.
Looking forward to hearing your thoughts!
Best,
-
Try deleting these from the Robots txt.
Disallow: /collections/+
Disallow: /collections/%2B
Disallow: /collections/%2bSubmit again and see what comes up. I'd hazard that this is the most logical explanation apart from having a noindex tag on the collections pages.
Are the blogs pages coming up by the way?
Right click 'view source' and have a look if there is one there.
Regards
Nigel
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rel canonical tag from shopify page to wordpress site page
We have pages on our shopify site example - https://shop.example.com/collections/cast-aluminum-plaques/products/cast-aluminum-address-plaque That we want to put a rel canonical tag on to direct to our wordpress site page - https://www.example.com/aluminum-plaques/ We have links form the wordpress page to the shop page, and over time ahve found that google has ranked the shop pages over the wp pages, which we do not want. So we want to put rel canonical tags on the shop pages to say the wp page is the authority. I hope that makes sense, and I would appreciate your feeback and best solution. Thanks! Is that possible?
Intermediate & Advanced SEO | | shabbirmoosa0 -
Can a duplicate page referencing the original page on another domain in another country using the 'canonical link' still get indexed locally?
Hi I wonder if anyone could help me on a canonical link query/indexing issue. I have given an overview, intended solution and question below. Any advice on this query will be much appreciated. Overview: I have a client who has a .com domain that includes blog content intended for the US market using the correct lang tags. The client also has a .co.uk site without a blog but looking at creating one. As the target keywords and content are relevant across both UK and US markets and not to duplicate work the client has asked would it be worthwhile centralising the blog or provide any other efficient blog site structure recommendations. Suggested solution: As the domain authority (DA) on the .com/.co.uk sites are in the 60+ it would risky moving domains/subdomain at this stage and would be a waste not to utilise the DAs that have built up on both sites. I have suggested they keep both sites and share the same content between them using a content curated WP plugin and using the 'canonical link' to reference the original source (US or UK) - so not to get duplicate content issues. My question: Let's say I'm a potential customer in the UK and i'm searching using a keyword phrase that the content that answers my query is on both the UK and US site although the US content is the original source.
Intermediate & Advanced SEO | | JonRayner
Will the US or UK version blog appear in UK SERPs? My gut is the UK blog will as Google will try and serve me the most appropriate version of the content and as I'm in the UK it will be this version, even though I have identified the US source using the canonical link?2 -
Weird Indexing Issues with the Pages and Rankings
When I found the my page was non-existent on the search results page, I requested Google to index my page via the Search Console. And then just a few minutes after I did that, that page rose to top 3 ranking on the search page (with the same keyword and browser search). It happens to most of the pages on my website. Maybe a week later the rankings sank again, and I had to do the process again to make my pages to the top. Any reasons to explain this phenomenon, and how I can fix this issue? Thank you in advance.
Intermediate & Advanced SEO | | mrmrsteven0 -
E-commerce site structure & link juice: Bouncing off an idea
Hi guys, Question from a new-comer in SEO. Summary of the situation: potential customers are searching for a generic product category (buy mountainbike) more often than a brand in that category (Specialized MTB). And the latter is searched more often than a specific product ('some specific product from Specialized brand'). Both the brand pages and product pages are not ranking good Then would it be a good idea to have the category pages only link to the brand pages? They may show the products, but the links wouldn't pass link juice. I'm not even sure if that is technically possible, but I wanted to figure out the merit first. I'm hoping this would support the brand pages to rank better as they take in more volume. Please do feel free to teach me!
Intermediate & Advanced SEO | | Peter850 -
Splash/Warning Pages at front of site
We are looking at working on a site that needs a warning for users visiting - This splash/warning page is the only just google sees this not performing well in search engine - The sites are Wordpress sites - Would we use script to force a full screen pop up? This would be needed on a visit but if the user leaves and returns to the site the warning would need to reappear. Any ideas?
Intermediate & Advanced SEO | | JohnW-UK0 -
Is there a way to redirect pages from an old site?
I have no access to an old wordpress site of a client's, but have parked the domain on their new site, gone into webmaster central and requested a change of address and wait... the old domain still shows in the search listings in place of the new site domain and the log files show 404 errors from links to the old site which go nowhere - can anyone suggest a way of managing this on the new site - is there a workaround to what should have been done - 301 redirects on the old site before it was taken down. many thanks
Intermediate & Advanced SEO | | Highlandgael0 -
Handling Similar page content on directory site
Hi All, SEOMOZ is telling me I have a lot of duplicate content on my site. The pages are not duplicate, but very similar, because the site is a directory website with a page for cities in multiple states in the US. I do not want these pages being indexed and was wanting to know the best way to go about this. I was thinking I could do a rel ="nofollow" on all the links to those pages, but not sure if that is the correct way to do this. Since the folders are deep within the site and not under one main folder, it would mean I would have to do a disallow for many folders if I did this through Robots.txt. The other thing I am thinking of is doing a meta noindex, follow, but I would have to get my programmer to add a meta tag just for this section of the site. Any thoughts on the best way to achieve this so I can eliminate these dup pages from my SEO report and from the search engine index? Thanks!
Intermediate & Advanced SEO | | cchhita0 -
Image optimization for e-commerce
Regarding image optimization for an ecommerce site.
Intermediate & Advanced SEO | | triplelootz
In your "category" pages you list your products with a small thumbnails / miniature image. When the user clicks on the product name or on the thumnails, he lands on the product page with the real size product image. How do you optimize the thumbnail image? Do you use a different ALT? Is Google smart enough to index the real size image? On one hand the image located on the "product" page has lot more content around, is bigger & more interesting for both the user and Google. On the other hand the "category" page has more autority ( links) than the product page... To reformulate my questions: Do you think ALT tag is important for your thumbnail image on your category pages. Do you write different ALT tag for your thumbnail image ( on your category pages) & and your real size image (on your product page)? Which ALT tag / image do you think is the most interesting for Google? What do you think? Cheers, Ludo0