Good to use disallow or noindex for these?
-
Hello everyone,
I am reaching out to seek your expert advice on a few technical SEO aspects related to my website. I highly value your expertise in this field and would greatly appreciate your insights.
Below are the specific areas I would like to discuss:a. Double and Triple filter pages:
I have identified certain URLs on my website that have a canonical tag pointing to the main /quick-ship page. These URLs are as follows:
https://www.interiorsecrets.com.au/collections/lounge-chairs/quick-ship+black
https://www.interiorsecrets.com.au/collections/lounge-chairs/quick-ship+black+fabricConsidering the need to optimize my crawl budget, I would like to seek your advice on whether it would be advisable to disallow or noindex these pages. My understanding is that by disallowing or noindexing these URLs, search engines can avoid wasting resources on crawling and indexing duplicate or filtered content. I would greatly appreciate your guidance on this matter.
b. Page URLs with parameters:
I have noticed that some of my page URLs include parameters such as ?variant and ?limit. Although these URLs already have canonical tags in place, I would like to understand whether it is still recommended to disallow or noindex them to further conserve crawl budget. My understanding is that by doing so, search engines can prevent the unnecessary expenditure of resources on indexing redundant variations of the same content. I would be grateful for your expert opinion on this matter.
Additionally, I would be delighted if you could provide any suggestions regarding internal linking strategies tailored to my website's structure and content. Any insights or recommendations you can offer would be highly valuable to me.
Thank you in advance for your time and expertise in addressing these concerns. I genuinely appreciate your assistance. If you require any further information or clarification, please let me know. I look forward to hearing from you.
Cheers!
-
@williamhuynh You're correct to pay attention to parameters in your URLs, as they can have an impact on how search engines crawl and index your site. It's crucial, however, to handle them strategically.
Using canonical tags on these pages is already a good move. It signals to search engines which version of the page should be treated as the main one. Canonicalization helps avoid potential duplicate content issues and makes your website easier to understand from a search engine's perspective.
However, I'd be careful to disallow these pages or use a "noindex" tag. Disallowing these URLs in your robots.txt file might seem like a good way to save the crawl budget, but it can have unintended side effects. When you disallow a URL, it means that search engines can't access it at all, which could impact the crawling and indexing of your main (canonical) pages. This is especially true if these parameterized URLs have unique backlinks or user engagement signals that could be beneficial for your canonical URLs.
As for the "noindex" approach, this tells search engines not to include the page in their index. However, if these pages have valuable backlinks or user engagement signals, you might be missing out on some SEO value by not indexing them.
In my opinion, if your website is large and you're genuinely concerned about the crawl budget, a more suitable approach might be to use Google Search Console's URL Parameters tool. This tool lets you inform Google how to handle specific URL parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Capturing Source Dynamically for UTM Parameters
Does anyone have a tutorial on how to dynamically capture the referring source to be populated in UTM parameters for Google Analytics? We want to syndicate content and be able to see all of the websites that provided referral traffic for this specific objective. We want to set a specific utm_medium and utm_campaign but have the utm_source be dynamic and capture the referring website. If we set a permanent utm_source, it would appear the same for all incoming traffic. Thanks in advance!
Technical SEO | | peteboyd0 -
Unsolved Orphaned unwanted urls from the cms
Hi
Technical SEO | | MattHopkins
I am working on quite an old cms, and there are bunch of urls that don't make any sense.
https://www.trentfurniture.co.uk/products/all-outdoor-furniture/all-outdoor-furniture/1
https://www.trentfurniture.co.uk/products/all-chairs/all-chairs/1
https://www.trentfurniture.co.uk/products/all-industries/all-chairs/1
https://www.trentfurniture.co.uk/products/all-chairs/all-industries/1
https://www.trentfurniture.co.uk/products/all-chairs/banqueting-furniture/1
https://www.trentfurniture.co.uk/products/all-chairs/bar-furniture/1
https://www.trentfurniture.co.uk/products/all-chairs/bentwood-furniture/1
For example there are no internal links. And fortunately not much traffic at all. But I can't see in the cms why they are generating? I've tried to check the html code to check why, what's the reason? But all I can think of is the structure....? something odd the cms writes?
Anyone have any ideas please? And would I redirect all these? Just thinking there could be a better solution/fix, rather than redirects since there are no links or traffic.....Like the devs solve why they are generating.....Unfortunately I get very slow responses from the devs as a 3rd pty company, hence on here ;0). (Some of those are indexed too)... :0) Thanks in advance....0 -
How to boost the number of visitors to a specific page
Hello, Payday loan consolidation, debt settlement, credit card settlement, and debt consolidation are just a few of the pages I have. While walking through the search console dashboard, I noticed that while the position is improving, but impressions and traffic are decreasing, indicating a significant disparity. Please advise on how to resolve the issue.
Link Building | | OVLG0 -
Using one robots.txt for two websites
I have two websites that are hosted in the same CMS. Rather than having two separate robots.txt files (one for each domain), my web agency has created one which lists the sitemaps for both websites, like this: User-agent: * Disallow: Sitemap: https://www.siteA.org/sitemap Sitemap: https://www.siteB.com/sitemap Is this ok? I thought you needed one robots.txt per website which provides the URL for the sitemap. Will having both sitemap URLs listed in one robots.txt confuse the search engines?
Technical SEO | | ciehmoz0 -
Which Version Url to Use for Canonical Tags and in General for Homepage.
I want to put canonical tags on the homepage of a site. cant figure out the version of URL of the homepage should be with a / at the end or without the / ( www.example.com of www.example.com/ ) if I put into the google the URL with / I get the URL without the / in my browser, and it isn't showing as a redirect in my moz extension or other tools. But when I copy the URL from browser and paste elsewhere it pastes with a / I have two questions 1 - in general how does it work with URLs of homepages - I see this happening with lots of sites? 2 - which URL should I set as the canonical version of my homepage? Thanks so much
Technical SEO | | Ruchy0 -
What is "evttag=" used for?
I see evttag= used on realtor.com, what looks to be for click tracking purposes. Does anyone know if this is an official standard or something they made up?
Technical SEO | | JDatSB0 -
Use of Location Folders
I'd like to understand the pro's and con's of using a location subfolder as an SEO strategy (example: http://sqmedia.us/Dallas/content-marketing.html), where the /Dallas folder is holding all of my keyword rich page titles. The strategy is to get local-SEO benefits from the use of the folder titled /Dallas (a folder which is unnecessary in the over all structure of this site), but how much is this strategy taking away from the page-title keyword effectiveness?
Technical SEO | | sqmedia0 -
How rel=canonical works with index, noindex ?
Hello all, I had always wondered how the index,noindex affects to the canonical. And also if the canonical post should be included in the sitemap or not. I posted this http://www.comparativadebancos.co... and with a rel=canonical to this that was published at the beginning of the month http://www.comparativadebancos.co... but then I have the first one in google http://www.google.com/search?aq=f... May be this is evident for you but, what is really doing the canonical? If I publish something with the canonical pointing to another page, will it still be indexed by google but with no penalty for duplicate content? Or the usual behaviour should have been to havent indexed the first post but just the second one? Should I also place a noindex in the first post in addition to the canonical? What am I missing here? thanks
Technical SEO | | antorome0