Robots.txt Question for E-Commerce Sites
-
Hi All,
I have a couple of e-commerce clients and have a question about URLs.
When you perform a search on website all URLs contain a question mark, for example: /filter.aspx?search=blackout
I'm not sure that I want these indexed.
Could I be causing any harm/danger if I add this to the robots.txt file? /*?
Any suggestions welcome!
Gavin
-
You're right on target, it's not a good idea to index search results. Google doesn't want to crawl or index other search results in its own search results. There are some exceptions for gigantico sites like Yelp or TripAdvisor when showing their search results pages are actually the best option, but if you're not at that level and especially if you're an ecommerce site, it's not recommended.
You wouldn't be harming anything by excluding search from your robots.txt file. In fact, many top sites exclude search results to preserve crawl capacity and for indexation reasons.
You'll also want to look at parameter handling in Search Console, this article from Google will get you started.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best site Template, Structure, etc. for SEO
If I were to spin up a new site what do people recommend as the best template, services, etc. Do you have an example of the perfect structure, I want to point my team to an example page and say - This is perfect, do this but for our product (structure, content amount, etc) Thank you,
On-Page Optimization | | Jamesmcd030 -
Company name in Site Meta description?
Ok. So I know that you should have your company name in the site title, but is it all that important in the site description? The reason I ask is because I am competing with another company for the #1 position (I was number 1, now #2) that has an 8 character name and mine is 22 taking away from a great deal of real estate in my 150-160 character site description in which I could provide additional information describing my company. Should I remove my name in the site description enabling me to use more descriptive keywords and actionable text such as (Find, research, contact, professional, info) etc. Thanks,
On-Page Optimization | | photoseo10 -
301 Redirect to external site
Hi guys, We have a client who is getting their website redesigned through us. They are discontinuing couple of their services which will not get featured in the new site. They are fairly well ranked for these services and my client wishes to 301 redirect these pages to an external site owned by his friend so that they benefit out of the ranking. The question is: Will my client's website's general ranking get affected due to 301 redirecting to an external site? The external site is not spammy or red-flagged by Google (at the moment, at least). Thanks in advance!
On-Page Optimization | | RameshNair
Ramesh Nair0 -
Question about overoptimization and images "alt"
Hello, I own a shop with lots of categories, in each category there is a lot of pictures, some have already alt, must I put the attribute alt in all images, it would be 100% beneficial for my site or could would I be over-optimizing the site? Thank you
On-Page Optimization | | yuyuyu0 -
Crosslinking Question - Footer or Header?
We own 9 fashion blogs that focus on different niches. We have Your Next Shoes focusing on shoes, Your Next Dress focusing on dresses, Your Next Handbag focusing on handbags, and so on. Previously we had links in the blog header to the other sister sites. Your Next Shoes had text links to its 8 sister sites, and so on. However, we were told that this could cause a cross-linking penalty, so we removed these links. My understanding now is that the advice we were given is not necessarily correct. We think it is useful for our visitors to see that we also operate other blogs, so we are thinking of adding back the links - this time to our footer. Basically I am looking for advise on the best way to go about this.
On-Page Optimization | | Jantaro0 -
Nofollow on site-wide banner links
Hi I have a blog in relation to my webshop. On the blog I have two banners linking directly to the webshop on every page (site-wide links). Should I make the banner links nofollow? Thanks, Rasmus
On-Page Optimization | | secubi0 -
Is there any benefit in on-site duplicate content?
I have about 50 internal pages on my site that I want to add a "Do it yourself tutorial" to in an effort to build the quality of the pages. Is this going to de-value the content if I put it on all 50 pages? It's difficult to write similar content 50 different ways.
On-Page Optimization | | BradBorst0 -
Site wide internal links in footer
What is your take on site wide inernal links in the footer, e.g. for the most competitive keywords? Do you think those links are still useful or are those links of little value from your experience? It is an old school tactic but I still see many sites doing it.
On-Page Optimization | | Florakel0