Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Using 2 wildcards in the robots.txt file
-
I have a URL string which I don't want to be indexed. it includes the characters _Q1 ni the middle of the string.
So in the robots.txt can I use 2 wildcards in the string to take out all of the URLs with that in it? So something like /_Q1. Will that pickup and block every URL with those characters in the string?
Also, this is not directly of the root, but in a secondary directory, so .com/.../_Q1. So do I have to format the robots.txt as //_Q1* as it will be in the second folder or just using /_Q1 will pickup everything no matter what folder it is on?
Thanks.
-
I'm not 100% positive, however it does make sense to use it this way.
User-agent: *
Disallow: /*_Q1$
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How safe is it to use a meta-refresh to hide the referrer?
Hi guys, So I have a review site and I'm affiliated with several partnership programs whose products I advertise on my site. I don't want these affiliate programs to see the source of my traffic (my site), so I'm looking for a safe solution to hide the referrer URL. I have recently added a rel="noreferrer" tag to all my affiliate links, but this method isn't perfect as not all browsers respect that rule. After doing some research and checking my competitors I noticed that some of them use meta-refresh, which seems more reliable in this regard. So, how safe is it to use meta-refresh as means of hiding referrer URL? I'm worrying that implementing a meta-refresh redirect might negatively affect my SEO. Does anybody have any suggestions on how to hide the referrer URL without damaging SEO? Thank you.
Intermediate & Advanced SEO | | Ibis150 -
Using H3 before or instead of an H2...
My designer and I have been having an argument: we have a blog with short, 400 words posts. They have an H1 with nice keywords and a catchy title, and then a few subheadings. I don't like making the subheadings H2, because the font looks way too large in Wordpress, so my designer wants to make them all H4s, so the font looks to be a nicer size. Here's my problem with that and why I usually just bold the subheadings: Is it really bad to put a bunch of H4s right under an H1, with not H2's or 3's to separate? I'm reading different arguments on the internet about this and gladly welcome more debate and/or case studies. Thank you!
Intermediate & Advanced SEO | | genevieveagar0 -
Disallow URLs ENDING with certain values in robots.txt?
Is there any way to disallow URLs ending in a certain value? For example, if I have the following product page URL: http://website.com/category/product1, and I want to disallow /category/product1/review, /category/product2/review, etc. without disallowing the product pages themselves, is there any shortcut to do this, or must I disallow each gallery page individually?
Intermediate & Advanced SEO | | jmorehouse0 -
How to track mentions with links using Mentions App
We have a large client that we've just taken on board for organic search marketing. A great client that continually gets links from the BBC, NY Times etc which actually takes care of having to do any marketing for them. However, they get 10-15 unlinked mentions per day. Just recently, they peaked at 32 mentions in one day from JUST websites. But I need a quick way to filter these out and check if there is a link pointing to their website. I want to be able to build up a list of opportunities without having to manually check each website. How do, Mozzlars!?
Intermediate & Advanced SEO | | jasondexter0 -
How does the use of Dynamic meta tags effect SEO?
I'm evaluating a new client site which was built buy another design firm. My question is they are dynamically creating meta tags and I'm concerned that it is hurting their SEO. When I view the page source this is what I see. <meta name="<a class="attribute-value">keywords</a>" id="<a class="attribute-value">keywordsGoHere</a>" content="" /> <meta name="<a class="attribute-value">description</a>" id="<a class="attribute-value">descriptionGoesHere</a>" content="" /> <title id="<a class="attribute-value">titleGoesHere</a>">title> To me it looks like the tags are not being added to the page, however the title is showing when you view it in a browser and if use a spider view tool, it sees the title. I'm guess it is being called from a DB. So I'm a little concerned though that the search engines are not really seeing the title and description. I'm not worried about the keywords tag. Can anyone shed some light on how this might work? Why it might not being showing the text for the description in the page code and if that will hurt SEO? Thanks for the help!
Intermediate & Advanced SEO | | BbeS0 -
Using Folkd for Video Backlink
Hi Mozzers, What are your thoughts on using www.folkd.com for video SEO? We have a few company videos and would like to possibly get a backlink by either embedding one of our youtube videos on our site or self hosting the video. Are bookmarking sites like this spammy?
Intermediate & Advanced SEO | | Travis-W0 -
Meta Keywords: Should we use them or not?
I am working through our site and see that meta keywords are being used heavily and unnecessarily. Each of our info pages will have 2 or 3 keyword phrases built into them. Should we just duplicate the keyword phrases into the meta keyword field, should put in additional keywords beyond or not use it at all? Thoughts and opinions appreciated
Intermediate & Advanced SEO | | Towelsrus1 -
Canonical & noindex? Use together
For duplicate pages created by the "print" function, seomoz says its better to use noindex (http://www.seomoz.org/blog/complete-guide-to-rel-canonical-how-to-and-why-not) and JohnMu says its better to use canonical http://www.google.com/support/forum/p/Webmasters/thread?tid=6c18b666a552585d&hl=en What do you think?
Intermediate & Advanced SEO | | nicole.healthline1