Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Using 2 wildcards in the robots.txt file
-
I have a URL string which I don't want to be indexed. it includes the characters _Q1 ni the middle of the string.
So in the robots.txt can I use 2 wildcards in the string to take out all of the URLs with that in it? So something like /_Q1. Will that pickup and block every URL with those characters in the string?
Also, this is not directly of the root, but in a secondary directory, so .com/.../_Q1. So do I have to format the robots.txt as //_Q1* as it will be in the second folder or just using /_Q1 will pickup everything no matter what folder it is on?
Thanks.
-
I'm not 100% positive, however it does make sense to use it this way.
User-agent: *
Disallow: /*_Q1$
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Move to new domain using Canonical Tag
At the moment, I am moving from olddomain.com (niche site) to the newdomain.com (multi-niche site). Due to some reasons, I do not want to use 301 right now and planning to use the canonical pointing to the new domain instead. Would Google rank the new site instead of the old site? From what I have learnt, the canonical tag lets Google know that which is the main source of the contents. Thank you very much!
Intermediate & Advanced SEO | Feb 19, 2019, 9:16 AM | india-morocco0 -
How to combine 2 pages (same domain) that rank for same keyword?
Hi Mozzers, A quick question. In the last few months I have noticed that for a number of keywords I am having 2 different pages on my domain show up in the SERP. Always right next to each other (for example, position #7 and #8 or #3 and #4). So in the SERP it looks something like: www.mycompetition1.com www.mycompetition2.com www.mywebsite.com/page1.html
Intermediate & Advanced SEO | Jul 10, 2014, 6:41 AM | rayvensoft
4) www.mywebsite.com**/page2.html**
5) www.mycompetition3.com Now, I actually need both pages since the content on both pages is different - but on the same topic. Both pages have links to them, but page1.html always tends to have more. So, what is the best practice to tell Google that I only want 1 page to rank? Of course, the idea is that by combining the SEO Juice of both pages, I can push my way up to position 2 or 1. Does anybody have any experience in this? Any advice is much appreciated.0 -
Using the same content on different TLD's
HI Everyone, We have clients for whom we are going to work with in different countries but sometimes with the same language. For example we might have a client in a competitive niche working in Germany, Austria and Switzerland (Swiss German) ie we're going to potentially rewrite our website three times in German, We're thinking of using Google's href lang tags and use pretty much the same content - is this a safe option, has anyone actually tries this successfully or otherwise? All answers appreciated. Cheers, Mel.
Intermediate & Advanced SEO | Apr 16, 2014, 9:04 AM | dancape1 -
Using pictures from another domain
We are building several sites for several clients which will be using images from the manufacturer. Our dev team wants to insert the manufacturer's url for the images, instead of actually downloading the image and hosting on our server. There are thousands of images, so downloading images to our server will be time consuming, so we are looking for a shortcut.... however I'm concerned this will cause other issues. Is using manufactueresdomain.com/12345.jpg going to cause SEO issues? will this generate Google penalties? Since we are not able to control the image file name, we cannot optimize it. We will add Alt text and Title tag for each image, but the file name is random characters. How important is the file name for SEO?
Intermediate & Advanced SEO | Apr 8, 2014, 1:17 PM | Branden_S0 -
How to track mentions with links using Mentions App
We have a large client that we've just taken on board for organic search marketing. A great client that continually gets links from the BBC, NY Times etc which actually takes care of having to do any marketing for them. However, they get 10-15 unlinked mentions per day. Just recently, they peaked at 32 mentions in one day from JUST websites. But I need a quick way to filter these out and check if there is a link pointing to their website. I want to be able to build up a list of opportunities without having to manually check each website. How do, Mozzlars!?
Intermediate & Advanced SEO | Jan 16, 2014, 1:33 PM | jasondexter0 -
Is there a limit to images file names?
Hi, I have an eCommerce site with hundreds of product images. For management reasons files are named in length to have the product details in them.
Intermediate & Advanced SEO | Dec 2, 2013, 8:05 PM | BeytzNet
Is there a limit for a filename length before it is considered ambiguous or spammy etc.?
(it usually ranges 50-70 chars). Thanks0 -
When to Use Schema vs. Facebook Open Graph?
I have a client who for regulatory reasons cannot engage in any social media: no Twitter, Facebook, or Google+ accounts. No social sharing buttons allowed on the site. The industry is medical devices. We are in the process of redesigning their site, and would like to include structured markup wherever possible. For example, there are lots of schema types under MedicalEntity: http://schema.org/MedicalEntity Given their lack of social media (and no plans to ever use it), does it make sense to incorporate OG tags at all? Or should we stick exclusively to the schemas documented on schema.org?
Intermediate & Advanced SEO | Oct 24, 2013, 1:12 PM | Allie_Williams0 -
How do you implement dynamic SEO-friendly URLs using Ajax without using hashbangs?
We're building a new website platform and are using Ajax as the method for allowing users to select from filters. We want to dynamically insert elements into the URL as the filters are selected so that search engines will index multiple combinations of filters. We're struggling to see how this is possible using symfony framework. We've used www.gizmodo.com as an example of how to achieve SEO and user-friendly URLs but this is only an example of achieving this for static content. We would prefer to go down a route that didn't involve hashbangs if possible. Does anyone have any experience using hashbangs and how it affected their site? Any advice on the above would be gratefully received.
Intermediate & Advanced SEO | Aug 24, 2011, 12:19 PM | Sayers1