Moz crawl showing up ?s=keyword pages as errors
-
Hi all,
Hoping someone can she some light on a fix with ref to wordpress and the search function it uses as Moz is craling some pages which reference the search
Errors showing up are duplicate pages, descriptions and titles.
The search function is not important on this site and I have tried to use a plugin which disables the search page which it does but these errors still show up. Can anyone assist as this is the final piece of the puzzle and then we're down to 0 issues on the site.
-
I have implemented the robots method for the moment and I will look at the exclusion of parameters too. thanks , i will now wait the next crawl from Moz .
-
Hi there,
I am not sure which fix will stop Moz from flagging this, but just in case it is also being picked up by Google, are you able to serve the canonical tag on domain.com/?s=keyword and any search query pages, pointing to a generic page?
You can remove parameters from Google in Webmaster Tools as well: https://support.google.com/webmasters/answer/1235687?hl=en
This ensures that Google does not crawl pages with the parameter you specify, meaning they won't be an issue in its index.
-
Different ways to do this - best way would be to eliminate the search from your template in the editor files. If you want Moz to stop crawling your internal search pages add this to your robots.txt file:
User-agent: RogerBot
Disallow: /?s=*?
This should block Moz's crawler from any page beginning with _/?s= _after the root domain. Do a regex test just to double check your internal search URL and that piece of regex.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Approach to Rank For Multiple Locations With Similar Targeted Keywords
I'm trying to determine the best way to set up a website to rank for a similar set of keyword phrases in three different cities. The keyword phrases I want to rank for are all pretty much the same with the only difference being the city associated with the keyword phrase. For example, "Austin water restoration" vs "San Antonio water restoration" vs "Houston water restoration". Each city needs about 7 or 8 pages of unique content to accurately target the group of keywords I'm trying to rank for. My initial thought was to write up unique content for each city and have each city act a site within the main site. For example, the main navigation for xyz.com/austin would be Austin specific, so when you land on xyz.com/austin and go to Services - Water Restoration, it would be all Austin specific content. The same would be true for San Antonio and Houston. The only problem with this approach is that I have to build up the page authority for a lot of different pages. It would be much easier to build up the page authority for one Water Restoration page and just insert a little "Areas we serve" on the page that includes "Austin, San Antonio, and Houston" and maybe work the coverage area in again at the bottom of the page somewhere. However, it would be much more difficult to work "Austin, San Antonio, and Houston" into the title tags and H1s though, and I couldn't logically work the cities into the content as much either. That would be a downside to this approach. Any thoughts on this? Wondering how large companies with hundreds of locations typically approach this? I'd really appreciate your input.
Web Design | | shaycw0 -
Should the primary navigation use keyword rich menu items?
There doesn't seem to be a definitive (or even authoritative) answer to this. The quandry is primary top navigation - the client wants to use 2-3 keyword length nav items, and I'm politely forcing him to use single word nav. The problem is that I have nothing to back up that assertion other than "this way looks better". Examples - Site is www.buybluewidgets.com. The client wants: About Our Blue Widgets Buy a Blue Widget Azure Widget Information Sell Your Blue Thingamajig My suggestion is: About Us Buy Learn Sell
Web Design | | AMSVansSEOTeam0 -
Changing URL's for a website redesign
Hi folks, We're redesigning our website and looking for some advice on how changing our URL's would affected our rankings. If the page URLs are changing how can we carry out redirects to avoid losing any SEO rank? Thanks, Ross
Web Design | | Will_Craig0 -
Landing pages vs internal pages.
Hey everyone I have run into a problem and would greatly appreciate anyone that could weigh in on it. I have a web client that went to an outside vendor for marketing. The client asked me to help them target some keywords and since I am new to the SEO world I have proceeded by researching the best keywords for the client. I found 6 that see excellent monthly searches. I then registered the .com and or .net domain names that match these words. I then started building landing pages that make reference to the keyword and then have links to his site to get more info. My customer sent the first of these sites to the marketer and he says I am doing things all wrong. He says rather then having landing pages like this I should just point the domain names at internal pages to the website. He also says that I should not have different looks for the landing pages from the main site and that I should have the full site menu on each landing page. I wanted to here what everyone here has to say about the pros and cons of the way to do this cause the guy giving the advice to me has a lower ranking site then I do and I have only started working on getting my site ranked this year. He has atleast according to him been doing this forever. Thanks, Ron
Web Design | | bsofttech0 -
Infinite Page Scrolling for e-commerce Product Catetegories
Hi There, I would like to know what's the pros and cons of Infinite Page Scrolling for e-commerce Product Categories that have over 700 products. Sample here Secondly how will this effect our on page SEO as far as google concerned? Many Thanks
Web Design | | Jvalops0 -
The use of foreign characters and capital letters in URL's?
Hello all, We have 4 language domains for our website, and a number of our Spanish landing pages are written using Spanish characters - most notably: ñ and ó. We have done our research around the web and realised that many of the top competitors for keywords such as Diseño Web (web design) and Aplicaión iPhone (iphone application) DO NOT use these special chacracters in their URL structure. Here is an example of our URL's EX: http://www.twago.es/expert/Diseño-Web/Diseño-Web However when I simply copy paste a URL that contains a special character it is automatically translated and encoded. EX: http://www.twago.es/expert/Aplicación-iPhone/Aplicación-iPhone (When written out long had it appears: http://www.twago.es/expert/Aplicación-iPhone/Aplicación-iPhone My first question is, seeing how the overwhelming majority of website URL's DO NOT contain special characters (and even for Spanish/German characters these are simply written using the standard English latin alphabet) is there a negative effect on our SEO rankings/efforts because we are using special characters? When we write anchor text for backlinks to these pages we USE the special characteristics in the anchor text (so does most other competitors). Does the anchor text have to exactly I know most webbrowsers can understand the special characters, especially when returning search results to users that either type the special characters within their search query (or not). But we seem to think that if we were doing the right thing, then why does everyone else do it differently? My second question is the same, but focusing on the use of Capital letters in our URL structure. NOTE: When we do a broken link check with some link tools (such as xenu) the URL's that contain the special characters in Spanish are marked as "broken". Is this a related issue? Any help anyone could give us would be greatly appreciated! Thanks, David from twago
Web Design | | wdziedzic0 -
Seo for flash home page?
I have a client who insists their home page be a single graphic image of their logo with php menus linking to all of the other pages up top. This appears to me to be an seo nightmare. They seem to be unwilling to have anything to do with changing the appearance and want to rely on seo for "all the other pages" on the site. What's an SEO to do in this situation? Is it possible to have a flash image that lands on a "homepage" for google to crawl, rather than a single image? What's the best seo approach here?
Web Design | | peaceland0