Moz crawl showing up ?s=keyword pages as errors
-
Hi all,
Hoping someone can she some light on a fix with ref to wordpress and the search function it uses as Moz is craling some pages which reference the search
Errors showing up are duplicate pages, descriptions and titles.
The search function is not important on this site and I have tried to use a plugin which disables the search page which it does but these errors still show up. Can anyone assist as this is the final piece of the puzzle and then we're down to 0 issues on the site.
-
I have implemented the robots method for the moment and I will look at the exclusion of parameters too. thanks , i will now wait the next crawl from Moz .
-
Hi there,
I am not sure which fix will stop Moz from flagging this, but just in case it is also being picked up by Google, are you able to serve the canonical tag on domain.com/?s=keyword and any search query pages, pointing to a generic page?
You can remove parameters from Google in Webmaster Tools as well: https://support.google.com/webmasters/answer/1235687?hl=en
This ensures that Google does not crawl pages with the parameter you specify, meaning they won't be an issue in its index.
-
Different ways to do this - best way would be to eliminate the search from your template in the editor files. If you want Moz to stop crawling your internal search pages add this to your robots.txt file:
User-agent: RogerBot
Disallow: /?s=*?
This should block Moz's crawler from any page beginning with _/?s= _after the root domain. Do a regex test just to double check your internal search URL and that piece of regex.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will changing product from Grouped to Simple on my magento category page affect my SEO?
Hi all, A category page on my site http://www.porcelainsuperstore.co.uk/wood-effect.html currently ranks number 3 on Google for the keyword "Wood Effect Tiles" We're currently reorganising some of our product and I would like to know if this is going to affect the SEO and ranking for the above page and keyword. The majority of products on that page are magento grouped products. I would like to change the page so that it displays only the different constituent simple products rather than the grouped products on the category page. My question is, will this have any impact on SEO? I intend on leaving all other data on the category page the same - so the metadata and the description/title etc. Any help/comments would be much appreciated! Ben
Web Design | | piazza0 -
CMS dynamicly created pages indexed?
Hey Moz'erz, Looking at the indexed pages of my clients eCommerce website I noticed that dynamically created pages are being indexed. For example this page does not "exist" but is created by a drop down filter menu that sorts by product tag: /collections/tools/TAG I can only conclude that this page got indexed either through a backlink or once upon a time there was an internal link pointing to this URL and got indexed (currently there is not). Are either of these cases possibilities? In either case before considering removal or any action I would of-course reference analytics to check for conversions, traffic and any backlinks for those "pages". I believe at the end of the day is recommend a drop down filer that doesn't create new pages as the best solution. Thoughts, comments and experience is greatly welcomed 🙂
Web Design | | paul-bold0 -
4XX (Client Error) on Wordpress Wesbite
I've just taken over the management of a website and am getting 4x 4XX (client Error) issues. Example: http://inter-italia.com/en/wp-login.php?action=lostpassword Can anyone give any guidance as how to fix this wordpress? I also see a lot of 'temporary redirects' due to multilingual plugin - is there anything I can do to fix this?
Web Design | | skehoe0 -
What seo benefit does setting up a photo gallery where each photo is a separate web page?
what seo benefit does setting up a photo gallery where each photo is a separate web page? My old SEO guy set up my photo gallery like that claiming that because each photo was a separate page, it added a big seo benefit and i never understood what he was talking about. Maybe alt text on the photo with key phrases in it pointing to my other pages to give my site a theme for google? I'm not really sure. He has since moved away and i am considering redoing the photo gallery to multiple images on one page to be more user friendly to my users. This photo gallery is 3 years old and the photos might have some page rank to them helping my site so i don't want to remove this gallery if there really is a benefit to it and it will hurt my site. I once removed four static page rank 3 pages from my site that weren't used for my site anymore and my rankings dropped 5 positions. Thoughts anyone? Thanks! Ron
Web Design | | Ron100 -
Redirects (301/302) versus errors (404)
I am not able to convincingly decide between using redirects versus using 404 errors. People are giving varied opinions. Here are my cases 1. Coding errors - we put out a bad link a. Some people are saying redirect to home page; the user at least has something to do PLUS more importantly it does NOT hurt your SEO ranking. b. Counter - the page ain't there. Return 404 2. Product removed - link1 to product 1 was out there. We removed product1; so link1 is also gone. It is either lying in people's bookmarks, OR because of coding errors we left it hanging out at some places on our site.
Web Design | | proptiger0 -
Question About Site Redesign and Nav / Page Structure
Hey guys, i am currently redesigning our company's site, and have come across some things that I'm not quite sure of. We used to have individual service pages in our main navigation (design, video, marketing) before the redesign. In this new design, i had the idea of making just one "services" or "capabilities" page, where these three services would each be outlined, and each service would have a list of links to more specific landing pages. Obviously, breaking it up correctly with HTML5 using the andtags. What I'm wondering is that if i'm going to be penalized for having those three services that aren't necessarily related too closely on the same page as opposed to having the one page for each service (like we have now). Any help would be greatly appreciated, and let me know if i need to elaborate more. Thanks in advance!
Web Design | | RenderPerfect0 -
404 page not found after site migration
Hi, A question from our developer. We have an issue in Google Webmaster Tools. A few months ago we killed off one of our e-commerce sites and set up another to replace it. The new site uses different software on a different domain. I set up a mass 301 redirect that would redirect any URLs to the new domain, so domain-one.com/product would redirect to domain-two.com/product. As it turns out, the new site doesn’t use the same URLs for products as the old one did, so I deleted the mass 301 redirect. We’re getting a lot of URLs showing up as 404 not found in Webmaster tools. These URLs used to exist on the old site and be linked to from the old sitemap. Even URLs that are showing up as 404 recently say that they are linked to in the old sitemap. The old sitemap no longer exists and has been returning a 404 error for some time now. Normally I would set up 301 redirects for each one and mark them as fixed, but there are almost quarter of a million URLs that are returning 404 errors, and rising. I’m sure there are some genuine problems that need sorting out in that list, but I just can’t see them under the mass of errors for pages that have been redirected from the old site. Because of this, I’m reluctant to set up a robots file that disallows all of the 404 URLs. The old site is no longer in the index. Searching google for site:domain-one.com returns no results. Ideally, I’d like anything that was linked from the old sitemap to be removed from webmaster tools and for Google to stop attempting to crawl those pages. Thanks in advance.
Web Design | | PASSLtd0 -
Why is Google sending traffic to our homepage, not our optimized pages?
Hello Forum, My team and I just completely redid a yoga eCommerce site, including its SEO. The old version of the site didn't feature page-specific optimization and, as a result, Google's search results for our keywords almost always directed visitors to the homepage. For example, a Google search for the term "yoga bolster" sent users to the homepage, not the product category page for yoga bolsters. After redoing the site and optimizing specific pages (i.e. the yoga bolster page is now optimized for the keyword "yoga bolster"), the Google search results are still taking users to the homepage, not the optimized page. (i.e. if you search for yoga bolster, find our search result, and click the search result link, you're taken to the homepage, not the bolster page) It's only been about 36 hours since we've launched the new website and submitted it to Google's webmaster tools. Does anyone know why Google is still sending people to our homepage and not the keyword-optimized pages we created? Is this a timing issue?
Web Design | | pano0