How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
A tale of two pages - one ranks great and one doesn't - but they are set up the same way
Hi everyone, First time poster 🙂 I will keep it simple - we have two pages on our website that are set up in the same way (or so we think) but achieve vastly different search engine rankings. We have spent weeks trying to figure out why this should be the case, but to no avail. Perhaps you guys can do a little diagnosis and see what we are missing: for search term: used machine tools www.machinetoolsonline.co.za/buy-used-machine-tools Is working well for us. for search term: new machine tools www.machinetoolsonline.co.za/buy-new-machine-tools is a disaster 😞 In fact, Google ranks our home page which is optimised for other terms, higher. Any ideas would be appreciated. Thanks!
Moz Pro | | Delorentis210 -
What impact has external links reported as http503
Hello I have 72 external broken links who are reported with a http 503 status. When I ask the owner of that site, he confirmed he has removed it from GA by using this status. My question is. Does this have an impact for the quality of my site? The site is available, there is no delay when you open the link. Thank you for your support. br
Moz Pro | | kjerstibakke
Kjersti Bakke0 -
Moz Crawl Report more urls?
Hi. I have used Moz Crawl Test and get my 3,000 urls crawled no issue. However, my site has more than that, is it possible to crawl the entire website? Alot of the crawl urls in the Moz test are search string urls and filters so Ive probably wasted about 2,500 urls on filter urls. Any advise or alternative software that wont cost a fortune?
Moz Pro | | YNWA
Thanks0 -
Rankings Different in Serps from Report
My site shows up on page one for many search queries, but I'm "not in the top 50" for those same terms according to the rankings report. Having troubleshooted the problem, it may be a canonical issue. I did not include www when I originally set up my reports and it seems that my site shows up in search results with www. However; if you type in the url without the www, it will automatically forward to the page with www. I did this intentionally in order not to get penalized for duplicate content pages. So here's my question: If I've got one page redirected to the other, shouldn't the rankings report also follow that redirect? If so, then it should show the correct results, right? Thanks, Dino
Moz Pro | | Dino640 -
Automated SEO Report for Agencies
I have a lot of SEO reports to send out, SEO Moz is awesome but it's not designed for this kind of work. I would like an automated report that does an on page optimization report on the potential client's site, finding errors etc it does not have to include fixes for each. This is just a basic report but it must look nice and be user friendly. I have looked at Raven and it looks ok however it does not have any decent On Page Optimization reports. I know I could crawl and then delete the campaign in SEO Moz but that is very time consuming and far from ideal. If anyone knows of a solution please let me know 🙂
Moz Pro | | SEOKeith0 -
SEOmoz keyword rankings in campaign report
Hi. Does anybody know where the rankings summary has moved to in the campaign reports? I want to know how many keywords have moved up or down in the last week and can't find it anywhere!
Moz Pro | | neooptic1 -
All seo reports
I want to find out what all the report's correct definition and when I see improving or going down what does it mean for eg. Organic search report? Keywords? or non-paid keywords?
Moz Pro | | ITWEBTEAM0 -
On Page Optimization Reports - Huh?
I've been working hard to use this EXCELLENT tool for optimize some of what I consider my most important pages . . . But the automatic tool that pulls pages and grades them (the "summary" of the "on page" report) . . . I don't get it. It only graded three of my pages, and I don't understand how it chose what keywords to grade it for? I'm just very confused. I don't understand how it chose the pages to grade, not the words it chose to grade it against. 😞
Moz Pro | | damon12120