How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
50,000 4xx errors listed in MOZ report :(
HI, I've just had a look at a customers MOZ report and discovered nearly 50,000 errors listed!! The site is a non dynamic, non database driven site so where these have come from is beyond me. Example - (Both links open in a new window)
Moz Pro | | Skips
LIVE PAGE - http://lunnonwaste.com/licenses/Heatherland-Limited-H&S-Policy-Document-JAN-15.pdf
NON EXISTANT PAGE - http://lunnonwaste.com/licenses/licenses/Heatherland-Limited-H&S-Policy-Document-JAN-15.pdf - if you hover over any links you'll see they're ALL ......./licenses/licenses/.......... which doesn't exist! It's the file system that seems to be the problem - .........../licenses/licenses/........... and this gets added onto (............./licenses/licenses/licenses/licenses/.............. , up to the 50,000 page errors LOL The problem is self replicating. It's very odd and not something I've ever seen before, NON of these 'extra' pages are listed on the server, so where are they coming from? Any suggestions or help would be gratefully appreciated1 -
SeoMoz keyword page grader report card
Hi Ive recently 301 redirected old pages/urls to new pages/urls due to a site redesign. The seomoz kw report card is still reporting the old urls though with a significantly reduced score for most pages (i.e. from A's to now C's, from B to now D's etc). Does this mean mozbot is following the 301's & its actually the new urls being graded as such or is it referring to the old pages ? I will be adding the new urls to report card shortly but just interested in clarifying the above since i would have thought, if it is the old pages, they would not be reporting any score for those, or its scoring the new pages ? Many Thanks Dan
Moz Pro | | Dan-Lawrence0 -
4XX (Client Error) Report - Referrer URL Feature Request
Why not allow me to click on the individual 404 errors in the 4XX (Client Error) report and allow me to actually see the where the broken links are coming from (as a hyperlinked url) so I can locate/click and fix them? Providing the referring url w/hyperlink will stop me from having to jump between the report and searching in site explorer, or downloading the complete error report and sorting.
Moz Pro | | WaterGuy0 -
Why does the CSV report on OSE limit the number of links I can download to less than it should?
I run an inbound link report for the domain www.acornstairlifts.co.uk on OpenSiteExplorer.org using the following filters: Show: All Links from: only external Pages to: pages on this root domain Show links: ungrouped It says that there are around 30,678 links for that query. I then go to the advanced tab, and run a report using the same filters, but when I click download report I only get 2874. I've run this for the past 2 days and I get the same. Why?
Moz Pro | | Brett-Harland0 -
Reset Crawler
Hello, Does anyone know how to reset the crawler? We recently uploaded our new website and deleted the current campaign but it seems the crawler is caching our old websites data and not the new so every time we try to create a new campaign with the same details, it's just pulling everything from cache it seems. Thanks
Moz Pro | | ForzaHost0 -
Why have my rankings dropped on my keyword report but there is no change for the same keywords on google??
I received a keyword ranking report over the weekend, was horrified to see some of my keywords have dropped ranks by 20+ for some 😮 I checked them on google and they haven't moved (confused.com) Any information would be greatly appreciated.
Moz Pro | | Anthonykal-group0 -
Crawl Report Technical Issue
I'm having a problem with our campaign especially the crawl report. Because the last update is Nov 4 and isn't supposed to be updated weekly? I already submit a helpdesk support ticket and even send a seperate e-mail regarding this issue but until now the report is still not updated. Anybody here can help me raise this issue/ Thanks.
Moz Pro | | shebinhassan0 -
What is mT/mR good for In the keyword tool difficulty full report results?
So the ratio is MozTrust to MozRank, but what is this good for? What can I deduce from this and what can I use it for?
Moz Pro | | MarloSchneider2