How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If I have set up my www as a cname, do I still need to add a redirect?
Hello, I am trying to minimize my redirect chains that have been popping up in my Moz crawl issues. So I'd like to get clear on some best practices. I have a few clients that are getting their first redesign in a while. We are moving into Wordpress from good ole .html files. So I've installed a redirection plug in to create 301 redirects for those .html versions of the pages. But apparently I am already redirecting from www to non-www. I have the www set up as a CNAME going to the IP address - is that counting as the redirection? Sometimes though, the www is set up by making it an A record. Is one way better than the other? Finally, I'm forcing https for all my new sites now. Is whatever the host does to "force" https also counting as a redirect and contributing to my chain? Any advice will be appreciated!
Moz Pro | | Dandelion0 -
Angular.js + Crawlers
I am working with a site that recently deployed Angular.js on the site. From an SEO standpoint its a little more tricky than we thought. We have deployed a couple updates to render pages for the bots but we not seeing changes in Moz weekly reports. When it comes to Angular.js, will the Moz bots read/access the site the same as the other major engines? I'm trying to figure out if our deployments are working or if there's something off in the Moz reports. Thanks.
Moz Pro | | JoshKimber0 -
How can I set up a campaign to track just directories on a specific subdomain?
I am trying to setup a campaign to track a specific subdomain and all its directories. For example, want to track example.abc.com/11111 and example.abc.com/22222 and so on. No interest in tracking abc.com itself. Is this possible?
Moz Pro | | BalihooSearch1 -
Ranking report not accurate
I've justed signed up for SEOMOZ and testing out the ranking report software. For a client that I'm doing an audit for I've ran a ranking report against the keywords. Using Chrome Incognito I've checked the ranking and they are on the 1st page but the ranking report is showing they aren't in the top 43, any ideas why this might be? I've checked my setting and set up and that's all fine.
Moz Pro | | CharlBritton0 -
Plurals and the SEOmoz On Page Report Card
I have some pages on my website that receive an "A" grade with the SEOmoz On Page Report Card if I use a plural term in a phrase (say, "landscape designers in Dallas") but an "F" grade if I use the singular term (say, "landscape designer in Dallas"). Is this a true and accurate reflection of how search engines see the page? Am I going to rank well for plural terms but not for singular, given the specific on-page content I'm using in this example? Or is this an opportunity for SEOmoz to tweak its' algorithm to better reflect the search engines?
Moz Pro | | ahirai1 -
Drop in number of Pages crawled by Moz crawler
What would cause a sudden drop in the number of pages crawled/accessed by the Moz crawler? The site has about 600 pages of content. We have multiple campaigns set up in our Pro account to track different keyword campaigns- but all for the same domain. Some show 600+ pages accessed, while others only access 7 pages for the same domain. What could be causing these issues?
Moz Pro | | AllaO0 -
How often are open site explorer reports updated?
I collect the information contained in open site explorer reports and csv backlinks audits on the 15th of every month. I noticed that the numbers are unchanged from 9/15-10/15. How often are the reports typically updated?
Moz Pro | | seagreen0 -
What's with the new Q&A set up?
In the old Q&A set up your response was automatically given a thumbs up by you so that you got a mozpoint for every response you gave. Now you have to manually thumb up your responses. Why was this change made? Can we get it back to the way it was?
Moz Pro | | DanDeceuster1