Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Mobile Googlebot vs Desktop Googlebot - GWT reports - Crawl errors
-
Hi Everyone,
I have a very specific SEO question. I am doing a site audit and one of the crawl reports is showing tons of 404's for the "smartphone" bot and with very recent crawl dates. If our website is responsive, and we do not have a mobile version of the website I do not understand why the desktop report version has tons of 404's and yet the smartphone does not. I think I am not understanding something conceptually.
I think it has something to do with this little message in the Mobile crawl report.
"Errors that occurred only when your site was crawled by Googlebot (errors didn't appear for desktop)."
If I understand correctly, the "smartphone" report will only show URL's that are not on the desktop report. Is this correct?
-
Hey Carla,
I'm not entirely sure what you're saying with:
"one of the crawl reports is showing tons of 404's for the "smartphone" bot and with very recent crawl dates. If our website is responsive, and we do not have a mobile version of the website I do not understand why the desktop report version has tons of 404's and yet the smartphone does not. I think I am not understanding something conceptually."
You say that the smartphone bot is seeing tons of 404s and the desktop report is showing tons of 404s, but the smartphone does not. If you can clarify that, I can probably better answer your question.
However, the answer is likely that Google may decide not to crawl URLs that it has already identified as 404s in one context. That is to say if they identify URLs on the mobile device as 404s they will know not to crawl them if they encounter them on desktop and vice versa.
-Mike
-
Ok I forgot to add something to this as well. Why would the URLS show up on the smartphone report if they are not on the desktop report? Afterall a 404 from either device is still a 404?
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is Amazon crawling my website? Is this hurting us?
Hi mozzers, I discovered that Amazon is crawling our site and exploring thousands of profile pages. In a single day it crawled 75k profile pages. Is this related to AWS? Is this something we should worry about or not? If so what could be a solution to counter this? Could this affect our Google Analytics organic traffic?
Intermediate & Advanced SEO | | Ty19860 -
Best SEO for table in mobile view
I'm wondering what the best way to present a table for mobile view in terms of SEO? It's a complicated table (not simple rows & columns but also col spans) which doesn't work with any responsive techniques I can find. I can offer different content for desktop / mobile so desktop is OK. But what's the best way forward with Google for mobile? I could offer a jpg or simply an explanation to revisit the page on desktop, but neither of those options seem particularly Google-friendly?
Intermediate & Advanced SEO | | Ann640 -
ScreamingFrog won't crawl my site.
Hey guys, My site is Netspiren.dk and when I use a tool like Screaming Frog or Integrity, it only crawls my homepage and menu's - not product-pages. Examples
Intermediate & Advanced SEO | | FrederikTrovatten22
A menu: http://www.netspiren.dk/pl/Helse-Kosttilskud-Blandingsolie_57699.aspx
A product: http://www.netspiren.dk/pi/All-Omega-3-6-9-180-kapsler_1412956_57699.aspx Is it because the products are being loaded in Javascript?
What's your recommendation? All best,
Fred.0 -
Lowercase VS. Uppercase Canonical tags?
Hi MOZ, I was hoping that someone could help shed some light on an issue I'm having with URL structure and the canonical tag. The company I work for is a distributor of electrical products and our E-commerce site is structured so that our URL's (specifically, our product detail page URL's) include a portion (the part #) that is all uppercase (e.g: buy/OEL-Worldwide-Industries/AFW-PG-10-10). The issue is that we have just recently included a canonical tag in all of our product detail pages and the programmer that worked on this project has every canonical tag in lowercase instead of uppercase. Now, in GWT, I'm seeing over 20,000-25,000 "duplicate title tags" or "duplicate descriptions". Is this an issue? Could this issue be resolved by simply changing the canonical tag to reflect the uppercase URL's? I'm not too well versed in canonical tags and would love a little insight. Thanks!
Intermediate & Advanced SEO | | GalcoIndustrial0 -
Canonical link vs root domain
I have a wordpress website installed on http://domain.com/home/ instead of http://domain.com - Does it matter whether I leave it that way with a canonical link from the domain.com to the domain.com/home/ or should I move the wordpress files and database to the root domain?
Intermediate & Advanced SEO | | JosephFrost0 -
301 or 302 Redirects to Mobile Site
When it's detected that a mobile device is accessing the site it has the ability to redirect from www.example.com to m.example.com. Does it make more sense to employ a 301 or 302 redirect here? Google says a 301 but does not explain why (although usually I stick to "when in doubt, 301") . It seems like a 302 would prevent passing link juice to the mobile site and having mobile-optimized results also showing up in Google's index. What is the preference here?
Intermediate & Advanced SEO | | SEOTGT0 -
How to stop Google crawling after 301 redirect?
I have removed all pages from my old website and set 301 redirect to new website. But, I have verified old website with Google webmaster tools' HTML verification file which enable me to track all data and existence of pages in Google search for my old website. I was assumed that, Google will stop crawling and DE-indexed all pages after 301 redirect. Because, I have set 301 redirect before 3 months. Now, I'm able to see Google bot activity on my website with help of Google webmaster tools. You can find out attachment to know more about it. How can it possible & How Google can crawl removed pages? You can see following image to know more about it. First & Second
Intermediate & Advanced SEO | | CommercePundit0 -
Robots.txt is blocking Wordpress Pages from Googlebot?
I have a robots.txt file on my server, which I did not develop, it was done by the web designer at the company before me. Then there is a word press plugin that generates a robots.txt file. How Do I unblock all the wordpress pages from googlebot?
Intermediate & Advanced SEO | | ENSO0
