What if page exists for desktop but not mobile?
-
I have a domain (no subdomains) that serves up different dynamic content for mobile/desktop pages--each having the exact same page url, kind of a semi responsive design, and will be using "Vary: User-Agent" to give Google a heads up on this setup.
However, some of the pages are only valid for mobile or only valid for desktop. In the case of when a page is valid only for mobile (call it mysite.com/mobile-page-only ), Google Webmaster Tools is giving me a soft 404 error under Desktop, saying that the page does not exist, Apparently it is doing that because my program is actually redirecting the user/crawler to the home page. It appears from the info about soft 404 errors that Google is saying since it "doesn't exist" I should give the user a 404 page--which I can make it customized and give the user an option to go to the home page, or choose links from a menu, etc..
My concern is that if I tell the desktop bot that mysite.com/mobile-page-only basically is a 404 error (ie doesn't exist), that it could mess up the mobile bot indexing for that page--since it definitely DOES exist for mobile users..
Does anyone here know for sure that Google will index a page for mobile that is a 404 not found for desktop and vice versa? Obviously it is important to not remove something from an index in which it belongs, so whether Google is careful to differential the two is a very important issue. Has anybody here dealt with this or seen anything from Google that addresses it? Might one be better off leaving it as a soft 404 error?
EDIT: also, what about Bing and Yahoo? Can we assume they will handle it the same way?
EDIT: closely related question--in a case like mine does Google need a separate sitemap for the valid mobile pages and valid desktop pages even though most links will be in both? I can't tell from reading several q&a on this.
Thanks, Ted
-
Monica,
I'm going to open a new thread to ask a similar question, as I think I didn't ask it very well.
Thanks for your input,
Ted
-
Thanks. If I understand you, the mobile bot won't crawl a url that the desktop bot has said needs to be fixed for it to work right for desktop. . Would you agree that doesn't really sound right on Google's part, since the url is fine for mobile use? I don't know why it wouldn't crawl for mobile, but if that's the way it is I can try fixing it on desktop to see if that enables the mobile to get crawled.
Once I do that I guess I'll find out whether a 404 not found for desktop will disable it from crawling for mobile (yes that link is accessible from other pages)--I was hoping to avoid trial and error on that because the time lag seems like it would be hard to pin down.
In a nutshell here's what I'm concerned will happen:
Google mobile bot crawls my mobile page and indexes it: Then the desktop bot crawls the same url and gets a 404 not found. Because of the desktop not found, Google removes it from the mobile page index.
I don't see a good way to test that since it depends on when each crawler is crawling. And, if this is what it is doing, I can't think of a good solution to having a responsive site with some content meant only for mobile indexing or only for desktop indexing.
-
If a URL is labeled a 404 it will not be crawled again unless there is a reason to, you mark it as fixed, or you edit the link in some form or fashion. Mark it as fixed and see if the error comes back. There is no harm in doing this.
Can you get to the page on your mobile device just by clicking through your site? If you can, that is good, it will eventually encourage a mobile bot to crawl it. If you can fetch and render as google, then I would just give it some time. I am not sure if there is a string of code you can add to the head of that page telling the robots that it is a mobile only page. I don't know how that works.
I would just mark it as fixed right now and see what happens over the next couple of days.
-
Hi Monica-thanks for your reply:
Ok, for a page that is supposed to be mobile only within a responsive-like setup(ie one domain) here's what I see:
The desktop bot crawls the link and gives a soft 404 error -- presumably because the page is currently being redirected to the home page.
The mobile bot is not crawling that link despite it being prominent on the main site home page, as my dbase is tracking the bot crawling and is not showing that it crawled that link for mobile (but is for desktop), and a search on my smartphone doesn't show that link either (even though it does show other links for pages used by both).. **Yet, if I fetch the mobile only page in webmaster tools using their mobile bot it finds it and renders it perfectly. ** So, why isn't it crawling it? Is it because when the mobile bot crawls it first looks and sees that that link is already 'flagged' as a soft 404 for the desktop? Or, is it because the mobile crawler is getting hung up on a link on the home page for mobile that has nothing to do with this mobile-only link?
It appears that the mobile bot is influenced by the desktop bot results--which is my fear: It seems to me their 2 bots should be independent of each other. If they aren't independent then if I change it to a 404 not found for desktop, would that even help, or would that prevent the mobile bot from ever trying to crawl it?
I would think that anybody who has a responsive page design and has blocked out certain content so that it renders only for mobile or only for non-mobile has had to face this issue.
Not sure what to do--I could fix the soft errors--change them to 404 not found and just see then if Google starts indexing for mobile or not, but was hoping to get some feedback before experimenting.
Thanks again, and please share more if you have more thoughts!
-
Did you look at your Mobile 404 errors? Google uses a different bot for mobile sites and anything related to that mobile page. Chances are, if it isn't reflecting a 404 in the Mobile errors in GWT, it is being indexed properly.
Check it out from you phone. Google the exact keyword and your company name. See if you can get to the page and if it is in fact the correct page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page not being ranked properly
Hi, Wondering if someone could possibly shed some light on why some of our pages are not being ranked properly on Google. For example this page https://www.mypetzilla.co.uk/dog-breeds Keyword "Dog Breeds" we can't be found on and we are absolutely baffled why? Could it be that we are listing all 100 and something dog breeds on one page? Should we introduce pagination or load more as user scrolls down. This page has been up for at least 4 years. Any suggestion or advice would be much appreciated. Many thanks
Intermediate & Advanced SEO | | Mypetzilla0 -
Why does Google display the home page rather than a page which is better optimised to answer the query?
I have a page which (I believe) is well optimised for a specific keyword (URL, title tag, meta description, H1, etc). yet Google chooses to display the home page instead of the page more suited to the search query. Why is Google doing this and what can I do to stop it?
Intermediate & Advanced SEO | | muzzmoz0 -
One page ranking for all key words, when other targeted pages not ranking
Hi everyone I am fairly new to SEO but have a basic understanding. I have a page that has a lot of content on it (including brand names and product types and relevant info) ranking for a quite a few key words. This is cool, except that I have pages dedicated to each specific key word that are not ranking. The more specific page still has a lot of relevant text on it too. eg. TYRES page - Ranks first for "Tyres". Ranks okay for many tyre key words, including "truck tyres"
Intermediate & Advanced SEO | | JDadd
TRUCK TYRES page - not ranking for "truck tyres" Further on, I then have pages not ranking all that well for more specific key words when they should. eg HONDA TRUCK TYRES - Then has a page full of product listings - no actual text. Not Ranking for "honda truck tyres". ABC HONDA TRUCK TYRE - not ranking for "abc honda truck tyre" key word
These pages don't have a lot of content on them, as essentially every single tyre is the same except for the name. But they do have text. So sometimes, these terms don't rank at all. And sometimes, the first TYRES page ranks for it. I have done the basic on page seo for all these pages (hopefully properly) including meta desc, meta titles, H1, H2, using key words in text, alt texting images where possible etc. According to MOZ they are optimised in the 90%. Link building is difficult as they are product listings, so other sites don't really link to these pages. Has anyone got ideas on why the top TYRES page might be so successful and out ranking more specific pages? Any ideas on how I can get the other pages ranking higher as they are more relevant to the search term? We are looking in to a website redesign/overhaul so any advice on how I can prevent this from happening on essentially a new site would be great too. Thanks!0 -
After adding a ssl certificate to my site I encountered problems with duplicate pages and page titles
Hey everyone! After adding a ssl certificate to my site it seems that every page on my site has duplicated it's self. I think that is because it has combined the www.domainname.com and domainname.com. I would really hate to add a rel canonical to every page to solve this issue. I am sure there is another way but I am not sure how to do it. Has anyone else ran into this problem and if so how did you solve it? Thanks and any and all ideas are very appreciated.
Intermediate & Advanced SEO | | LovingatYourBest0 -
Page 1 Reached, Further Page Improvements and What Next ?
Moz, I have a particularly tricky competitive keyword that i have finally climbed our website to the 10th position of page 1, i am particularly pleased about this as all of the website and content is German which i have little understanding of and i have little support on this, I am pleased with the content and layout of the page and i am monitoring all Google Analytics values very closely, as well as the SERP positions, So as far as further progression with this page and hopefully climbing further up page 1, where do you think i should focus my efforts ? Page Speed optimization?, Building links to this page ?, blogging on this topic (with links) , Mobile responsive design (More difficult), further improvements to pages and content linked from this page ? Further improvements to the website in general?,further effort on tracking visitors and user experience monitoring (Like setting up Crazyegg or something?) Any other ideas would be greatly appreciated, Thanks all, James
Intermediate & Advanced SEO | | Antony_Towle0 -
Different Header on Home Page vs Sub pages
Hello, I am an SEO/PPC manager for a company that does a medical detox. You can see the site in question here: http://opiates.com. My question is, I've never heard of it specifically being a problem to have a different header on the home page of the site than on the subpages, but I rarely see it either. Most sites, if i'm not mistaken, use a consistent header across most of the site. However, a person i'm working for now said that she has had other SEO's look at the site (above) and they always say that it is a big SEO problem to have a different header on the homepage than on the subpages. Any thoughts on this subject? I've never heard of this before. Thanks, Jesse
Intermediate & Advanced SEO | | Waismann0 -
Google + Local Pages
Hi, If I have a company with multipul addresses, Do I create separate Google + page for each area?
Intermediate & Advanced SEO | | Bryan_Loconto0 -
How can you indexed pages or content on pages that are behind a pay wall or subscription login.
I have a client that has a boat of awesome content they provide to their client that's behind a pay wall ( ie: paid subscribers can only access ) Any suggestions mozzers? How do I get those pages index? Without completely giving away the contents in the front end.
Intermediate & Advanced SEO | | BizDetox0