What if page exists for desktop but not mobile?
-
I have a domain (no subdomains) that serves up different dynamic content for mobile/desktop pages--each having the exact same page url, kind of a semi responsive design, and will be using "Vary: User-Agent" to give Google a heads up on this setup.
However, some of the pages are only valid for mobile or only valid for desktop. In the case of when a page is valid only for mobile (call it mysite.com/mobile-page-only ), Google Webmaster Tools is giving me a soft 404 error under Desktop, saying that the page does not exist, Apparently it is doing that because my program is actually redirecting the user/crawler to the home page. It appears from the info about soft 404 errors that Google is saying since it "doesn't exist" I should give the user a 404 page--which I can make it customized and give the user an option to go to the home page, or choose links from a menu, etc..
My concern is that if I tell the desktop bot that mysite.com/mobile-page-only basically is a 404 error (ie doesn't exist), that it could mess up the mobile bot indexing for that page--since it definitely DOES exist for mobile users..
Does anyone here know for sure that Google will index a page for mobile that is a 404 not found for desktop and vice versa? Obviously it is important to not remove something from an index in which it belongs, so whether Google is careful to differential the two is a very important issue. Has anybody here dealt with this or seen anything from Google that addresses it? Might one be better off leaving it as a soft 404 error?
EDIT: also, what about Bing and Yahoo? Can we assume they will handle it the same way?
EDIT: closely related question--in a case like mine does Google need a separate sitemap for the valid mobile pages and valid desktop pages even though most links will be in both? I can't tell from reading several q&a on this.
Thanks, Ted
-
Monica,
I'm going to open a new thread to ask a similar question, as I think I didn't ask it very well.
Thanks for your input,
Ted
-
Thanks. If I understand you, the mobile bot won't crawl a url that the desktop bot has said needs to be fixed for it to work right for desktop. . Would you agree that doesn't really sound right on Google's part, since the url is fine for mobile use? I don't know why it wouldn't crawl for mobile, but if that's the way it is I can try fixing it on desktop to see if that enables the mobile to get crawled.
Once I do that I guess I'll find out whether a 404 not found for desktop will disable it from crawling for mobile (yes that link is accessible from other pages)--I was hoping to avoid trial and error on that because the time lag seems like it would be hard to pin down.
In a nutshell here's what I'm concerned will happen:
Google mobile bot crawls my mobile page and indexes it: Then the desktop bot crawls the same url and gets a 404 not found. Because of the desktop not found, Google removes it from the mobile page index.
I don't see a good way to test that since it depends on when each crawler is crawling. And, if this is what it is doing, I can't think of a good solution to having a responsive site with some content meant only for mobile indexing or only for desktop indexing.
-
If a URL is labeled a 404 it will not be crawled again unless there is a reason to, you mark it as fixed, or you edit the link in some form or fashion. Mark it as fixed and see if the error comes back. There is no harm in doing this.
Can you get to the page on your mobile device just by clicking through your site? If you can, that is good, it will eventually encourage a mobile bot to crawl it. If you can fetch and render as google, then I would just give it some time. I am not sure if there is a string of code you can add to the head of that page telling the robots that it is a mobile only page. I don't know how that works.
I would just mark it as fixed right now and see what happens over the next couple of days.
-
Hi Monica-thanks for your reply:
Ok, for a page that is supposed to be mobile only within a responsive-like setup(ie one domain) here's what I see:
The desktop bot crawls the link and gives a soft 404 error -- presumably because the page is currently being redirected to the home page.
The mobile bot is not crawling that link despite it being prominent on the main site home page, as my dbase is tracking the bot crawling and is not showing that it crawled that link for mobile (but is for desktop), and a search on my smartphone doesn't show that link either (even though it does show other links for pages used by both).. **Yet, if I fetch the mobile only page in webmaster tools using their mobile bot it finds it and renders it perfectly. ** So, why isn't it crawling it? Is it because when the mobile bot crawls it first looks and sees that that link is already 'flagged' as a soft 404 for the desktop? Or, is it because the mobile crawler is getting hung up on a link on the home page for mobile that has nothing to do with this mobile-only link?
It appears that the mobile bot is influenced by the desktop bot results--which is my fear: It seems to me their 2 bots should be independent of each other. If they aren't independent then if I change it to a 404 not found for desktop, would that even help, or would that prevent the mobile bot from ever trying to crawl it?
I would think that anybody who has a responsive page design and has blocked out certain content so that it renders only for mobile or only for non-mobile has had to face this issue.
Not sure what to do--I could fix the soft errors--change them to 404 not found and just see then if Google starts indexing for mobile or not, but was hoping to get some feedback before experimenting.
Thanks again, and please share more if you have more thoughts!
-
Did you look at your Mobile 404 errors? Google uses a different bot for mobile sites and anything related to that mobile page. Chances are, if it isn't reflecting a 404 in the Mobile errors in GWT, it is being indexed properly.
Check it out from you phone. Google the exact keyword and your company name. See if you can get to the page and if it is in fact the correct page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate pages and Canonicals
Hi all, Our website has more than 30 pages which are duplicates. So canonicals have been deployed to show up only 10 of these pages. Do more of these pages impact rankings? Thanks
Intermediate & Advanced SEO | | vtmoz0 -
Code to Redirect Mobile Subdomain to Desktop Site
Hi Everyone, My client is switching from a mobile subdomain to a responsive site. All URLs are the same on mobile subdomain vs desktop so we just need a wildcard rule to redirect m. to www. Anyone have this wildcard redirect code for an .htaccess file? Much appreciated! Dan
Intermediate & Advanced SEO | | kernmedia0 -
New pages not ranking
I published some new landing pages about a month a go which are much better quality than previous pages and on an optimised URL. The old pages never ranked and the new pages aren't ranking either although they are much better. The old pages 301 redirect to the new pages. Any quick ways I can at least get them ranking? Not expecting Page 1 overnight but to at least see the new pages on Page 5 would be great!
Intermediate & Advanced SEO | | Marketing_Today0 -
How should I deal with this page?
Hey Mozzers, I was looking for a little guidance and advice regarding a couple of pages on my website. I have used 'shoes' for this example. I have the current structure Parent Category - Shoes Sub Categories - Blue Shoes
Intermediate & Advanced SEO | | ATP
Hard Shoes
Soft Shoes
Big Shoes etc Supporting Article - Different Types of Shoe and Their Uses There are about 12 subcategories in total - each one links back to the Parent Category with the keyword "Shoes". Every sub category has gone from ranking 50+ to 10-30th for its main keyword which is a good start and as I release supporting articles im sure each one will climb. I am happy with this. The Article ranks no1 for about 20 longtails terms around "different shoes". This page attracts around 60% of my websites traffic but we know this traffic will not convert as most are people and children looking for information only for educational purposes and are not looking to buy. Many are also looking for a type of product we dont sell. My issue is ranking for the primary category "Shoes" keyword. When i first made the changes we went from ranking nowhere to around 28th on the parent category page targeted at "Shoes". Whilst not fantastic this was good as gave us something to work off. However a few weeks later, the article page ranked 40th for this term and the main page dropped off the scale. Then another week some of the sub category pages ranked for it. And now none of my pages rank in the top 50 for it. I am fairly sure this is due to some cannibalisation - simply because of various pages ranking for it at different times.
I also think that additional content added by products on the sub category pages is giving them more content and making them rank better. The Page Itself
The Shoes page itself contains 400 good unique words, with the keyword mentioned 8 times including headings. There is an image at the top of the page with its title and alt text targeted towards the keyword. The 12 sub categories are linked to on the left navigation bar, and then again below the 400 words of content via a picture and text link. This added the keyword to the page another 18 or so times in the form of links to longtail subcaterogies. This could introduce a spam problem i guess but its in the form of nav bars or navigation tables and i understood this to be a necessary evil on eCommerce websites. There are no actual products linked from this page. - a problem? With all the basic SEO covered. All sub pages linking back to the parent category, the only solution I can think of is to add more content by Adding all shoes products to the shoe page as it currently only links out the the sub categories Merging the "Different Type of Shoe and Their Uses" article into the shoe page to make a super page and make the article pages less like to produce cannibalistic problems. However, by doing solution 2, I remove a page bringing in a lot of traffic. The traffic it brings in however is of very little use and inflates the bounce rate and lowers the conversion rate of my whole site by significant figures. It also distorts other useful reports to track my other progress. I hope i have explained well enough, thanks for sticking with me this far, i havn't posted links due to a reluctance by the company so hopefully my example will suffice. As always thanks for any input.0 -
Please help with page
We used to use this page http://www.discountbannerprinting.co.uk/banners/vinyl-pvc-banners.html to rank for the words vinyl banner and PVC banner but we have tried to focus the page only on PVC banners and move the vinyl banners word to http://www.discountbannerprinting.co.uk/ yet for some reason even though they have both been spidered google has now chosen to rank this page http://www.discountbannerprinting.co.uk/stickers/vinyl-stickers.html for the vinyl banner words- how do I stop this from happening I thought the home page would be powerful enough to rank for the word with a title inclusion and a spread of the word on the page. Also if anyone can give their opinion on why they thinkhttp://www.discountbannerprinting.co.uk/banners/vinyl-pvc-banners.html does not rank very well I would be truly appreciative.
Intermediate & Advanced SEO | | BobAnderson0 -
How can you indexed pages or content on pages that are behind a pay wall or subscription login.
I have a client that has a boat of awesome content they provide to their client that's behind a pay wall ( ie: paid subscribers can only access ) Any suggestions mozzers? How do I get those pages index? Without completely giving away the contents in the front end.
Intermediate & Advanced SEO | | BizDetox0 -
Mobile alternates and redirects
Hi! We have a desktop version of our site at http://www.domain.com, and some weeks ago, we launched a mobile edition at http://m.domain.com, replicating the most important sections of the site, but not yet all of them. Actually, if you access with a mobile device userAgent to any desktop url you are redirected to the home of the mobile web. This is the only redirect implemented about mobile and desktop versions. A) Shall we also redirect "Googlebot-Mobile" to our mobile site, or it could be considered cloaking?
Intermediate & Advanced SEO | | marianoSoler98
B) Its necessary to implement the rel="alternate" media="handheld" tag in all of our Desktop SEO URLs? And in our mobile ones? Can't it be implemented via sitemaps like the rel="alternate" hreflang="x" tag?
C) Would the linkbuilding job done on the Desktop version affects the Mobile also, or we would still need to do a separate job? Thanks!0 -
Stuck on Page 2 - What Would You Do?!?
My site is : http://goo.gl/JgK1e My main keyword is : Plastic Bins i have been going back and forth between page 1 and 2 for this keyword and i was wondering if any of you could provide any guidance as to why i can't get on the top of page 1, and stay there... My site has been around for a while, we believe we have a great user experience, all unique, fresh content, and the lowest prices... I must be missing out on something major if I cannot get a steady page 1 ranking... Any thoughts? Thanks in advance...
Intermediate & Advanced SEO | | Prime850