Navigation for search
-
We are getting ready to launch a site that has great navigation for users, but it is not so great for search engines. As long as we are ethical about it, does anyone see a downside to detecting a bot user agent and displaying different nav to it? I suppose some could consider it cloaking, but I noticed amazon uses this strategy and they don't seem to be getting a big penalty lol. We are not going to do anything shady with it, just offer the bot a different way to access our content. Any thoughts?
-
I would never recommend altering your site in any way specifically for a bot. Google has repeatedly stated that would be seen as cloaking and you definitely run the risk of a penalty. It can happen any time. Do you really want that cloud over your head?
Even if Amazon is able to do it, you are not Amazon. Despite any statement to the contrary, a huge site like Amazon can do things that you or I may not be able to do. We can debate the fairness of it, but we are not on a level playing field with them.
You seem to really want to do this, so feel free. But x months from now you will likely make a post "My rankings dropped and I can't figure out why", and then someone will notice the setting and that will be the answer.
-
Could you please share the site in question? we might be able to give a much more insightful thought
-
Thanks for your suggestion Alex!
-
All the content will be accessible through the sitemap which will be displayed in both versions (the user and the bot) version of the site. The only difference is the navigation will change from one version to another. It could be perceived as a gray area, even though we really are just making it more friendly to bots not trying to be deceptive at all.
-
And that should be okay if they see the content is the same between your normal version and the spider version, just a difference in presentation. It becomes a problem when you start serving drastically different content.
-
How about adding an indexable version of the navigation and then hiding it with CSS? The crawlers would see that. I haven't thought about whether that'd be a good idea, just a quick thought.
-
We usually operate under the principal that would this be ok if it were manually reviewed? In our plan, I am pretty confident that it would pass. We are planning on making what is displayed to bots very basic text link navigation. Our XML and html sitemaps will correspond well with the bot navigation (so to show G we really are not trying to hide anything). And yes we could make the nav more search friendly, but one of our main goals is to have the site look as clean as possible. As far as I can tell Amazon uses this method to avoid duplicate content issues and make their site generally more crawlable which is why I think it is on the table as an option.
-
Depending on how much you're changing, I think it could be considered cloaking and end up getting you in trouble. You'll get looked at any time you're sniffing for bots and serving up different content. Are there other ways you could make the navigation index-able?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Where can we see voice search impressions/searches?
Hi Guys, As voice search seems to be getting more popular, I would like to start accounting for these searches. I've tried looking for low impressions/long tail searches in the Search Console but feel like I am just guessing which queries are search. Does anybody have any clever way of finding these searches or could you list an example of how/if you search using voice? Thanks!
On-Page Optimization | | Xtend-Life0 -
Google Search Console issue: "This is how Googlebot saw the page" showing part of page being covered up
Hi everyone! Kind of a weird question here but I'll ask and see if anyone else has seen this: In Google Search Console when I do a fetch and render request for a specific site, the fetch and blocked resources all look A-OK. However, in the render, there's a large grey box (background of navigation) that covers up a significant amount of what is on the page. Attaching a screenshot. You can see the text start peeking out below (had to trim for confidentiality reasons). But behind that block of grey IS text. And text that apparently in the fetch part Googlebot does see and can crawl. My question: is this an issue? Should I be concerned about this visual look? Or no? Never have experienced an issue like that. I will say - trying to make a play at a featured snippet and can't seem to have Google display this page's information, despite it being the first result and the query showing a featured snippet of a result #4. I know that it isn't guaranteed for the #1 result but wonder if this has anything to do with why it isn't showing one. VmIqgFB.png
On-Page Optimization | | ChristianMKG0 -
Structuring navigation for maximum effect
Hi, I am working with a client (in the property niche) who has 200+ links on each page of their site mainly due to an extensive navigation menu. They have good domain authority (although some competitors have a lot better) and some excellent links from some fantastic domains but the keywords just aren’t moving. (Sidenote: most links point to the home page with some going to property detail pages not location pages which is where I’d like people to be landing). I am reviewing the site structure and other technical aspects and have some questions regarding how the navigation is structured. Firstly is 200+ links an ok number to have? Everything I read points to 100 being a magic number to aim for. Secondly, the site navigation menu contains a list of locations. The first tier being country, the second tier drops down to list the regions within that country, then a third tier drop down appears to list the towns and cities in those regions. So from any page in the site you can drill down to town/city locations. (Sidenote: I have run Hotjar on the site which shows most people are using the search facility not the navigation menu to search) Is this style of navigation ok or does it dilute the link authority/pagerank/juice being past to each page? Would a better option be to have the first and second tier in the drop down then the third level town navigation to appear in the sidebar at page level in the appropriate sections? What effect would such a change have on rankings?
On-Page Optimization | | caravan0 -
Internal Linking - in content vs navigation menu
Would like to get some thoughts on whether navigation menus or in-content links are best for internal linking, from an SEO standpoint. A few thoughts to get started with: For sites with a lot of content, you can have a navigation menu linking to your higher-level pages, then in-content links to deeper pages on your site. For smaller sites, this is not an option, as the navigation menu will probably link to all your important pages. You could add in-content links, but Google only counts the first link on the page, so the in-content links would be ignored if you'd already linked yp the page in your top nav menu. I can think of several possible reasons navigation menu links could be less desirable than in content links from a Google perspective. (They are sitewide boilerplate content without context.) If you setup your navigation structure based on what is best for the user, small sites don't have much wiggle room to optimize internal link structure, as all their money pages will be linked to from the top nav menu. Do you think Google prefers in content links to navigation menu links? If so, how do you get around the fact that for many sites, all their money pages are being linked to from their main navigation menu?
On-Page Optimization | | AdamThompson0 -
Incoming Search Terms
Hi guys, I saw a blog post recently where the author added a list of "incoming search queries" to the bottom of his post, obviously to improve the post's ranking for those terms. On one hand, I suppose it it does help users find that post. On the other, it seems lazy and somewhat dodgy, but I haven't found any opinions on it elsewhere and have not seen this practice in my experience. What're your thoughts? Outright search engine manipulation? Cheers, Carlo SCWYt
On-Page Optimization | | mtgconsulting0 -
SEO and Pagination on search results
Hi Our new site is still in beta mode, and I just wanted to check from an SEO perspective that on search results and on the pagination at the bottom of the page, is there anything we need to do so its compatible with the search engines and not have duplicate content issues etc I have implemented search friendly URL's, but I would welcome any other issues or suggestions you have that will help improve the sites SEO performance The URL of a search results page is http://www.cheapfindergames.com/category-arcade-and-platform
On-Page Optimization | | ocelot0 -
How do I avoid cannibalization and point search engines to the correct page?
Currently this page ranks for the phrase "keep calm and carry on" http://www.paper-source.com/cgi-bin/paper/item/Keep-Calm-and-Carry-On-Bandages/3307.010/413160.html This page is just one product of many in the Keep Calm category. I'd rather this page be the result: http://www.paper-source.com/cgi-bin/paper/gifts/keep-calm-and-carry-on.html seomoz states: "To prevent engines from potentially seeing a signal that this page is not the intended ranking target and creating additional competition for your page, we suggest staying away from linking internally to another page with the target keyword(s) as the exact anchor text." So obviously I should remove the link from the product detail page, but what other things can I do to get the category page to rank higher than the one product page? Thanks!
On-Page Optimization | | leighw0 -
What's the best practice for implementing a "content disclaimer" that doesn't block search robots?
Our client needs a content disclaimer on their site. This is a simple "If you agree to these rules then click YES if not click NO" and you're pushed back to the home page. I have this gut feeling that this may cause an upset with the search robots. Any advice? R/ John
On-Page Optimization | | TheNorthernOffice790