Navigation for search
-
We are getting ready to launch a site that has great navigation for users, but it is not so great for search engines. As long as we are ethical about it, does anyone see a downside to detecting a bot user agent and displaying different nav to it? I suppose some could consider it cloaking, but I noticed amazon uses this strategy and they don't seem to be getting a big penalty lol. We are not going to do anything shady with it, just offer the bot a different way to access our content. Any thoughts?
-
I would never recommend altering your site in any way specifically for a bot. Google has repeatedly stated that would be seen as cloaking and you definitely run the risk of a penalty. It can happen any time. Do you really want that cloud over your head?
Even if Amazon is able to do it, you are not Amazon. Despite any statement to the contrary, a huge site like Amazon can do things that you or I may not be able to do. We can debate the fairness of it, but we are not on a level playing field with them.
You seem to really want to do this, so feel free. But x months from now you will likely make a post "My rankings dropped and I can't figure out why", and then someone will notice the setting and that will be the answer.
-
Could you please share the site in question? we might be able to give a much more insightful thought
-
Thanks for your suggestion Alex!
-
All the content will be accessible through the sitemap which will be displayed in both versions (the user and the bot) version of the site. The only difference is the navigation will change from one version to another. It could be perceived as a gray area, even though we really are just making it more friendly to bots not trying to be deceptive at all.
-
And that should be okay if they see the content is the same between your normal version and the spider version, just a difference in presentation. It becomes a problem when you start serving drastically different content.
-
How about adding an indexable version of the navigation and then hiding it with CSS? The crawlers would see that. I haven't thought about whether that'd be a good idea, just a quick thought.
-
We usually operate under the principal that would this be ok if it were manually reviewed? In our plan, I am pretty confident that it would pass. We are planning on making what is displayed to bots very basic text link navigation. Our XML and html sitemaps will correspond well with the bot navigation (so to show G we really are not trying to hide anything). And yes we could make the nav more search friendly, but one of our main goals is to have the site look as clean as possible. As far as I can tell Amazon uses this method to avoid duplicate content issues and make their site generally more crawlable which is why I think it is on the table as an option.
-
Depending on how much you're changing, I think it could be considered cloaking and end up getting you in trouble. You'll get looked at any time you're sniffing for bots and serving up different content. Are there other ways you could make the navigation index-able?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Snippet showed in google search is not from metaDescription
This is my page https://www.collegehippo.com/graduate-school/programs/top-ranked-masters-degree-museum-museology-and-curatorial-studies The metaDescription is | |
On-Page Optimization | | etattva
| name="description" content="Master's degree in Museum, Museology and Curatorial Studies is offered by 49 American universities. New York University had highest number of international students receiving a Master's degree. Johns Hopkins University had the most women graduates in this program. Job outlook for Museum, Museology and Curatorial Studies Museum, Museology and Curatorial Studies is projected to grow 13 percent from 2016 to 2026, faster than average for all occupations. Median pay for Museum, Museology and Curatorial Studies in 2018 was $53,360. The number of jobs were 11170. Check out best universities offering online Master's program in Museum, Museology and Curatorial Studies "/> |
| | | But when I see the page in google search results for (museum studies graduate programs), This is how it appears in the search results. It is showing the breadcrumbs from the page. I am not sure why is google is treating the page like this. It was not like that 5 months back. Nothing much changed in page and google is displaying the page content like this . How can I fix this?0 -
Fixing Index Errors in the new Google Search Console - Help
Hi, So I have started using the new Search Console and for one of my clients, there are a few 'Index Coverage Errors'. In the old version you could simply, analyse, test and then mark any URLs as fixed - does anyone know if that is possible in the new version? There are options to validate errors but no 'mark as fixed' options. Do you need to validate the errors before you can fix them?
On-Page Optimization | | daniel-brooks0 -
Frustrated by Google Search Result
We have a page on our website for our review of the "Voltage" shisha flavor by "Social Smoke" (Social Smoke is the brand). Voltage is one of their hookah tobacco flavors. https://www.hookah.org/social-smoke-voltage-flavored-hookh-tobacco/. When I search for "Social Smoke Voltage Review", our page is at the bottom of the first page result. We have a video, decent content on the page, and a review function. We've implemented correct Schema code too: https://goo.gl/iwCP7E. When I use the page grade tool on Moz. Our page ranks B for that keyword but the results number 1 and 2 and 3 on Google all rank C or D. Our video and review schemas don't show up on Google search result either. We have a good community online. Our social media pages are popular. We share the blog posts on the social media accounts fairly regularly too. We have an old and established website. From what I understand we are following all of Google's standards and rules too. What does a website owner gotta do?
On-Page Optimization | | Heydarian0 -
Your tactics on improving organic search for a site in a struggling industry
We work with a client of ours with organic search initiatives. The problem is: the industry (e.g. the core of the business) is sagging. It has been for a couple of years. And they're finally feeling these sagging losses. Google Trends support this and shows it quite nicely. It pretty much mirrors organic search referrals as well. The industry (e.g. the core phrase that the company and its competitors have historically hit very hard) as well as Google Trends for the client (middle of the pack) and their two big competitors are attached. Wondering if anyone else has had this type of circumstance with their clients and some of their go to tactics that helped them stop the skid (and even make it start going up). Thanks SpztkvF.png LPHGo76.png
On-Page Optimization | | ChristianMKG0 -
How to exclude URL filter searches in robots.txt
When I look through my MOZ reports I can see it's included 'pages' which it shouldn't have included i.e. adding filtering rules such as this one http://www.mydomain.com/brands?color=364&manufacturer=505 How can I exclude all of these filters in the robots.txt? I think it'll be: Disallow: /*?color=$ Is that the correct syntax with the $ sign in it? Thanks!
On-Page Optimization | | neenor0 -
Navigation Links
Our developers typically have a CSS driven html menu at the top of the page with links to inside pages. They then have the same links in the footer. Does this double navigation cause any SEO issues or does Google disregard the second set of links? Thanks, Mark
On-Page Optimization | | DenverKelly0 -
Google indexing Internal Search Results
Greeting, Currently I have noticed that Google is starting to index our internal search page results. Should I block those pages in our robot txt file or have you ever heard of any websites that actually gained traffic or rank by letting Google index those pages? Thanks
On-Page Optimization | | Tonyd230 -
Magento Layered Navigation & Duplicate Content
Hello Dear SeoMoz, I would like to ask your help with something that I am not sure off. Our ecommerce web site is built with Magento. I have found many problems so far and I know that there will be many more in the future. Currently, I am trying to find the best way to deal with the duplicate content that is produced from the layered navigation (size, gender etc). I have done a lot of research so far in order to understand which might be the best practice and I found the following practices: **Block layered navigation URLSs from the Google Webmaster Tools (**Apparently this works for Google Only). Block these URLs with the robots.txt file Make links no-follow **Make links JavaScript from Magento *** Avoid including these links in the xml site map. Avoid including these link in the A-Z Product Index. Canonical tag Meta Tags (noindex, nofollow) Question If I turn the layered navigation links into JavaScript links from the Magento Admin, the layered navigation links are still found by the crawlers but they look like that: | http://www.mysite.com/# instead of: http://www.mysite.com/girls-basics.html?gender_filte... | Can these new URLS (http://www.mysite.com/# ) solve the duplicate content problems with the layered navigation or do I need to implement other practices too to make sure that everything is done right. Kind Regards Stefanos Anastasiadis
On-Page Optimization | | alexandalexaseo0