Best method to stop crawler access to extra Nav Menu
-
Our shop site has a 3 tier drop down mega-menu so it's easy to find your way to anything from anywhere. It contains about 150 links and probably 300 words of text.
We also have a more context-driven single layer of sub-category navigation as well as breadcrumbs on our category pages.
You can get to every product and category page without using the drop down mega-menu.
Although the mega-menu is a helpful tool for customers, it means that every single page in our shop has an extra 150 links on it that go to stuff that isn't necessarily related or relevant to the page content. This means that when viewed from the context of a crawler, rather than a nice tree like crawling structure, we've got more of an unstructured mesh where everything is linked to everything else.
I'd like to hide the mega-menu links from being picked up by a crawler, but what's the best way to do this?
I can add a nofollow to all mega-menu links, but are the links still registered as page content even if they're not followed? It's a lot of text if nothing else.
Another possibility we're considering is to set the mega-menu to only populate with links when it's main button is hovered over. So it's not part of the initial page load content at all.
Or we could use a crude yet effective system we have used for some other menus we have of base encoding the content inline so it's not readable by a spider.
What would you do and why?
Thanks,
James
-
I agree Alan,
Mega Menu's are a good way to dilate the link equity of your page and in most cases it isn't needed at all. Keep the top-level navigation simple and have a submenu on all pages that contain links relevant to that section.
EG: Mega Menu could be:
Home Mens (Mens Tops, Mens Jeans, Mens Coats), Women (Womens Tops, Womens Jeans etc) Contact us
In this example it would be better to have one top level menu for:
Home | Mens | Women | Contact us
Then when your in the men or women section show links to "Tops", "Jeans" and "Coats". That way those links are relevant to the section you're in and reinforces the structure of that section to search engines.
After giving it further thought I would suggest not having a mega menu at all, because it may harm your SEO on-page optimisation efforts in the long term.
-
Ben's partially correct. Unfortunately Google has been claiming they do process Javascript for a while, and they recently stated they've begun reading AJAX. Of course they do a lousy job of it and don't always get it right, which just makes things even more muddy.
So from an SEO best practices perspective, you shouldn't have the menu(s) in the first place, at all.
You may also THINK their good for users but has any significant study been performed to confirm that? You'd need to check click-through rates on all the links to know for sure.
What I've found through years of auditing sites that have such menus is that it almost always turns out to be the case where most of the deeper links NEVER get clicked on from within these menus. Instead, they're overwhelming to users. This is why it's better to not have them from a UX perspective.
If you abandon them and go with more traditional hierarchical index and sub-index pages, and if those are properly optimized, you'll not only eliminate the massive SEO problem but in fact get more of your category pages to have higher ranking strength and authority over time.
IF you're going to keep them in any form because you don't want to go to the extreme I recommend, then yes - AJAX would likely be the only scenario that offers the least likelihood of search engines choking on the over-use of links.
And for the record, the real current problem with all those links on every page is duplicate content confusion - all of those URLS at the source level dilutes the uniqueness of content on every page of the site. And that also means you're harming the topical focus of every page as well. So whatever you do, AJAX or doing away with them altogether is going to be of high value long term.
- Alan Bleiweiss - Click2Rank's Search Team Director
-
From my experience I don't think you can really 'hide' the megamenu links from a crawler if they are generated using a content management system (code server side). If the link is on the page in the HTML then it will be crawled by a bot etc.
The general method of getting a mega menu to work is through the use of CSS and JavaScript, so you might want to have a look at using AJAX to get the relevant links from the database and then use JavaScript to put the links into the page.
This isn't a great solution, but bots cannot load JavaScript, so what they will see is only the links that are served up from the content management system.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best less expensive graphic design
Hello, We have an Ecommerce store and we need our category buttons to be redone and to shine. Unfortunately, I've tried all my places and none are 10X, though none of them cost very much (like Fiverr, Freelancer) What I would like is some advice on where to go for inexpensive but still very good quality graphics. I'm a good designer, and so far, I can do better graphics than the designers I've found, I just don't have the time as a busy SEO. Please let me know If you have any gems you are willing to share. I'm searching the community colleges as we speak. Thanks.
Web Design | | BobGW2 -
Using Button Links vs Sidebar Menu
I have a services page with a lot of rich text and a slideshow of images. Currently, I am using a column of buttons to various services, and am wondering if a sidebar menu would be more effective for Google to crawl and rank?
Web Design | | cinchmedia0 -
Best E-Commerce Platform for Fashion Store
We run a fashion/bridal store, Fashionably Yours, and we currently use the BigCommerce e-commerce platform, however it does not allow us to implement specific features. In particular, we would like to have GIFs in the product images to display various colours/styles, and to include videos of our products at the top of the product pages. Could anyone advise of an alternative platform that would be ideal for a fashion/bridal store and that would allow us to implement these features? Thank you.
Web Design | | CostumeD0 -
Best Practices for Leveraging Long Tail Content & Gated Content
Our B2B site has a lot of of long form content (e.g., transcriptions from presentations and webinars). We'd like to leverage the long tail SEO traffic driven to these pages and convert those visitors to leads. Essentially, we'd like Google to index all this lengthy, keyword-rich content AND we'd like to put up a read gate that requires users to register before viewing the full article. This is a B2B site, and the goal is to generate leads. Some considerations and questions: How much of the content to share before requiring registration? Ask too soon and it's a terrible user experience, give too much away and our business objectives are not met. Design-wise, what are good ways to do this? I notice Moz uses a "teaser" to block Mozinar content, and I've seen modals and blur bars on other sites. Any gotchas that Google doesn't like that we should be aware of? Trying to avoid anything that might seem like cloaking. Is it better to split the content across several pages (split a 10K word doc across 10 URLs and include a read gate on each) or keep to one page? Thank you!
Web Design | | Allie_Williams0 -
Multi-page articles, pagination, best practice...
A couple months ago we mitigated a 12-year-old site -- about 2,000 pages -- to WordPress.
Web Design | | jmueller0823
The transition was smooth (301 redirects), we haven't lost much search juice. We have about 75 multi-page articles (posts); we're using a plugin (Organize Series) to manage the pagination. On the old site, all of the pages in the series had the same title. I've since heard this is not a good SEO practice (duplicate titles). The url's were the same too, with a 'number' (designating the page number) appended to the title text. Here's my question: 1. Is there a best practice for titles & url's of multi-page articles? Let's say we have an article named: 'This is an Article' ... What if I name the pages like this:
-- This is an Article, Page 1
-- This is an Article, Page 2
-- This is an Article, Page 3 Is that a good idea? Or, should each page have a completely different title? Does it matter?
** I think for usability, the examples above are best; they give the reader context. What about url's ? Are these a good idea? /this-is-an-article-01, /this-is-an-article-02, and so on...
Does it matter? 2. I've read that maybe multi-page articles are not such a good idea -- from usability and SEO standpoints. We tend to limit our articles to about 800 words per page. So, is it better to publish 'long' articles instead of multi-page? Does it matter? I think I'm seeing a trend on content sites toward long, one-page articles. 3. Any other gotchas we should be aware of, related to SEO/ multi-page? Long post... we've gone back-and-forth on this a couple times and need to get this settled.
Thanks much! Jim0 -
Keywords in the page url for best SEO
Hello all, I am working in the keywors structure of a web and I have the following doubt: If I want to target these keywords: great food madrid and my website is: http://www.madridlive.com I do not know if I should keep either: OPTION 1: page url: www.madridlive.com/great-food-madrid or OPTION 2: page url www.madridlive.com/great-food I do not know if the search engines "understands" madrid in "madridlive", therefore I can avoid the "madrid" keyword, dicarding option 1 and going for option 2. Additionally I avoid duplication of the madrid keyword that can be seen as redundancy and also have a shorter page url. Thank you very much and sorry for such a question but I am new in this SEO field...just the excellent SEOMOZ's SEO Guide for beginners! Best regards, Antonio
Web Design | | aalcocer20030 -
Anyone have a good example of a CSS-based multi-level nav bar that is semantic (including link level subordination) and is ux positive?
Anyone have a good example of CSS-based multi-level nav bar that is semantic (including link level subordination) and is ux positive? Or am I gonna have to actually make one? Anyone have a good example of CSS-based multi-level nav bar that is semantic (including link level subordination) and is ux positive? Or am I gonna have to actually make one?
Web Design | | anns0 -
Best Site navigation solution
Hi there, We are getting our website redesigned and would like to know whether to increase the links on our site wide navigation or not. At the moment we have around 30 links from the navigation. We want to use exploding navigation menu and increase the links to our most important categories. Say if we increase to 60-70 would that be alright. (what will be the highest we can go for) At the moment categories that get links from navigation are ranking pretty good. If we increase would we loose those rankings. What will be the pros and cons of increasing navigation links? Second question we are also adding fooer links to top 10 categories in the footer. Would this be ok as far as seo and google concerned. Many Thanks
Web Design | | Jvalops0