Best method to stop crawler access to extra Nav Menu
-
Our shop site has a 3 tier drop down mega-menu so it's easy to find your way to anything from anywhere. It contains about 150 links and probably 300 words of text.
We also have a more context-driven single layer of sub-category navigation as well as breadcrumbs on our category pages.
You can get to every product and category page without using the drop down mega-menu.
Although the mega-menu is a helpful tool for customers, it means that every single page in our shop has an extra 150 links on it that go to stuff that isn't necessarily related or relevant to the page content. This means that when viewed from the context of a crawler, rather than a nice tree like crawling structure, we've got more of an unstructured mesh where everything is linked to everything else.
I'd like to hide the mega-menu links from being picked up by a crawler, but what's the best way to do this?
I can add a nofollow to all mega-menu links, but are the links still registered as page content even if they're not followed? It's a lot of text if nothing else.
Another possibility we're considering is to set the mega-menu to only populate with links when it's main button is hovered over. So it's not part of the initial page load content at all.
Or we could use a crude yet effective system we have used for some other menus we have of base encoding the content inline so it's not readable by a spider.
What would you do and why?
Thanks,
James
-
I agree Alan,
Mega Menu's are a good way to dilate the link equity of your page and in most cases it isn't needed at all. Keep the top-level navigation simple and have a submenu on all pages that contain links relevant to that section.
EG: Mega Menu could be:
Home Mens (Mens Tops, Mens Jeans, Mens Coats), Women (Womens Tops, Womens Jeans etc) Contact us
In this example it would be better to have one top level menu for:
Home | Mens | Women | Contact us
Then when your in the men or women section show links to "Tops", "Jeans" and "Coats". That way those links are relevant to the section you're in and reinforces the structure of that section to search engines.
After giving it further thought I would suggest not having a mega menu at all, because it may harm your SEO on-page optimisation efforts in the long term.
-
Ben's partially correct. Unfortunately Google has been claiming they do process Javascript for a while, and they recently stated they've begun reading AJAX. Of course they do a lousy job of it and don't always get it right, which just makes things even more muddy.
So from an SEO best practices perspective, you shouldn't have the menu(s) in the first place, at all.
You may also THINK their good for users but has any significant study been performed to confirm that? You'd need to check click-through rates on all the links to know for sure.
What I've found through years of auditing sites that have such menus is that it almost always turns out to be the case where most of the deeper links NEVER get clicked on from within these menus. Instead, they're overwhelming to users. This is why it's better to not have them from a UX perspective.
If you abandon them and go with more traditional hierarchical index and sub-index pages, and if those are properly optimized, you'll not only eliminate the massive SEO problem but in fact get more of your category pages to have higher ranking strength and authority over time.
IF you're going to keep them in any form because you don't want to go to the extreme I recommend, then yes - AJAX would likely be the only scenario that offers the least likelihood of search engines choking on the over-use of links.
And for the record, the real current problem with all those links on every page is duplicate content confusion - all of those URLS at the source level dilutes the uniqueness of content on every page of the site. And that also means you're harming the topical focus of every page as well. So whatever you do, AJAX or doing away with them altogether is going to be of high value long term.
- Alan Bleiweiss - Click2Rank's Search Team Director
-
From my experience I don't think you can really 'hide' the megamenu links from a crawler if they are generated using a content management system (code server side). If the link is on the page in the HTML then it will be crawled by a bot etc.
The general method of getting a mega menu to work is through the use of CSS and JavaScript, so you might want to have a look at using AJAX to get the relevant links from the database and then use JavaScript to put the links into the page.
This isn't a great solution, but bots cannot load JavaScript, so what they will see is only the links that are served up from the content management system.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Indexing Multi-Store Best Practice
Hi Guys, We currently have a main store view and a uk store view setup with a Litespeed Redirect for our website, redirecting UK IP Customers to the UK Store. We recently noticed that we were running into some issues with Google indexing pages from the uk site as well as the main store view. With trying to avoid duplicate content, my question being: What is the best practice for google indexing the UK and Main store views? Any advice would be greatly appreciated. Thanks.
Web Design | | centurysafety0 -
Is it still necessary to have a "home" page button/link in the top nav?
Or is it not necessary to have a "home" tab/link because everybody by this time knows you can get to the home page by clicking on the logo?
Web Design | | FindLaw0 -
Best Magento Hosting in the US?
Hello We have a UK Based client that we host their UK site on TSO. They are moving their US site to Magento and I am worried about the impact of hosting a US Magento site in the UK. Which US Magento host would you recommend?
Web Design | | peeveezee0 -
Heres a puzzle for you... Htags on left hand nav for ecomm category pages
Hi Guys, So I am back asking more questions, but I am slowly learning 🙂 This next one, I have looked everywhere and I can't get my head around h tags on ecomm sites. I have looked at competitors and non competitors and still am not sure which is the right or wrong thing to do, specifically in this instance, category pages. Our I.T. dept is somewhat under resourced and I don't want to waste their time with test and trial on this particular issue. Category landing pages... There are shed loads of category listing pages on our site, at the moment the h1 tag for each sub-category is listed in the end path of the breadcrumb, there is no other spaces on the page accept the left hand navigation. Which is the better to use, breadcrumb or nav? We would have to totally recode our left hand nav which is pretty set up for the whole site. The reason I ask this question to you is because an SEO agency recommended that we... Add the H1 to the left hand navigation and make it customisable so that it is not the same as the breadcrumb keywords. I have attached an image of a competitor of ours that so the same thing currently, to show what I mean... Right now I am not sure what to tell the agency and what the right thing to do is. I read a post saying that we are actually doing the right thing under the circumstances. Does anyone have a best practise good example of generally what we should do for category pages? Your help is always muchly appreciated Kindest, Kay new
Web Design | | eLab_London0 -
Best practice for product detail when all products are onepage
HI there,
Web Design | | Hephey
I have a page utilizing isotope with multiple products with small text excerpts and when you click an item i opens a detailed view without requiring a new page load. I've read some of the one page posts but can't get my head around what's best SEO wise when dealing with possible duplicate content. I guess one method could be to have the product list with small excerts of text and all the details hidden in some json and then when the user clicks it, it will open the product and fill with details from inline json. The click action is overring the a tag action e.g. with jquery, so the the a tag has a clean url to a proper subpage with meta, h1 and all that stuff so google can follow it. The jquery thing enables the navigation without a page reload and I can update the document url with pushState.
The subpage, if visited directly, includes the same animation stuff as the master but now has h1, p meta specific to that product but still with same effect, navigation and layout as the master page. Does anybody know if there is a better way to do this with one page sites when wanting to seo optimize detailed contents?0 -
A/B Testing.. Are you doing? how is it been? What do you think would be the best path for who is starting now?
Hey Mozers, One of my 2014 resolutions is to start doing A/B Testing, so far I have been following "best practices" and "common sense" when comes to website design, but I would like to go above and beyond. I was hoping a could get a few tips some of you that are already doing A/B testing. How is it been? Do you see a great ROI? What do you think would be best path for who is starting now? Any book or links you would recommend? Thanks
Web Design | | Felip30 -
Is there a best practice for using a general iso code for the EAME region and APAC region or should you break it out by country?
I am creating a strategy for multiple regions and the US comes to market different than EAME (Europe, Africa, Middle East) and China. We were planning on using language and iso codes in subfolder's but the corporation only wants their content to be in German, English, and Queens English. Our current decision is to use /en-US/, /en/, /de/, /en-CN/, /zh-CN/. /en/ and /de/ will be what we use for EAME. This doesn't seem like the best idea as I think /en/ will get indexed as the US version and not the EAME version. Any suggestion or if clarification is needed is greatly appreciated.
Web Design | | GodfreyB2B0 -
Which Shopping Cart is best for SEO? Magento vs. X-Cart
Comparing X-Cart and Magento, which do you think is better for SEO and why? I am leaning towards Magento and wanted to get some opinions?
Web Design | | BlinkWeb0