Summarize your question.Sitemap blocking or not blocking that is the question?
-
Hi from wet & overcast wetherby UK
Ones question is this...
" Is the sitemap plus boxes blocking bots ie they cant pass on this page http://www.langleys.com/Site-Map.aspx "
Its just the + boxes that concern me, i remeber reading somewherte javascript nav can be toxic.
Is there a way to test javascript nav set ups and see if they block bots or not?
Thanks in advance
-
I use Screaming Frog SEO Spider (free version) to check the internal link structure of a website. If a page is blocking ALL spiders it will pick it up.
Another thing I would say would be to check in Google Webmaster Tools to see if there are any crawl errors.
And the last thing I would add is to make sure that you have a non-JavaScript way to find all the pages on your website - through strong internal linking or a manual sitemap page that isn't generated through JS.
Hope this helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Canonical question for cross-listed product listings
We have products that are listed across multiple categories. This results in muliple urls for the PDP, for example: mystore.com/shirts/shirt-101.html mystore.com/shirts/pink-shirts/shirt-101.html They make use of the canonical tag and point back to only one product listing url, however Google has indexed both urls in some cases. Has anyone else run up against this and does anyone have advice on how this should be handled?
Technical SEO | | LivDetrick0 -
Basic Redirection Question
I am doing a 301 Redirect from site ABC to site XYZ. I loaded the following .htaccess file by ftp to the ABC.com/ server: Redirect 301 / http://XYZ.com/ This was completed over 30 days ago, OSE is not showing any of the links and is failing to show that abc.com is redirected even though the MozBar shows a successful 301 http status code. Is this still just a waiting game or is it not advised to do a redirect this way for seo? PS: ahrefs is showing the redirect itself, however, it is not showing the links going to site ABC.com/ as passing to site XYZ.com/ . Any help is appreciated.
Technical SEO | | Vspeed0 -
Title Keyword Order Question
Hey there, Hoping someone could provide me with an answer / some insight into this. As an example, if I were to be targeting "Fish markets in" and "Fish shops in" as my keyword(s). And my site name was "The Fish Guide". I am trying to populate my title tags as best as possible without sacrificing readability and definitely making an effort to avoid the the spam-factor. Ideally I'd like to rank well (just focusing on titles at the moment, nothing else) when a user searches for "Fish markets in London" and **"Fish shops in London" **or "Fish markets in Los Angeles" or "Fish shops in Berlin" etc. If I were to use the following structure in my titles (in this case for London): "Fish markets and fish shops in London - The Fish Guide" Would this work as I hope? I.E. If a user searched for either "Fish shops in London" or "Fish markets in London" would the format of my title in this example work or would I need to have "Fish Markets in London" and "Fish Shops in London" grouped together? E.G. "Fish Markets in London - Fish Shops in London - The Fish Guide" Thanks!
Technical SEO | | Gorkonola0 -
Duplicate content question...
I have a high duplicate content issue on my website. However, I'm not sure how to handle or fix this issue. I have 2 different URLs landing to the same page content. http://www.myfitstation.com/tag/vegan/ and http://www.myfitstation.com/tag/raw-food/ .In this situation, I cannot redirect one URL to the other since in the future I will probably be adding additional posts to either the "vegan" tag or the "raw food tag". What is the solution in this case? Thank you
Technical SEO | | myfitstation0 -
Duplicate Titles and Sitemap rel=alternate
Hello, Does anyone know why I still have duplicate titles after crawling with moz (also google webmasters shows the same) even after I implemented (since 1 week or 2) a new sitemap with rel=alternate attribute for languges? In fact, the duplicates should be in the titles like http://socialengagement.it/su-di-me and http://socialengagement.it/en/su-di-me. The sitemap is on socialengagement.it/sitemap.xml (please note formatting somehow does not show correctly, you should see the source code to double check if its done properly. Was made by hand by me). Thanks for help! Eugenio
Technical SEO | | socialengaged0 -
Sitemap Question - E-commerce - Magento
Good Morning... I have an ecommerce site running on Magento and the sitemap is automatically generated by Magento based on the categories and sub categories and products. I have recently created new categories that i want to replace the old categories, but they are both in the auto-generated sitemap. The old categories are "active" (as in still exist if you know the URL to type) but not visible (you can't find it just by navigating through the site). The new category pages are active and visible... If i want Google to rank one page (the new category page) and not the old page (old category page) should i remove the old page from the sitemap? Would removing the old page that used to target the same keywords improve my rankings on the newer category page? Sitemap currently contains: www.example.com/oldcategorypage www.example.com/newcategorypage Did I confuse you yet? Any help or guidance is appreciated. Thanks,
Technical SEO | | Prime850 -
Benefits to having an HTML sitemap?
We are currently migrating our site to a new CMS and in part of this migration I'm getting push-back from my development team regarding the HTML sitemap. We have a very large news site with 10s of thousands of pages. We currently have an HTML sitemap that greatly helps with distributing PR to article pages, but is not geared towards the user. The dev team doesn't see the benefit to recreating the HTML sitemap despite my assurance that we don't want to lose all these internal links since removing 1000s of links could have a negative impact on our Domain Authority. Should I give in and concede the HTML sitemap since we have an XML one? Or am I right that we don't want to get rid of it?
Technical SEO | | BostonWright0