Sitemap include all site links or just ones we want indexed?
-
Got a quick sitemap question. We have a clients site built in opencart and are getting ready to submit the sitmap. The default sitemap setting generates urls right off of the root. For example site.com/product. These urls are also accessible through the site itself. We prefer to give the site some depth and have structured the products so the urls are site.com/category/product. All of the product pages have canonicals including the category so we should not have to worry about duplicate content on the /product page vs the /category/product page. My question is both types of product pages are included in the sitemap at the moment. Since we don't want google to index the /product urls should we leave them off of the sitemap even though they are readily accessible from the frontend(though not linked)? Or just leave them and let the canonical tag be used in directing google as to which urls to index. Thanks in advance.
-
Hi again JS,
I think it's great that you continue to evaluate your platform from all perspectives and evaluate its strengths/weaknesses. Many times, a platform can do a lot of the basics well, but fall short on the details that differentiate us from our competition. For example, opencart may do the basic SEO requirements well, but not include ecommerce microdata (schema.org) which have a high impact on our search listings.
You can do a lot of harm/good with the robots.txt file - like deindex entire website (probably not a good thing) or block certain directories (your /product issue). I would gain some deeper knowledge about what you can do with the robots.txt file and how you need it to perform for your business.
-
Hey Raymond,
Thanks for the response, feel like I'm over thinking this a bit, as usually we just leave our opencart setups as is, other then a few minor tweaks. Lately I've really been scrutinizing opencart's SEO setup and how to improve it, since it seems there are a lot of gaps in he way it handles this.
I thought the robots.txt would have been a good way to block the pages, but the issue is I would need to block every single product page as opencart automatically creates a page for every product that is site.com/product and since we are adding lots of products there should be a better way to handle this. After I posted I came across this tidbit from a 6 year old google webmaster central blog post. Basically it states that 'While we can't guarantee that our algorithms will display that particular URL in search results, it's still helpful for you to indicate your preference by including that URL in your Sitemap. '. I think going this route along with the canonical should do the trick.
-
Hi JStrong,
Great question to be asking and an important topic to be doing your due diligence on, especially when dealing with an eCommerce related website.
Google uses a sitemap as a guideline for crawling your site. So, just because you put a URL in your sitemap, doesn't mean that they URL will actually be indexed. You can see those stats in your Google Webmaster Tools account, under the Sitemap area. It will display how many URLs are in the sitemap and how many out of those URLs are indexed.
If you do not want certain pages to be indexed by Google, then you would need to adjust your robots.txt file to give Google those instructions.
As long as you have the correct Canonical configurations, you should avoid any duplicate content issues from the URLs you've described above.
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Nofollow on internal links
Hi, What do you all think about using nofollow on internal links? On pages that arent so important and use dofollow on important pages maybe for example from the sites menu.
On-Page Optimization | | Rob-0 -
Indexed/Submitted URLS vs Total Indexed
Hello, My site is www.colbysphotography.com. I have Total Indexed 195 while I have 87 URLs submitted and only 79 URLs Indexed. What is the difference and is there a problem? Thanks ahead of time,
On-Page Optimization | | littlecolby
Colby0 -
Question Regarding Site Structure
I have a quick question regarding site structure that I hope some of you guys could share your opinion on. I watched a white board friday from Rand a little while back where he explains that you need to try and make the site structure as flat as possible. He was saying try having no more that 3 links from the home page to get to the desired location. My question is this. I am looking at a site that has a pretty complex structure that I am trying to clear up as much as possible without making any of there rankings suffer. So they have www.domian.com/general-category/district/town/ and sometimes www.domian.com/general-category/district/town/item-specifics Now i know it is not good as it is, but they are dubious about changing too much as they have some serious traffic coming to the site. But, my question is that all the pages can be found from the home page through the menus/sub-menus. But do these count as a direct link from the home page. Also a problem is that because of this mozbot has detected that there are too many links from the home page and suggested that it should be below 200. But should I make these menu links no index or no follow. Obviously, by doing this, if the link does count as direct from the home page it wont after doing this. Thanks Jenson
On-Page Optimization | | jensonseo0 -
I have one page on my site... but still get duplicate name and content errors.
i have only the index.html page. my domain has a permanent 301 to the root. why am i getting duplicate problems? i only have one page the index .html???
On-Page Optimization | | one4u2see0 -
Is it best to optimize your site for just one or two keywords?
My company/website makes and sells a product that's not that competitive but still has about 20 key words/phrases that people search for. My site is not a huge site maybe 35 pages after you include the blog posts.We sell samples off the site but it's mostly used as a brochure but we also want it to be a successful tool at bringing in leads. Should I optimize for the most popular key word phrases focusing on only one or two per page and forget about the rest or should I try to optimize for as many keywords as possible on all pages or should I optimize for just the few (3-5) heavy hitting keywords but on all pages? Right now I've got it optimized for around 3 keyword phrases for the whole site and only 1 or 2 per page with the most popular phrases on the most important pages.
On-Page Optimization | | JAARON0 -
product links
If you sell a range of products say 3 at the most, all on their own pages, is it ok to link to the other products within the range from each page? I have tried this and it eventually leads back to the same page is this a good, bad or doesn't really matter thing? Also is the anchor text still important?
On-Page Optimization | | LadyApollo0 -
Why some pages are not indexed?
I have a furniture´s ecommerce. When searching for "site: movstore.com.br" returned 1080 results, but if I search for "site: movstore.com.br / Product" returned 1020 results. I mean, that 1080 indexed pages, 1020 are products pages and the other 60 pages are irrelevant. Where are the category pages? "site: movstore.com.br / Categories" - 0 results
On-Page Optimization | | maisempresas
"site: movstore.com.br / Departments" - 0 results
"site: movstore.com.br / Marks" - 0 results What might be happening?0 -
Is anchor text link juice passed in internal links?
Let's say I have www.bobswidgets.com. If I have "widgets for sale in tuscon arizona' be the anchor text on an internal link on www.bobswidgets.com/page1 which goes to www.bobswidgets.com/page2 - does the value of the anchor text (widgets for sale in tuscon arizona) get passed internally? External links pass the value of the anchor text but do internal links?
On-Page Optimization | | qlkasdjfw0