Should I include unnecessary pages in the sitemap.xml
-
I have a lot of pages that I don't want Google to index, so for most of them, I used cannonical, were they were duplicates, noindex were I wanted to remove the pages, but the question is: Should I include these pages in the sitemap.xml, or just the important pages?
Also should I include them in order to get the changes indexed fastet by Google?
-
That clearly changes my ideas about this ;-). As we're talking about a couple of million pages I wouldn't include them in the sitemaps then and to make sure they're absolutely made sure that it's noindexed.
-
One of the main problem is that there are a lot of such pages (aprox. 2-3 milions) and my indexation rate is really slow for a site this big. The old sitemap structure was to complex, and I wanted so simplify it, so Google wiil crawl only the important pages
-
Hi Silviu,
Hard question, related to your use case I would suggest not to include them. But on the other hand it also shouldn't harm your performance as the URLs in a sitemap are mostly meant to search engines as a full list of URLs they might miss otherwise. It would also help you to see what your indexation rate is. Curious to see what other people think about this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I utilize URL re-writes to include keywords and other optimised page elements on my website?
Hi there, I am working on a medical recruitment website: https://wave.com.au/ I have noticed that our Job Search and Job Listings section of the website is lacking a little bit in terms of SEO optimisation. For example, at the moment this is our search page (with locum work type selected): https://wave.com.au/search-locum/results If you add a location, for example, NSW and then click search again, it updates the URL to: https://wave.com.au/search-locum/results?LocumSearchForm_Location[]=NSW&action_doSearch=Search+jobs I did a check what some competitors and leaders in the recruitment industry were doing and I came across: https://www.ochrerecruitment.com.au/jobs/anaesthetics/new-south-wales/locum/ If you click the different filters/options, it updates the URL to include more clearly defined categories. Some websites would even have a H1 heading tag that would update based on the filters/options you selected. Should I set up a set of URL re-writes and re-structure my website a little bit so that dynamic URLs change to static etc.? Does anyone have any best practice knowledge in regards to this? I have been referencing the following article: https://moz.com/blog/dynamic-urls-vs-static-urls-the-best-practice-for-seo-is-still-clear
On-Page Optimization | | Wavelength_International0 -
Where to position a new page?
Hi there 🙂
On-Page Optimization | | Enrico_Cassinelli
Our website is about a particular region in Italy, the Langhe area, famous for food and wine (barolo and barbaresco are produced here). We need to rollout a few new pages about cellar/winery tours: one main page with the list of tours, and the various subpages for each tour. We already have a page about travel, and a page about wine (with a sub-page about wineries). The URLs looks like:
langhe.net/travel/
langhe.net/wine/wineries/
(Note: i'm translating from italian here) Now, I'm wondering where is better to position the new pages:
langhe.net/travel/winery-tours/name-of-tour/ or
langhe.net/wine/wineries/tours/name-of-tour/ From an SEO perspective (within my limited experience) the first option has a shorter URL, but the second feels more "natural" to me. What do you think? Thanks 🙂
Best0 -
Category pages - SEO or deindex?
What is the best thing to do with category pages? Should I deindex them or use SEO on them? I use the Thesis theme and the Wordpress SEO plugin. I am just not sure what to do with category pages. Also will they create duplicate content?
On-Page Optimization | | dealblogger0 -
Duplicate Page Content on Empty Manufacturer Pages
I work for an internet retailer that specializes in pet supplies and medications. I was going through the Crawl Diagnostics for our website, and I saw in the Duplicate Page Content section that some of our manufacturer pages were getting flagged. The way our site is set up is that when products are discontinued we mark them as discontinued and use 301 redirects to redirect their URLs to other relevant products, brands, or our homepage. We do the same thing with brand and manufacturer pages if all of their products are discontinued. 90% of the time, this is a manual process. However, the other 10% of the time certain products come and go automatically as part of our inventory system with one of our fulfillment partners. This can sometimes create empty manufacturer pages. I can't redirect these empty pages because there's a chance that products will be brought back in stock and the page will be populated again. What can we do so that these pages won't get marked as duplicates while they're empty? Write unique short descriptions about the companies? Would the placement of these short descriptions matter--top of the page under the category name vs bottom of the page underneath where the products would go? The links in the left sidebar, top, and in the footer our part of our site architecture, so those are always going to be the same. To contrast, here's what a manufacturer page with products looks like: Thanks! http://www.vetdepot.com/littermaid-manufacturer.html
On-Page Optimization | | ElDude0 -
Page Analyzer & Page 1
I follow the recommended things from the Page Analyzer or Grader, and I am like position #40, so how do I get to page #1 as a minimum.
On-Page Optimization | | sansonj0 -
No follow for html sitemap?
Hi, I have been working on an e-commerce site and have been wondering if i should add the meta robot tag with no follows, on pages like delivery , terms, returns and the html sitemap? From what I have read this seems a good idea, but i am a little confused as what to do with the html sitemap. I can understand that having this on the homepage will allow crawlers quick access to the deeper pages within the site. But would it be better to guide them down the natural route of the category navigation instead? Thanks
On-Page Optimization | | Bmeisterali0 -
Organic Landing Pages...
For one of our sites (fastcubes.com) I noticed our landing pages were ranking and getting a few organic visits. Considering they were made specifically for PPC, I thought maybe we should create landing pages that would not be present in the navigation of the site but for the purpose of optimizing for keyword variations. For example work station cubicles vs office workstation. We have a page optimized for office workstation but having another optimized for work station cubicles is redundant. Would it be a good idea to create this as a page that is not present in the navigation for the sole person of hopefully being ranked and getting traffic for that specific keyword? Thank you in advance for your help!!
On-Page Optimization | | DevonIntl0 -
Duplicate pages
Hi, I am using a CMS that generates dynamic urls that according to the SeoMoz tool will be indexed as duplicate pages. The pages in questions are forms, blog-posts etc. that are not crucial to achieve ranking for. I do worry though about the consequences of having 20 (non-duplicate)pages with static urls and about 100 pages that are duplicates with dynamic urls. What consequences will this have for the speed that the robots crawl the site and could there be negative effects on ranking for the entire domain?
On-Page Optimization | | vibelingo0