Sitemap Size effect SEO
-
So I've noticed that the sitemap I use has a capacity of 4500 URLs, but my website is much larger. Is it worth paying for a commercial sitemap that encompasses my entire site?
I also notice that of the 4500 URLs which have been submitted, only 104 are indexed. Is this normal, if not, why is the index rate so low?
-
As Thomas said, 50k is the max but you can also have an index file with multiple sitemaps within it so you can have lots more if needed. A couple of things I would add.
- Manual crawling is fine but remember, that also means that pages that can't be crawled will be excluded from the sitemap which defeats the purpose of the sitemap
- As for why the indexation levels are so low, the first thing I would check is if everything in the sitemap is a 200 response. Make sure there aren't any redirects or 404s otherwise Google may decide not to trust the sitemap.
I hope this helps
-
Sitemaps can go up to 50,000 urls, so 4500 isn't an issue for Google to handle.
https://support.google.com/webmasters/answer/183668?hl=en
Just to add on to Bernadette's post, this section "If your website's CMS can't do that, then I would recommend crawling the website yourself and updating the sitemap file. However, keep in mind that if you do it manually then you'll need to update it whenever you add or remove a page on the website." You can use something like Screaming Frog for this if you were wondering what software to use. It's a very common tool for an SEO to use, so hopefully you'll have it to save purchasing a license to something else.
-
Spacecollective, your website's sitemap file(s) don't have any direct impact on search engine rankings. If you have a large website with over 4500 URLs, then most likely you're using content management system (CMS) that "should" be able to create a sitemap file.
If your website's CMS can't do that, then I would recommend crawling the website yourself and updating the sitemap file. However, keep in mind that if you do it manually then you'll need to update it whenever you add or remove a page on the website.
Typically, if your navigational structure is set up in a way that all pages on the website can be crawled via links on the site, you generally shouldn't have anything to worry about.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Getting 'Indexed, not submitted in sitemap' for around a third of my site. But these pages ARE in the sitemap we submitted.
As in the title, we have a site with around 40k pages, but around a third of them are showing as "Indexed, not submitted in sitemap" in Google Search Console. We've double-checked the sitemaps we have submitted and the URLs are definitely in the sitemap. Any idea why this might be happening? Example URL with the error: https://www.teacherstoyourhome.co.uk/german-tutor/Egham Sitemap it is located on: https://www.teacherstoyourhome.co.uk/sitemap-subject-locations-surrey.xml
Technical SEO | | TTYH0 -
Help Center/Knowledgebase effects on SEO: Is it worth my time fixing technical issues on no-indexed subdomain pages?
We're a SaaS company and have a pretty extensive help center resource on a subdomain (help.domain.com). This has been set up and managed over a few years by someone with no knowledge of SEO, meaning technical things like 404 links, bad redirects and http/https mixes have not been paid attention to. Every page on this subdomain is set to NOT be indexed in search engines, but we do sometimes link to help pages from indexable posts on the main domain. After spending time fixing problems on our main website, our site audits now flag almost solely errors and issues on these non-indexable help center pages every week. So my question is: is it worth my time fixing technical issues on a help center subdomain that has all its pages non-indexable in search engines? I don't manage this section of the site, and so getting fixes done is a laborious process that requires going through someone else - something I'd rather only do if necessary.
Technical SEO | | mglover19880 -
Optimized filename management for SEO
Hi, I'm currently undertaking a project to manage better my images and to rank for image searches. My site uses mod_pagespeed to optimises images, therefore image urls are changed to a pretty seo unfriendly url or even sometimes inlined within my page. On top of that I generally resize all my images by passing the desired file dimension within the image url in order to optimise the loading time of my site. Is there a way to provide a hint to google that I have a much better quality image with higher dimensions and better name? Can i specify within my image sitemap an image that actually does have different dimensions and a different within the page it serves? Are there better tricks to achieve that? Thanks in advance
Technical SEO | | mattam0 -
When Should You Start SEO?
I am launching a new website (related to IT services) on Monday 6th May 2013. What should be my SEO/SMO/PPC strategy for a brand new website with new domain ? I have a blog within the website as well. Is it better to promote internal blog or should i focus on external bogs like wordpress ?
Technical SEO | | afycon0 -
Cumulative effect
<a name="_GoBack"></a>I have a real estate client who creates individual property sites (mini-sites), each with their own URL (123MainStreet.com), each linking to 2 or 3 diverse pages on the main broker site (main site). There are a couple thousand on these pages at any one time, feeding into a large broker site. The SEO effect of these mini-sites is that there are many incoming links to the main site from various URLs, however, the mini-sites have little SEO relevance, given that there are few, if any inbound links to the mini-sites themselves, and the mini-sites only exist for the market life of a property. My question is: Does it make more SEO sense to create a series of pages (externalsite.com/123mainstreet) on a single site that already has good traction and relevance? The inbound links to the main site will have more relevance, but they will all be coming from the same domain (or subdomain). And although the links will all be coming from the same external site, the inbound links will be from a variety of pages on external site to a variety of pages on the main site. Any insight would be appreciated.
Technical SEO | | mikescotty0 -
Domain redirection and seo implications
We have an existing site that is a subdomain but we recently acquired an exact match domain. Will building links to the exact match domain and having the domain point at our existing subdomain work or should we convert the entire site and redirect our existing subdomain to the new domain? What I'm trying to figure out is how to maximize the benefit here and how the existing mass of links pointing to our existing subdomain (shop.domain.com) can be used. New domain: keywordshop.com Existing URL: shop.domain.com
Technical SEO | | CHarkins0 -
What are SEO factors in re-doing a website?
Most of my work now involves converting older websites to CMS-based sites (in Wordpress) and I'm wondering about best practices here. If I create a "dev" or "sandbox" directory for my development work how do I keep the pages from being indexed while I am working on the new site? Can I "noindex" a directory? What do I do with the old html files when the new site goes live? I'm assuming I will do a 301 redirect from domain.com/index.html to the new domain.com/, and also on all of the inner pages that have equivalent pages in the new site. But there will be a lot of old files left that have no equal in the new site. Do I just delete these, or noindex nofollw them?
Technical SEO | | bvalentine0