Indexed URLs in Webmaster Tools
-
Hi everybody! I've been looking at my Webmaster Tools stats, and it looks like not all the URLs in the sitemap tree have been indexed, according to WMT at least. Is this reliable, and if so, is it worth investigating further?
|
|
|
|
|
|
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
1,309
|
| || |
Index
|
Dec 15, 2011
|
8,695
|
4,127
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
998
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
819
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
719
|
| || |
Sitemap
|
Dec 15, 2011
|
695
|
282
|
Thanks!
-
Hi,
Your example doesn't look out of the ordinary.
Submitting a sitemap doesn't automatically result in indexing of all pages. There are a number of factors that will influence which pages are indexed. These include predetermined allocations of crawl resources by the search engine, relevance/quality of content on the pages, accessibility etc.
As a site gains in popularity, adds and improves content etc then allocations and indexed pages can increase over time. If you see a sudden or significant decrease in the number of pages indexed (without any action by you to remove them) then you would need to start investigating what might be causing it.
If there are specific "money pages" that you are particularly concerned about you can manually check for them in search engines. If they are not returned in the result pages, that would indicate that they are not indexed. In this situation you can use manual indexing features to add individual URLs to both Google & Bing through WMT & Webmaster Central. To do this in Google, use "fetch as Googlebot" and once the URL is successfully fetched, you will be offered the option to add the URL to the index. There are monthly & daily limits on the number of URL's you can add this way.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to turn on persistent urls in WordPress?
I'm using an appointment form on my website and I have the option to add a referral url to form submissions so that i know which pages the form submission came from. I need to be able to distinguish between organically generated form submissions and those that come in via AdWords. If referral url shows the AdWords tracking code i know the form submission came in from AdWords. My problem is that when a visitor comes in after clicking an ad and then visits another page on my website that AdWords tracking code disappears from the url. I was told that there was a way to turn on persistent urls in WordPress but I can't figure out how to do it. I'm assuming that if i turn persistent urls on the AdWords tracking code will remain on every subsequent url that they visit on my website. Is this true? Any help with this will be greatly appreciated.
Reporting & Analytics | | SpaMedica0 -
Sudden Increase In Number of Pages Indexed By Google Webmaster When No New Pages Added
Greetings MOZ Community: On June 14th Google Webmaster tools indicated an increase in the number of indexed pages, going from 676 to 851 pages. New pages had been added to the domain in the previous month. The number of pages blocked by robots increased at that time from 332 (June 1st) to 551 June 22nd), yet the number of indexed pages still increased to 851. The following changes occurred between June 5th and June 15th: -A new redesigned version of the site was launched on June 4th, with some links to social media and blog removed on some pages, but with no new URLs added. The design platform was and is Wordpress. -Google GTM code was added to the site. -An exception was made by our hosting company to ModSecurity on our server (for i-frames) to allow GTM to function. In the last ten days my web traffic has decline about 15%, however the quality of traffic has declined enormously and the number of new inquiries we get is off by around 65%. Click through rates have declined from about 2.55 pages to about 2 pages. Obviously this is not a good situation. My SEO provider, a reputable firm endorsed by MOZ, believes the extra 175 pages indexed by Google, pages that do not offer much content, may be causing the ranking decline. My developer is examining the issue. They think there may be some tie in with the installation of GTM. They are noticing an additional issue, the sites Contact Us form will not work if the GTM script is enabled. They find it curious that both issues occurred around the same time. Our domain is www.nyc-officespace-leader. Does anyone have any idea why these extra pages are appearing and how they can be removed? Anyone have experience with GTM causing issues with this? Thanks everyone!!!
Reporting & Analytics | | Kingalan1
Alan0 -
Bing Webmaster Tools data discrepency on traffic
Hi, For a given time period in Bing Webmaster Tools it says that combined clicks from Bing and Yahoo organic search are about 4,700 total, for the same time period in Google Analytics, combining sessions for Bing and Yahoo Organic comes out to about 8,900 total. Has anyone else experienced this discrepancy? Is this common? How can I get 8900 visits from 4700 clicks?
Reporting & Analytics | | IrvCo_Interactive0 -
How to setup Google Analytic goals that go to different urls?
I would like to setup my form's conversion rate as a goal in my Google Analytics account. The goal is complete when someone enters their information and hits submit. Logically, the "Thank you page" would be the destination url for the goal. My problem is that the thank you page is outside of the primary domain, for example, my site is: SiteExampl.com but the Thank you URL is thankyouurl.com Is it possible to setup a goal like this? If so how. Thanks in advance!
Reporting & Analytics | | christinarule0 -
Link being indexed
So I found this link to my website on the huffington post http://www.huffingtonpost.com/2012/11/13/california-car-insurance-rates-vary-study_n_2122614.html it's at the bottom in the "around the web" section. My question is this article has been around for almost 4 months yet the link does not show up in WMT. I would like to know if the link to http://www.shiftins.com is indexed and passing authority. Thank you.
Reporting & Analytics | | jameswalkerson0 -
URL Re-Structure - Tracking success of it
Hi guys, I was wondering what would be the best approach to track the success of a URL restructure? What we plan to do is to implement the URL re-structure slowly by only having it on new pages which go live for property listings. Any previous listings will use the old URL structure. I thought it would be best to limit any potential problems by testing it on a smaller number of pages. So my question really is, what metrics should I be looking at to determine the success of this given the fact that we remove any property listings once they get rented or sold?
Reporting & Analytics | | MarkScully0 -
Interpreting Keyword Difficulty Tool results
I have been running various reports using the Keyword Difficulty tool. The columns are color coded - are there one set of signals that are more important than others? I am trying to analylze why we are ranking where we are and where our competitors are weak enough to overcome in a short term. I cannot seem to find a formula for the top indicatators for moving up in ranking. One of my sites in ranking number 5 but is neck and neck with the number one site for results. The other site is number 6 for the same term but has virtually no strength in the data like links, mozrank, trust, Social, etc...The sites 7-10 are much stronger on data. Maybe since they are from the same C block that helps? Any advice or experience would be helpful.
Reporting & Analytics | | devonkrusich0 -
Google News traffic spike mystery; referring URLs all blank, Omniture tags didn't fire.
Our content is occasionally featured in Google News. We recently have had two episodes where this happened, but (a) nearly all the referring URLs were blank, and (b) our backend logs show 3-4x more requests for the article in question than Omniture does. In other words, hundreds of thousands of visitors requested a URL from our site (as proven by the traffic logs), but don't seem to have come from Google News (because HTTP_REFERER was blank), and didn't execute the onpage javascript tag to notify Omniture of the pageview. Perhaps this has nothing to do with Google News, but it is too strong a coincidence that the two times we were on there recently, the same thing happened: big backend traffic spike that is not seen by Omniture. It is as if Google News causes browsers to pre-fetch our article without executing the javascript on the page. And without sending a referring URL. Has anyone else seen anything like this before? Stats from the recent episode:
Reporting & Analytics | | mcglynn
- 835,000 HTTP requests for the article URL (logged by our servers) - these requests came from 280,000 distinct IP addresses (70% US) - the #1 referring URL is blank. This accounts for 99.4% of requests. Which, in itself, is hard to believe. These people had to come from somewhere. I believe browsers don't pass HTTP_REFERER when you click from an SSL page to a non-SSL page, but I think Google News doesn't bounce users to SSL by default.That said, we do see other content pages with 70-90% blank referring URLs. Rarely 99+% though.0