How to Diagnose "Crawled - Currently Not Indexed" in Google Search Console
-
The new Google Search Console gives a ton of information about which pages were excluded and why, but one that I'm struggling with is "crawled - currently not indexed". I have some clients that have fallen into this pit and I've identified one reason why it's occurring on some of them - they have multiple websites covering the same information (local businesses) - but others I'm completely flummoxed.
Does anyone have any experience figuring this one out?
-
@intellect did you find a solution to that?
-
-
@dalerio-consulting what should can we do with excluded section then. let say this page of my website is under duplicate canonical tag in excluded section. then should i leave it if its not very serious or should i request indexing ? Are these excluded pages issues very serious to take?
-
Hey Brett!
Basically what we believe this status means is Google saying "I can crawl and access the URL but I don't believe this page belongs in the index". They key here is to figure out why Google might not believe the page should be considered for indexation. We analyzed a good number of Index Coverage reports across all of our different clients.
Here are the most commons reasons URLs get reported as "Crawled - Currently Not Indexed":
- False positives
- RSS Feed URLs
- Paginated URLs
- Expired products
- 301 redirects
- Thin content
- Duplicate content
- Private-facing content
You can find a breakdown of each reason on the post we wrote here: https://moz.com/blog/crawled-currently-not-indexed-coverage-status
However, there's likely many more reasons why Google does't think the page is eligible for indexation.
-
Crawled - Currently not indexed is the most common way for pages or posts on your site not to be indexed. It is also the most difficult one to pinpoint because it happens for a multitude of reasons.
Google needs computing power to analyze each website. How it works is that Google assigns a certain crawl budget to each site, and that crawl budget determines how many pages of your site will be indexed. Google will always index your top pages, therefore, the excluded pages are of less quality rank-wise.
Every website has pages that are not indexed, and the healthy ratio of non-indexed pages will depend on the niche of the website.
There are however 2 ways for you to get your pages out of the "Crawled - Currently not indexed" pit:
- Decrease the number of pages/posts. It's a matter of quality v quantity, so make sure that put more attention into linking every new post so that they get indexed in no time. Don't forget to utilize robots.txt to block pages that aren't useful to the site from indexing so that the crawl budget can be assigned to the other posts.
- Increase the crawl budget. You can do that by raising the quality of the pages/posts. Make more internal and external backlinks for your posts and homepage, make sure that the articles are unique and keyword-optimized, and work hard to aim so that each article will rank on that first page.
SEO is a tough business, but if managed carefully, over time it will pay off.
Daniel Rika - Dalerio Consulting
https://dalerioconsulting.com
info@dalerioconsulting.com -
Crawled - currently not indexed list includes sitemap and robots.txt
We have searched and try to understand this issue. But we did not get final result regarding this issue
If any one fixed this issues, please share your suggestions as soon as possible
-
Hi There,
Google has been struggling to eliminate spam pages, content and structurally ordering them; this is an inherent problem especially with badly structured e-commerce websites.
You might be aware that "Crawled - Currently Not Indexed" means that your page(s) has been found by Google but it is not currently indexed, this might not be an error, just that your pages are in a queue. That might be due to the following reasons:
- There are a lot of pages to index, so it's going to take Google some time to get through them and mark them as either indexed or not.
- There might be duplicate pages / canonical issues for the website of the pages. Google might be seeing a lot of duplicate pages without canonical tags on your site, to improve the number of pages indexed you need to either improve pages so they are no longer duplicated or add canonical tags to help Google attribute it to the correct page
You need to justify each and every page for their merits, and then let google decide whether it think it should be available in their search and also against what keywords at what rank. To summarise, just help 'Google search' by structuring your data right, it might reward you by ranking your pages at right places for the right keywords.Thanks and Regards,Vijay
-
Search Console > Status > Index Coverage > Crawled - currently not indexed
Yes, I had the same Issues last month, in my case the crawler took it 6 weeks to update the Index Coverage. And apparently, there are not too many things that you can do it about it.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Drop in indexation but increase in organic traffic
We've had a puzzling drop in indexed pages on our ecommerce website. My crawl returns just over 25k items. Until 19/6 we had about 23-24k indexed. Then we experienced a sudden drop from 19/6 to 26/6: from 23,400 to 18,999, losing 4.4k pages from one week to the next. At the same time, our organic traffic has not decreased, it actually increased, however, it's only been a couple of weeks so that may be coincidence. A few things that have happened during the past few weeks: 31/5: we implemented pagination on category pages to avoid issues with duplicate content - could it be that this led to a decrease in indexed pages 3 weeks later? However, I can only find about 1.5k pages in my crawl that are page 2+ 18-19/6: we had some website outages over the weekend; as a B2B business, we don't get much traffic over the weekend, so I can't see an impact to traffic. However, the following week, indexation dropped by another 250 (then stayed the same this past week), so I don't think this was a factor. 21/6: we retired another website and migrated it to our main website. However, all pages were redirected to existing pages so no new pages were created for the migration. This doesn't really explain a decrease in indexation, but may account for some of the increase in organic traffic; however not all as the retired website hardly got any organic traffic. So, should we be worried? As our website is quite large, it would probably be quite difficult to pin point exactly which pages dropped off the index, but a loss of 19% of pages is quite significant. Then again, it doesn't appear to have negatively impacted organic traffic... Have you got any suggestions for what I should be looking at to find out what happened? Should I be worried at this point? I will definitely continue to have an eye on how our organic traffic (and indexation) develops but I am not sure if there is anything I can do at this point. I'd appreciate your advice on this, to make sure I am not missing something blindingly obvious. Thanks! RmWaNib JJm4tC3
Reporting & Analytics | | ViviCa10 -
Alternative to Google Analytics
Hey Everyone, My company has just changed the order processing software we are using and it's causing some issues with Google Analytics conversion funnel tracking. Specifically, there is one point in the funnel where making certain selections (which about half the people do) causes the page to reload. Also, on the login/register page, if they miss a field, the software tells them missed the field, but loads a new page which has a different URL which is not a part of the funnel when a mistake like that isn't made. All of this is causing Google Analytics to report people as leaving the conversion funnel when they really haven't. About a third of the traffic is being shown as exiting the funnel with the exit URL being the exact same URL as the step they are supposedly exiting from (example: the visitor enters on page1, moves to page2, Google is showing that they exit on page2 and go to...page2. Does anyone have any suggestions of how to deal with this in Google Analytics? If not, do you have a recommendation of an alternative analytics program which can deal with the situations mentioned above? PS - Changing the way the checkout software works does not appear to be a viable option. Kurt Steinbrueck
Reporting & Analytics | | Kurt_Steinbrueck
OurChurch.Com1 -
Setting up goals - google analytics
Hi Im new user google analytics I would like to set up goal for website. I asked to basic questions: What are my bussiness objectives? Answer:make the visitor to click on a advertisment (affiliate links and affiliate banners) to be redirected to company website to finish purchase. How to set up goals? How can I track ads (affiliate banners and affiliate links) on my website?
Reporting & Analytics | | info_tipovanie-stavkovanie.com0 -
Figuring Out the Source of "direct traffic" by looking at landing page parameters
I have a client who runs an e-commerce website, and I noticed that 40% of his traffic and 25% of his sales are all attributable to Direct Traffic. At first, I tried to solve this problem by tagging all of the previously untagged links in his e-newsletter, which I expect to be very helpful. However, then I looked at the landing pages for his direct traffic, and I see that it is almost entirely filled with thousands of unique URLs that begin with a question mark followed by the name of his e-newsletter or shopping cart vendor. It would be the equivalent of having a url like the following: "www.willmarlow.com/?constantcontact=keya;sldkfjsdlfkjdf;sldkjf" If we have this amount of information in the link, shouldn't there be a way to add additional parameters to the URL to move this traffic out of the Direct column? Has anyone encountered this before? Thanks.
Reporting & Analytics | | williammarlow0 -
Setting up goals in Google Analytics
Hello! This question is seems so obvious that I'm also ashamed to ask... almost. If the goal of a website is to have a visitor complete a contact form, when setting up the Goal in Google Analytics (as a "URL destination") the URL of choice would be the form's thank you page, correct? Because that's the page that proves the visitor completed the task we want them the achieve. Right? Thanks!
Reporting & Analytics | | SmileMoreSEO
Erik0 -
How to create an advanced segmentation for Google+?
Hey Mozzers, I have a question regarding Google Analytics. The problem: I have created advanced segments for Social Media: Facebook, Twitter, Youtube, etc. They all work fine... Now I am trying to create something similar for Google+ (since soon we are launching a campaign for it and I want to be able to check the evolution fast and effective). My advanced segments look like: Include -> Source -> Matching RegExp -> (Facebook|Youtube|Twitter) How should the Google+ be inserted into the group? I know this question might be easy for some advanced Analytics users but I am stuck at this point. Any help appreciated! Thanks in advance! Istvan
Reporting & Analytics | | Keszi0 -
Google Search Results inconsistent from different computers
Recently after some optimization activities - I do not see much movement in search rankings - my client is seeing the results on page 1 position 3 and I see page 2 for the same keyword. How does Google change ranking based on past searches and how can I get an accurate picture of what the actual rank is?
Reporting & Analytics | | devonkrusich0 -
Backlinks on Google Webmaster Tools
As I was reviewing my Google Webmaster tools, I notices a major drop in the number of back links. It used to show over 4,500 links to my site yet the other day it dropped down to 41. None of the directories I've submitted to are showing, nor any blog comments I've posted. Since then, my SEO traffic has started to drop for some keywords. Anyone know why?
Reporting & Analytics | | MikeAndres0