Is the Google results serp broken?
-
Hi everyone!
We've been trying to decipher how many of our pages are indexed by google at the moment. If we do the usual "site:https://www.hobbydb.com" search term, the serp says that we have more than 740,000 pages indexed. However, when I do a deep dive and click through to the last page of results, I can only get to page 54, and then there are no more results. This would mean that I only have 540 pages indexed, not 740,000. We have also done other queries for other sub-sections of our website, and the results also truncate at 50 pages. Has anyone run into this problem? Any suggestions are appreciated!
Best,
Alex
-
Google search console Index Status report
Google Webmaster tools tell you in the
https://support.google.com/webmasters/answer/2642366?hl=en
if you want to know without a doubt you have to connect Google Analytics to Deepcrawl it will say exactly how many URLs are in the index.
- https://www.deepcrawl.com
- https://www.deepcrawl.com/knowledge/best-practice/how-to-measure-indexed-pages-more-accurately/
- https://www.deepcrawl.com/knowledge/best-practice/how-can-non-indexable-pages-receive-google-organic-traffic/
- http://www.stateofdigital.com/40-deepcrawl-tweaks/
Now that Google Analytics can be added to screaming frog you can do something very similar.
- https://www.screamingfrog.co.uk/seo-spider/
- https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#crawling
- https://www.screamingfrog.co.uk/seo-spider/faq/
- https://moz.com/community/q/how-to-determine-which-pages-are-not-indexed
I Hope this helps,
Tom
-
Michael, the site: command typically has not "worked" or provided accurate results for years now. It's just they way it is--and I don't expect Google to really give us the number of pages that are actually indexed anytime soon.
That said, typically I recommend using the site:hobbydb.com command to see the pages, and then click to the final page of results (if you can). I'm currently seeing about 766,000 pages indexed.
-
That index query returns dubious results. URLProfiler wrote an excellent article last year sharing the results of some experimentation they did around the accuracy of that command and other tools used to help determine the extent of your site indexation. I recommend you give it a read - http://urlprofiler.com/blog/google-index-checker/. It will take some some work and access to their tool, but I think you'll gain a lot of insight.
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Not Indexing Submitted Images
Hi Guys! My question isn't too dissimilar to one asked a couple of years ago, regarding Google and image indexing, but having put my web address into a Google image search, I get a return of 15 images, so something isn't right. 5 months ago I submitted our 'new' site to Google webmaster. We have just moved it onto a Shopify platform. They (Shopify) are good at providing places to add titles and Alt tags and likewise we fill them in (so that box ticked!) However I have noticed over the last couple of months that despite 161 images being submitted, only 51 have been indexed. Furthermore and as I said earlier, when you put our site, site:http://www.hartnackandco.com into Google images, it only returns a total of 15 images. Any suggestions and help would be wonderful! Cheers Nick
Technical SEO | | nick_HandCo0 -
PDF in search results?
Hello community! I am not an SEO professional, though I am a practitioner, I would say. I am seeking a solution on behalf of a friend. If you search the term "Peter Blatt" you will discover a "black eye" on the first page, towards the bottom of SERPs. It's a PDF published on the Florida Department of Financial Services website regarding the final order for a settlement he and his company ("Blatt Financial Group") reached with the state as it related to professional conduct allegations. Does anyone have any advice on how to address this? I don't want "game" the search engines, but at the same time, this document looks really scary and much worse than it actually is to people, and I would love for it do drop below page one. Any advice or suggestions from the community? Thanks! Tom
Technical SEO | | 800GoldLaw0 -
Google Search Results Flip-Flop
For a site we manage, Google can’t seem to decide which of two pages to present for a search for “skid steer attachments.” Almost weekly, it flip-flops from the home page to an interior page (which is a shopping cart category page that we have not actually optimized for the phrase.) The site is berlon.com. Have any of you had a similar experience and, if so, how did you address it? I’ve attached a Moz screen shot that shows the changes. mNfmJoY
Technical SEO | | PKI_Niles0 -
Google autorship in specific field?
Hi, I want to ask you about something I 've read about google and authorship. It is written that it is better to show yourself as a author in a specific field. I myself have knowledge and interest in many fields - like SEO, vegan living, martial arts. And I want to be seen as specialist in all of them. Does it mean that we are limited to mark with autorship articles in only one field, in order to be seen as expert in a specific field? f.e. Should I mark with "rel=author" the articles that are about SEO because I want to be seen as author in that specific field for sure. Iif I mark with "rel=author" articles also about martial arts would these affect the understanding about my expertise in SEO?
Technical SEO | | vladokan0 -
How do you get a Google+ pic in your SERP snippet
Hi from from 20 degrees C 83% humidity wetherby UK 🙂 A few weeks back i decided i needed to get my pretty face appearing in my serps for www.davidclick.com But after having set up a Gppgle+ account and linking my site to the Google+ account i think I may have done something wrong 😞 I linked to the Google+ page via a footer link in www.davidclick.com but alas I'm not able to get my face in my SERP which this website has: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/google-plus-picJPGcopy.jpg So my question is please - "How do you get your Google+ account image to appear in the SERPS. Ta muchly,
Technical SEO | | Nightwing
David0 -
Sitemaps for Google
In Google Webmaster Central, if a URL is reported in your site map as 404 (Not found), I'm assuming Google will automatically clean it up and that the next time we generate a sitemap, it won't include the 404 URL. Is this true? Do we need to comb through our sitemap files and remove the 404 pages Google finds, our will it "automagically" be cleaned up by Google's next crawl of our site?
Technical SEO | | Prospector-Plastics0 -
Why is google ignoring my sitelinks demotions?
I'm referring to the sitelinks that appear in the SERPs when searching for my brand name. 6 subpages come up- some found in my main navigation, some not. 3 of the 6 sitelinks have been demoted in Webmaster Tools under "Site Configuration > Sitelinks." I realize that they say in their instructions: "Google doesn't guarantee that demoted URLs will never appear as a sitelink, but we do consider a demotion a strong hint that we'll try to honor when generating sitelinks." I am just surprised they would ignore 3 of the demotions and for so long. The pages that are demoted do not have very many internal links pointing to them (unlike other pages of my site that are targeted specifically). Also, the site has tens of thousands of pages to choose from. Why are they ignoring my request? What else can I do to fix this?
Technical SEO | | Hakkasan0 -
How to disallow google and roger?
Hey Guys and girls, i have a question, i want to disallow all robots from accessing a certain root link: Get rid of bots User-agent: * Disallow: /index.php?_a=login&redir=/index.php?_a=tellafriend%26productId=* Will this make the bots not to access any web link that has the prefix you see before the asterisk? And at least google and roger will get away by reading "user-agent: *"? I know this isn't the standard proceedure but if it works for google and seomoz bot we are good.
Technical SEO | | iFix0