Google site: search showing twice amount of indexed pages. why?
-
I have around 50k pages indexed on my site but when I do a google site: of my site it shows around 100k pages indexed. Why is it showing so much more?
It is also only showing around 700 pages indexed in my web masters account for the site.
Background: We have a custom site map being generated automatically.
Let me know if you would like more info, Thanks.
-
are you sure it's an http vs https, and not a www vs non-www issue?
Thanks again
-
Thank you!
-
You haven't blocked http but you've implemented https. So you're allowing Google to crawl both versions of each page. And yes, you should probably block the non-https.
-
First of thanks for the response,
our site is a online database, We have many pages that show examples of the database, Would this count as duplicated pages?
and as of 12/30/12 I had 50K pages indexed on GWT and now I have around 700.
-
Does your sitemap include duplicate pages or pages that crawlers wouldn't want to list? (like search results pages, pagination of duplicate pages, etc.)
How do you know that you have 50K indexed pages if GWT reports 700 and a site: search reports over 50k?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Long list or paginated pages
Hi peeps, I am just interested in this from a usability POV and to see what you would prefer to see when you are met with a page that has multiple options. Lets say that the page looks like a list of services, each clearly marked out in its own segment, but there are 50-60 options that match your requirements. Do you like to keep scrolling, or would you prefer to take what is there and then move on if you feel you want to dig deeper? Would you like to see a long list, of have the options loaded in as you get to them? -Andy
Search Behavior | | Andy.Drinkwater2 -
Location Pages
Hi. A client of mine offers multiple services and covers a region of the UK. They want to target each major town/city within this area. However, there are 20 cities and services offered range from 5-15 services. I am in the process of creating a location page for each city, so it can be optimised separately however I am not sure if there is a better way to do it? Or should I create a page for each city & service. So I for example I end up with 10 London pages with each one offering a different service? These can all be optimised for different services within London then? Any suggestions? Thanks
Search Behavior | | YNWA0 -
Displaying different site content to users who have already visited your site
I've seen and heard about the concept of treating repeat site visitors differently, displaying different content based on behavior etc. Not sure what it's called buy Hubspot seems to offer something like this with their platform. Anyone know of a third party app (Wordpress perhaps) or tool that does this? How does this even work? Thanks for the help!!
Search Behavior | | RickyShockley0 -
When Googling site:mydomain.com what does listing order tell us?
To find all the pages on my site that are indexed by Google I can search using site:mydomain.com and it gives pages of results. But what does the order of results relate to? Is it page rank or strength? My list of pages doesn't appear to be in order of strength. And it's definitely not by age or alphabetical...
Search Behavior | | GregB1230 -
My website disappears off google!
So this might be kinda of a weird question... Every morning and night I check the ranking of a website that I am building.. The ranking has gone up a lot the last two months. It went from the fifth page to now the second page. I have a issue where some days I check Google my website is completely gone! I go through every page for my keyword and it's not there! After a couple of days of frustration I check again and all of a sudden it is there but now at a higher ranking... I went through the code to make sure there's a not a not follow code in the robots.txt page... Btw another weird thing is so then I look up my website on a google out of country like google.sg and I'm ranking first page like number 5 but again disappeared off google usa. Literally driving my crazy.. does anyone know why this could be? Btw the first time it disappeared I went into webmasters and sent a request because I thought I got penalized but they responded they could not find any spam and I was NOT penalized...
Search Behavior | | BecCan0 -
Google ranks our competitor above us on 1000's of branded queires!!!
Hi all, I have noticed a very bizarre phenomenon in Google SERPs. When I search for a branded keyworks [Product + our brand].
Search Behavior | | ref.price
Amazon.fr appear above us on thousands of results. Google even ranks Amazon above us for queries like [ PriceMinister google plus]. I have tried to ask Google about it but I can’t seem to get an answer. Here is the topic I posted on Google’s forum: http://productforums.google.com/forum/#!category-topic/webmasters/crawling-indexing--ranking/DFvTPr14o_o This seems like a mistake on Google’s side, some kind of semantic association with our two brands! Basically they are sending our customers to our main competitor even though they specifically searched for our brand (PriceMinister). I find the phenomenon quite interesting for the SEO community and frustrating for our company. Does anyone have ideas on this one? Do you think it's a bug from Google? Cheers Oliver0 -
Blog posts not getting indexed and being outranked by scrapper sites.
Our Google traffic has dropped significantly over the last year and now we're struggling to even get our blog posts indexed. It's been extremely discouraging and we're trying to do what ever we can to fix it. I've included a screenshot of our Google traffic as well as Pages Indexed according to WebmasterTools. http://i.imgur.com/Wu1D8.jpg The Problem Our blog posts are frequently not getting indexed. Many times they are outranked by low authority scraper sites, our Twitter/FB account, etc. Sometimes our homepage will rank instead of the blog post. Sometimes we'll break a news story, get tons of quality backlinks, and still be nowhere in Google. Pretty much the only Google traffic we see is from existing posts. Still 3,200 pages indexed when we have only 1,600 posts. I guess this isn't really a problem... just waiting for the meta noindex to take effect. More details We've seen no duplicate content or other warnings from WebmasterTools. We've been constantly acquiring quality backlinks from credible sites. We deleted the useless content and fixed the canonical issues that were a result of switching servers. History Our site is a news/entertainment blog. The traffic usually has spikes depending on what's going on in the news. Nov 1, 2011 - Site kept maxing out at 30k+ visits so we switched servers. Jan 30, 2012 - Hired a writer so we could focus on other aspects of the site. Apr 19, 2012 - Noticed our posts weren't getting indexed like they used to. Suspected our writer was spinning articles but couldn't find any evidence. 90% of our blog posts were nowhere to be found in Google. Scrapper sites would outrank us for our own stories... even our Twitter account was ranking ahead of us. IF our story would show up in Google it would usually be the home page instead of the blog post. Sep 2012 - Finally got more serious about addressing the problem. Noticed a couple potentially big problems and started making changes. Canonical Issues non-www site didn't redirect to www. It showed 2 different link profiles according to OpenSiteExplorer and 0 backlinks according to Webmaster Tools. Wordpress shortlinks weren't redirecting to the actual permalink. For instance http://www.domain.com/?p=123 and http://www.domain.com/post-example were both getting indexed. For every post there were 4 different versions that Google had to choose from. http://domain.com/?p=123, http://www.domain.com/?p=123, http://domain.com/post-example, and http://www.domain.com/post-example I figured the canonical issues must have happened when we switched servers which was the reason for the drop in WebmasterTools indexed pages and increase in Not Selected pages. FIXED (Sep 15): One we fixed the canonical issue the Indexed Pages went back up however the Not Selected is still the same. Duplicate Content When we first created our site we wanted to have tons of images for each musician/athlete/actor/etc. so we uploaded about 5-10 for each person. We created a blog post for each image with no writing and the exact same post titles. As a result there were TONS of low-quality, similar posts, with virtually identical permalinks. e.g. http://www.domain.com/james-smith1, http://www.domain.com/james-smith2, http://www.domain.com/james-smith3, etc. A crawl on Sep 26 showed over 550 duplicate content warnings. FIXED (Oct 1): We deleted/301 redirected the useless pages (they weren't getting traffic anyways) and by the next crawl the number was almost to 0... which it's at now. We also had TONS of tags (since there're constantly new names in the media) that were getting indexed so we had meta robots noindex them. Questions: Why aren't a majority of our posts getting indexed? Were we penalized or just stuck because of a filter? How long should it take for meta robots to noindex the tags pages? (I did it on Sep 25 but they are still there) If a site is scraping our content (same title, image, excert) but linking to us, should we contact them and tell them to remove it? Is there anything else we need to do start getting our blog posts indexed like they used to? Should we try contacting Google to re-evaluate our site? Sorry, that was a LOT of writing. If anyone wants the URL please let me know so I can PM it to you. Any help would be greatly appreciated! Wu1D8.jpg
Search Behavior | | gfreeman230 -
Long page - good or bad?
Our attorney wrote a dozen articles that range from 300 to 700 words on various topics of the certain law area. These articles are all placed on our FAQ page with anchored table of contents. This page does frequently come up on the first page of the google when people search for the questions discussed in these articles. 90% of these visits are not local therefore they are not potential clients. Attorney views it more like a community service then a marketing tool. However, I think there might be a problem. People read though the page and close it because usually they can find what they were looking for right there, however GA counts it as bounce because they did not browse to another page. Would large number of bounces hurt our standing with Google? Would it be better to separate the page into multiple pages for each article to make visitors browse?
Search Behavior | | SirMax0