Site: Query Question
-
Hi All,
Question around the site: query you can execute on Google for example. Now I know it has lots of inaccuracies, but I like to keep a high level sight of it over time.
I was using it to also try and get a high level view of how many product pages were indexed vs. the total number of pages.
What is interesting is when I do a site: query for say www.newark.com I get ~748,000 results returned.
When I do a query for www.newark.com "/dp/" I get ~845,000 results returned.
Either I am doing something stupid or these numbers are completely backwards?
Any thoughts?
Thanks,
Ben
-
Barry Schwartz posted some great information about this in November of 2010, quoting a couple of different Google sources. In short, more specific queries can cause Google to dig deeper and give more accurate estimates.
-
Yup. get rid of parameter laden urls and its easy enough. If they hang around the index for a few months before disappearing thats no big deal, as long as you have done the right thing it will work out fine
Also your not interested in the chaff, just the bits you want to make sure are indexed. So make sure thise are in sensibly titled sitemaps and its fine (used this on sites with 50 million and 100 million product pages. It gets a bit more complex at that number, but the underlying principle is the same)
-
But then on a big site (talking 4m+ products) its usually the case that you have URL's indexed that wouldn't be generated in a sitemap because they include additional parameters.
Ideally of course you rid the index of parameter filled URL's but its pretty tough to do that.
-
Best bet is to make sure all your urls are in your sitemap and then you get an exact count.
Ive found it handy to use multiple sitempas for each subfolder i.e. /news/ or /profiles/ to be able to quickly see exactly what % of urls are indexed from each section of my site. This is super helpful in finding errors in a specific section or when you are working on indexing of a certain type of page
S
-
What I've found the reason for this comes down to how the Google system works. Case in point, a client site I have with 25,000 actual pages. They have mass duplicate content issues. When I do a generic site: with the domain, Google shows 50-60,000 pages. If I do an inurl: with a specific URL param, I either get 500,000 or over a million.
Though that's not your exact situation, it can help explain what's happening.
Essentially, if you do a normal site: Google will try its best to provide the content within the site that it shows the world based on "most relevant" content. When you do a refined check, it's naturally going to look for the content that really is most relevant - closest match to that actual parameter.
So if you're seeing more results with the refined process, it means that on any given day, at any given time, when someone does a general search, the Google system will filter out a lot of content that isn't seen as highly valuable for that particular search. So all those extra pages that come up in your refined check - many of them are most likely then evaluated as less than highly valuable / high quality or relevant to most searches.
Even if many are great pages, their system has multiple algorithms that have to be run to assign value. What you are seeing is those processes struggling to sort it all out.
-
about 839,000 results.
-
Different data center perhaps - what about if you add in the "dp" query to the string?
-
I actually see 'about 897,000 results' for the search 'site:www.newark.com'.
-
Thanks Adrian,
I understand those areas of inaccuracy, but I didn't expect to see a refined search produce more results than the original search. That just seems a little bizarre to me, which is why I was wondering if there was a clear explanation or if I was executing my query incorrectly.
Ben
-
This is an expected 'oddity' of the site: operator. Here is a video of Matt Cutts explaining the imprecise nature of the site: operator.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should Google Trends Match Organic Traffic to My Site?
When looking at Google Trends and my Organic Traffic (using GA) as percentages of their total yearly values I have a correlation of .47. This correlation doesn't seem right when you consider that Google Trends (which is showing relative search traffic data) should match up pretty strongly to your Organic Traffic. Any thoughts on what might be going on? Why isn't Google Trends correlating with Organic Traffic? Shouldn't they be pulling from the same data set? Thanks, Jacob
Reporting & Analytics | | jacob.young.cricut0 -
Had suspicious spike in Adsense clicks, next day site ranking tanks
Yesterday, one of my sites had extreme Adsense clicks for several hours in the morning, which brought it up to CTRs of around 120%. My normal CTR is about 10-15%. It added several hundred dollars income over and above my normal amount. After that, it went back to normal. I have waited to see if Google would adjust the income down, as someone or some bot seemingly clicked the heck out of the site's ads. Nothing has been adjusted; it's been 24 hours. Question #1: what usually causes this type of insane clicking to occur (i.e. competitors messing me?) Then, today I noticed something else disturbing. I cannot find my site in the top 100 SERPs for the main keyword. I was at #1 for a couple years, then, when I changed themes from Thesis to Genesis (site otherwise exactly the same) a couple months ago, I bounced around various positions on the first page. In the last couple weeks we've been bouncing between the teens and the thirties. Two days ago we were at #15. (the site is still indexed when I use "site:" to check. It seems awfully coincidental that yesterday I had the Adsense click explosion, and today I'm not even in the top 100 for the first time in my pretty stable two-year history, and have no idea how far behind 100 I am. I went to Google Webmaster Tools and see no errors or warnings relating to this. Adsense has not sent me any messages. So... Question #2: does Google search apply some sort of penalty to site that have suspicious Adsense clicking? By the way, I don't have any funny business going on with any bad SEO practices, it's all above board, and I have thousands of real readers each day Liking and commenting on the pages. It's a very real site. Note: I have been checking the ranking each day via a Google Incognito window and searching for the term. Of course I use MOZ but I do the Incognito search for a quick real time check, which I've found to be accurate.
Reporting & Analytics | | bizzer0 -
Google analytics question
Ok so in my traffic sources break down I have 3 sections: direct, organic, and referral. My question is under the referral tab I have recently noticed a new traffic source, my own website.... How is this possible? My top referring site is my own website.... Is this considered direct traffic or how is this being traced?
Reporting & Analytics | | jameswalkerson0 -
Does anyone know what's happened to google analytics -> traffic sources -> SEO -> queries many of my accounts are showing a drop to zero in the laste few days
Howdy mozzers It's in the question title really. Zero impressions showing for the last few days on. Multiple accounts Any thought out there
Reporting & Analytics | | Big_Partnership0 -
Google Analytics Queries
Hi we used a web company who put our analytics under their company name we have left them and set up a new tracking code not associated with them, but we still have access to the data and it is all contained under the same login, is it possible to merge the two tracking codes? so that we have seemless history. Also does anyone know if you can change the primary e-mail address (gmail account) to a corporate email account? Any help would be appreciated as we have researched both and from what we have found we can't merge the data and also we are stuck with the primary email address.
Reporting & Analytics | | loopylu030 -
An initial set up question - linking to GA
when viewing campaign it indicates there is a problem linking to our GA account but the details entered are correct. Any ideas what the problem might be - nothing on the campaign settings page gives me a clue. There is a highlighted box saying disconnect from GA but nothing else. The only thing i can think of is that the GA account was set up by a third party and we have access to view it - could it be a permissions problem?
Reporting & Analytics | | GreatGifts4Kids0 -
Should you get a new Google Analytics account if your site has a new domain after a site redesign/new development?
We recently developed a new site for a client and they have opted to move forward with a domain change. Should we create a new Google Analytics account for the new site?
Reporting & Analytics | | TheOceanAgency0 -
Unable to use Open Site Explorer
I have repeatedly tried to use Open Site Explorer on a site. I always get the same response: No Data Available for this URL The site is 4 months old. It is listed by Alexa in 1.3 million page category. The site ranks well on numerous key terms for the industry it covers. OSE offers 4 reasons for not having data: 1. Recency of page creation. In the explanation offered 45-60 days is the longest estimate it should take. The site has been live close to 120 days. 2. Deep down in the web. The tool doesn't even have any data for the site's home page. 3. Blocked pages. The site is listed in Google and these pages are not blocked. 4. No links. There are plenty of links. Site url is www.terapvp.com What can I do to fix this issue and get data?
Reporting & Analytics | | RyanKent0