Bing's indexed pages vs pages appearing in results
-
Hi all
We're trying to increase our efforts in ranking for our keywords on Bing, and I'm discovering a few unexpected challenges. Namely, Bing is reporting 16000+ pages have been crawled... yet a site:mywebsite.com search on Bing shows less than 1000 results.
I'm aware that Duane Forrester has said they don't want to show everything, only the best. If that's the case, what factors must we consider most to encourage Bing's engine to display most if not all of the pages the crawl on my site?
I have a few ideas of what may be turning Bing off so to speak (some duplicate content issues, 301 redirects due to URL structure updates), but if there's something in particular we should monitor and/or check, please let us know. We'd like to prioritize
Thanks!
-
Yep, if Bing Webmaster Tools doesn't show problems with the sitemap, I'd focus on the points I highlighted back in mid-June on this thread (make content robust, unique, and make sure text is in HTML).
Good luck,
Kristina
-
Hello again Kristina
Bing's showing 38,885 pages indexed... and I've noticed the amount of pages vary after clicking through several pages.
So I guess the problem isn't why aren't they indexing, but rather why aren't they showing all pages. I'd assume this is related to page quality (content, on-page ranking factors, etc)?
-
I haven't heard of Bing keeping historically submitted sitemaps and confusing them, although I know that they're very picky about the number of inaccuracies they find in a sitemap, so it's possible they keep the latest one around so they can refer to it if the current one seems to have holes.
That said - when you search for your site, are the same pages coming up on the first page? What about the second? Third? The number of pages that come up when you search for site:mysite.com are approximations and can vary even as you scroll through the results pages. The more important question is, how many pages does Bing say are indexed in Bing Webmaster Tools?
-
Just an update:
Bing reported a successful crawl after submitting a new one, then rejected it based on an error that it didn't describe. Took it down, made a change to URL itself (somehow the .gz extension wasn't there) and resubmitted on 7/7/13.
Since then, Bing has reported a successful crawl, then reported a successful crawl on 6/30/13 (7 days before submission?), then reported a failed crawl on 7/5/13 (2 days before submission?) and now today again reporting a successful crawl on 7/7/13.
So my question now is... does Bing keep record of historically submitted sitemaps and confuse them with new submissions of the same ones? I've yet to see Bing actually index what's in the sitemaps, as a site: operator search is still a daily fluctuation between 1200 and 3300 results, sometimes going up to 4400. But again, this is daily. Right now, searching site:roadtrippers.com on Bing reports 4,420 results. Later today, I imagine it'll be around 3,300 or 1,200.
Any suggestions at all would be greatly appreciated.
-
Good luck!
If these tips don't work, you should follow up here again, but include a little more information about your site. It's possible that Bing IS crawling all of your pages properly, but something about them is making Bing think that they aren't valuable enough to be in their indexes. I'd particularly look to see if:
- Content seems to be duplicate, either within your site or if it's duplicated elsewhere
- Content is extremely thin (less than 100 words on a page/no unique text above the fold)
- Content is unreadable by Bing: check the cached version of a page that's not indexed and make sure you can read the unique content
Hope this helps! I'm going to mark this question as "answered," only because if you have a follow up question, it'll probably be more specific now that you have more information, and I'd like all of that info to be included in the original question.
Best,
Kristina
-
Hey Kristina
It has not unfortunately.
Bing reports successful crawls, however it's not crawling it - at all.
After reading more about Bing's sitemap preferences, there are a few things left to try. I'm using this post on Bing's forums http://www.bing.com/blogs/webmaster/f/12248/t/659635.aspx#9602607 as a reference for now. We're going to make a temporary separate sitemap for Bing to test what is suggested in that link. Hopefully something sticks and we can make some progress going forward!
Brandon
-
Hi Brandon,
Just wanted to check in - did using 1 sitemap work?
Kristina
-
I believe I've found the solution - as recently as 2009, Bing was only crawling one sitemap per website. It also said Bing would only crawl the most recently submitted sitemap but it doesn't appear that was the case for our site.
So I've since removed the old sitemap and am waiting to see some evidence of our new sitemap being crawled and indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Clicks are the ultimate factor to stick the page on position?
Hi all, We know many factors contribute to make a page rank at (top) position like somewhere in top 5 results. I have seen some of our pages suddenly spike to that positions and locked there. They been receiving clicks too. Will they be dropped if they don't get estimated clicks? I think many factors contribute to make a page rank higher but clicks are the one factor which makes the page consistently rank at its best position. What do you say? Thanks
Algorithm Updates | | vtmoz0 -
Homepage Index vs Home vs Default?
Should your home page be www.yoursite.com/index.htm or home.htm or default.htm on an apache server? Someone asked me this, and I have no idea. On our wordpress site, I have never even seen this come up, but according to my friend, every homepage HAS to be one of those three. So my question is which one is best for an apache server site AND does it actually have to be one of those three? Thanks, Ruben
Algorithm Updates | | KempRugeLawGroup0 -
Canonical from NOINDEX,FOLLOW pages - Bad idea?
Hi, We have an extensive online shop in Magento - to ensure that some of the pages with query strings are not indexed, we implemented a conditional NOINDEX,FOLLOW so that it will stop indexing any pages that have querystrings on it - We do need to use Canonical also - for other reasons - so my question is: If you have a page that is NOINDEX,FOLLOW and it has a rel canonical pointing to original, would it transfer that NOINDEX,FOLLOW to the main original page causing it problems? Thanks!
Algorithm Updates | | bjs20100 -
Rankings fluctuating by around 10 pages between night and day
Hi all, I'm experiencing something very odd with my website ranking at the moment. My homepage is fluctuating in rank for my main keyword by 10 pages every day and night. So, during the day i am on page 14, 15 or 16 for my main keyword yet by night i am on page 5 or 6. This trend has continued for the past 7 days now and i can't quite understand why this is. I'm using pagewash dot net to carry out manual searches and a ranking tool - both of which produce exactly the same result. Does anyone have any experience of this or why this is happening? My domain is around 8 years old and has around 50,000 pages. Any pointers would be greatly appreciated.
Algorithm Updates | | MarkHincks0 -
Can a google data refresh knock your pages out of the rankings?
I see that around mid November 2013 a handful of my sites pages dropped off of Google completely. It was around the data refreshes in November, and while everyone says it doesn't effect that much I was wondering if anyone knew if it could knock some of my pages out of the rankings for a specific keyword. Note - we had previously held muliple listings for different pages on our site for this particular keyword. Google kept the highest ranking and knocked the lower ones off. See attached image of our keyword ranking history to see what I mean. DcJJM0M
Algorithm Updates | | franchisesolutions0 -
New linkbuilding: If networks are useless, and I need high volume through a 1-man team, what's the best option?
I work for an online retailer, and we have thousands of product pages and our vertical for content is brutal -- half of them are owned by our competitors. Are there any new linkbuilding strategies that can be done through a 1-man team? I'm not talking about bots or traditional link networks. Our current strat revolves around the following: 1. Link prospecting through buzzstream tools and singular contacts 2. Finding bloggers/vloggers, sending product and having them send backlinks to our homepage level with their reviews (slow turnaround, low juice). 3. Syndicating our videos through multiple avenues. 4. Being active on social. We need to gain more authority outside of simple content building. Are there any alternatives to link networks to optimize build outs via a 1-man team? Many thanks!
Algorithm Updates | | eugeneku0 -
Gifts.com - Multiple domain pages in SERPs
One of our big natural search competitors for gift keywords is Gifts.com. We are competing for many keywords like "teen gifts", "gifts for him", "gifts for her". For many of these, the Google SERP has multiple Gifts.com pages on the first page. I have never seen more than one of our pages (uncommongoods.com) on a SERP page. Any clue how/why Gifts.com has multiple pages in search results ? Thanks!
Algorithm Updates | | znotes0 -
Why would my product pages no longer be indexed in Google?
Our UK site has 72 pages in our sitemap. 30 of them are product pages which take a productid parameter. Prior to 1st Feb 2011, all pages were indexed in Google but since then all of our product pages seem to have dropped from the index? If I check in webmaster tools, I can see that we have submitted 72 pages and 42 are indexed. I realise we should have some better url structuring and I'm working on that but do you have any ideas on how we can get our product poages back into googles index http://www.ebacdirect.com
Algorithm Updates | | ebacltd0