Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Bing Won't Index Site - Help!
-
For the past few weeks I’ve been trying to figure out why my client's site is not indexed on bing and yahoo search engines.
My Google analytics is telling me I’m getting traffic (very little traffic) from Bing almost daily but Bing webmaster tools is telling me I’ve received no traffic and no pages have been indexed into Bing since the beginning of December.
At once point I was showing ranking in Bing for only one keyword then all of a sudden none of my pages were being indexed and I now rank for nothing for that website. From Google I’m getting over 1200 visits per month. I have been doing everything I can to possibly find the culprit behind this issue.
I feel like the issue could be a redirect problem. In webmaster tools on Bing I’ve used “Fetch as Bingbot” and every time I use it I get a Status of “Redirection limit reached.”. I also checked the CRAWL Information and it’s saying all the URL’s to the site are under 301 redirect. A month or so ago the site was completely revamped and the canonical URL was changed from non www to www. I have tried manually adding pages to be indexed multiple times and Bing will not index any of the sites pages. I have submitted the sitemap to Bing and I am now at a loss. I don’t know what’s going on and why I can’t get the site listed on Bing.
Any suggestions would be greatly appreciated.
Thanks,
Stephen -
You have been extremely helpful. Thank you for all your help. I will get right on this to get resolved.
-
Okay, so the crawler is seeing it for the www. version but not for the non www version.
If we look in a browser at:
http://dunninggolf.com it redireects to www.dunninggolf.com
If I look in webbug at the headers it does not redirect and returns a 200 Okay status code for the non www version of the URL.
I think this could be the issue here and that for some reason, when bing visits the site is giving out some strange 301 response which then sends out a loop of 302 redirects.
Okay, there is a tool here that lets you visit the site as if you were bing bot and get the HTTP headers:
http://view-http-headers.gishan.net/fetch-as-bing-bot-tool.jsp
You have to select bingbot rather than IE6 from list of user agents.
If you visit the www.dunninggolf.com version then it returns a 200 OK
If you visit the http://dunninggolf.com version then it returns a 301 redirect (which in turn seemingly issues the loop of 302 temp redirects that never go anywhere).
Okay, there is your issue. The site has some issue with bingbot (and possibly others) when visited via the http://dunninggolf.com URL. This issue is not visible in webbug or a browser. That said, we do seem to get some redirection from http://dunninggolf.com to http://www.dunninggolf.com.
So, taking a stab in the dark, I would say there is some kind of meta refresh or something going on after the page has been returned as a HTTP 200 result. I am not on my windows PC so can't examine the headers more closely but this is certainly your issue here.
You need to find out the following.
1. How are they doing the redirect?
2. why are they redirecting this way?
3. why are they serving up these 301 / 302 loops via http://dunninggolf.com
Ultimately, this is an issue for the developers. You should advise them on correct protocol (a 301 via the HTTP headers from http://dunninggolf.com to http://www.dunninggolf.com and not via this whacky redirect they currently have in place).
Wowzers, that was a tricky one!
Hope this helps and sorts out your headache!
Marcus
-
I have used SEO analyzer before and i did it again just now and it gave me some weird error.
This is the result:
<label class="error" for="seoUrl">The URL has too many redirects</label>
| ! Redirects detected for the current URL |
I also have submitted the URL in the "submit URL toolbox page" a couple times.
I will get working on the robots.txt and make sure it's put in place.
I don't do any of the development work on the site i just perform the SEO duties. Seeing those redirect errors is part of what im talking about with it saying there are to many redirects. Any idea what that's all about if its showing everything should be working properly?
I just scanned it again and based on the URL i enter i get different results.
those redirect errors come when i analyze dunninggolf.com. But when i analyzed www.dunninggolf.com I only 2 errors on homepage for missing alt tags on images and no H1 tag. -
Hmm, five months is a bit long.
Lets look at what we do know.
1. Bing is telling you that the page it is trying to fetch is redirecting and the bingbot does not follow redirects (1).
2. If I fetch your page in a browser or a even just the headers in webbug I get a HTTP 200 OK status code and no 301 redirect.
So, the site itself seems fine. The pages are serving up fine and they are returning the correct HTTP status codes.
So.. have you tried analysing the page in the SEO Analyser? http://www.bing.com/webmaster/help/seo-analyzer-97615e21
If so, what does that tell you? It should detail the HTTP response codes that bing is seeing.
One possiblity is that there is something whacky going on for the bing user agent but fetching the page as bingbot also returns a 200 status code.
There is no robots.txt so I would put that straight first up as suggested and at the very least go with:
User-Agent: *
Allow: /Without access tp webmaster tools and digging further it is hard to see what is going on but 5 months seems too long. That said, there are no issues preventing crawling or indexation in so far as a 15 minute look can identify.
Daft question but have you submitted the URL via:
http://www.bing.com/toolbox/submit-site-url
I have just submitted it just in case you have only done via webmaster submit.
Anyhow, it's not secret that bing can be horribly slow and it seems that it still believes you have 301 in place for some reason and I could not find one.
Maybe one for Bing Webmaster Support as it seems there is no good reason why they would not index this site and the tools at your disposal are not working as they should: http://www.bing.com/webmaster/help/webmaster-support-24ab5ebf
References
1. http://www.bing.com/webmaster/help/fetch-as-bingbot-fe18fa0d
-
I have been working on getting some new links and have gotten a couple but nothing substantial as of yet.
This client is new for me and i am only aware of this site being around since June or July of last year.
I've been waiting for bing to show search results for the site for almost 5 months now and nothing has changed. I am getting to the point where i feel the culprit lies somewhere else besides saying bing indexing slowly as being an excuse.
-
It looks like the site was down for a long period of time and was redirecting to another site: http://web.archive.org/web/20090615041707/http://dunninggolf.com/
It also then looks like there was a holding page up for a while.
I can't see the complete history on archive.org and nothing since Feb 2011 but it looks like these historical redirects are your problem.
Few questions
How long has the new site been up?
Have you won any new links?
It seems that Bing still believes this site is redirecting somewhere else so you may just have to wait a while till they update their index. You could ask them to remove all of the links that they do currently hold.
If Bing webmaster tools is anyting like Google Webmaster tools the information there is horribly out of date so it may just be a matter of waiting this out till they recrawl the site and refresh everything.
A good use of your time may be just getting your on page ducks in a row and making sure everything else is okay.
- The canonical looks fine.
- There are no 301's on the pages I visited
- You have no robots.txt file
- Google is indexing it okay
Ultimately, I think this is a case of waiting it out. If you want to speed things up, try to get some fresh links to the site from well ranking sites to help drive the crawlers your way.
Hope that helps!
Marcus -
Yeah, the URL is http://www.dunninggolf.com/
-
I did check a small handful of pages and i did see rel="canonical" in the page source code. I only saw it on product pages and category pages etc. blog and other pages not advertising product i didn't see it.
But like i said i only checked about 8 pages and 6 out of the 8 had that tag on it.
-
Hey Stephen
Can you post a URL so we can take a look? Anything else would be a complete shot in the dark.
Marcus
-
Double check that the recent rel=can update for the main URL / homepage (your non-www to www) has not been accidentally applied to ALL the pages. That single mistake can hoark a site's indexing 100%.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Speed or Site Speed which one does Google considered a ranking signal
I've read many threads online which proves that website speed is a ranking factor. There's a friend whose website scores 44 (slow metric score) on Google Pagespeed Insights. Despite that his website is slow, he outranks me on Google search results. It confuses me that I optimized my website for speed, but my competitor's slow site outperforms me. On Six9ja.com, I did amazing work by getting my target score which is 100 (fast metric score) on Google Pagespeed Insights. Coming to my Google search console tool, they have shown that some of my pages have average scores, while some have slow scores. Google search console tool proves me wrong that none of my pages are fast. Then where did the fast metrics went? Could it be because I added three Adsense Javascript code to all my blog posts? If so, that means that Adsense code is slowing website speed performance despite having an async tag. I tested my blog post speed and I understand that my page speed reduced by 48 due to the 3 Adsense javascript codes added to it. I got 62 (Average metric score). Now, my site speed is=100, then my page speed=62 Does this mean that Google considers page speed rather than site speed as a ranking factor? Screenshots: https://imgur.com/a/YSxSwOG **Regarding: **https://six9ja.com/
Reporting & Analytics | | Kingsmart1 -
Losing referrer data on http link that redirects to an https site when on an https site. Is this typical or is something else going on here?
I am trying to resolve a referral data issue. Our client noticed that their referrals from one of their sites to another had dropped to almost nothing from being their top referrer. The referring site SiteA which is an HTTPs site, held a link to SiteB, which is also an HTTPs site, so there should be no loss, however the link to SiteB on SiteA had the HTTP protocol. When we changed the link to the HTTPs protocol, the referrals started flowing in. Is this typical? If the 301 redirect is properly in place for SiteB, why would we lose the referral data?
Reporting & Analytics | | Velir0 -
Redirecting all URLs appended with index.htm or index.html
It has come to my attention with one of my clients (WordPress website) that for some time they have within their Landing Page report (of GA - Google Analytics) URLs that should all be pointing to the one page, example: domain.com/about-us, also has a listing in GA as domain.com/about-us/index.htm Is this some kind of indication of a subdirectory issue? Has anyone had experience with this in such wordpress plugins as Yoast SEO, or other SEO plugin? My thoughts here are to simply redirect any of these non-existent files with a redirect in .htaccess - but what I'm using isn't working. I will insert the redirect here - - and any help would be greatly appreciated. RewriteEngine onRewriteCond %{THE_REQUEST} ^./index.html?
Reporting & Analytics | | cceebar
RewriteRule ^(.)index.html?$ http://www.dupontservicecenter.com/$1 [R=301,L] and this rewrite doesn't work: RewriteEngine on
RewriteRule ^(.+).htm$ http://dupontservicecenter.com/$1.php [R,NC] _Cindy0 -
Migrated website but Google Analytics still displays old URL's and none new?!
I migrated a website from a .aspx to a .php and hence had to 301 all the old urls to the new php ones. It's been months after and I'm not seeing any of the php pages showing results but I'm still getting results from the old .aspx pages. Has any one had any experience with this issue or knows what to do? Many thanks,
Reporting & Analytics | | CoGri0 -
Can you track two Google Analytics Accounts on one site?
If you have a site that had an old analytics account and then implemented a new one is it possible to run tracking code that records to both accounts without causing your site or data issues? We are doing this so we don't loose data at any point - ideally it wouldn't have been split between the two but making one redundant isn't an option. Ideally we would have merged the data from both accounts and had one - however the research we have done points to this not being a possibility - unless one of you guys knows different? It would be great if anyone has experience on any this.. Thanks
Reporting & Analytics | | ChrisAllbones0 -
Sudden Increase In Number of Pages Indexed By Google Webmaster When No New Pages Added
Greetings MOZ Community: On June 14th Google Webmaster tools indicated an increase in the number of indexed pages, going from 676 to 851 pages. New pages had been added to the domain in the previous month. The number of pages blocked by robots increased at that time from 332 (June 1st) to 551 June 22nd), yet the number of indexed pages still increased to 851. The following changes occurred between June 5th and June 15th: -A new redesigned version of the site was launched on June 4th, with some links to social media and blog removed on some pages, but with no new URLs added. The design platform was and is Wordpress. -Google GTM code was added to the site. -An exception was made by our hosting company to ModSecurity on our server (for i-frames) to allow GTM to function. In the last ten days my web traffic has decline about 15%, however the quality of traffic has declined enormously and the number of new inquiries we get is off by around 65%. Click through rates have declined from about 2.55 pages to about 2 pages. Obviously this is not a good situation. My SEO provider, a reputable firm endorsed by MOZ, believes the extra 175 pages indexed by Google, pages that do not offer much content, may be causing the ranking decline. My developer is examining the issue. They think there may be some tie in with the installation of GTM. They are noticing an additional issue, the sites Contact Us form will not work if the GTM script is enabled. They find it curious that both issues occurred around the same time. Our domain is www.nyc-officespace-leader. Does anyone have any idea why these extra pages are appearing and how they can be removed? Anyone have experience with GTM causing issues with this? Thanks everyone!!!
Reporting & Analytics | | Kingalan1
Alan1 -
Switch to www from non www preference negatively hit # pages indexed
I have a client whose site did not use the www preference but rather the non www form of the url. We were having trouble seeing some high quality inlinks and I wondered if the redirect to the non www site from the links was making it hard for us to track. After some reading, it seemed we should be using the www version for better SEO anyway so I made a change on Monday but had a major hit to the number of pages being indexed by Thursday. Freaking me out mildly. What are people's thoughts? I think I should roll back the www change asap - or am I jumping the gun?
Reporting & Analytics | | BrigitteMN0 -
How do you set www vs non-www in Bing Webmaster Tools?
A site of mine is showing as example.com instead of www.example.com in Bing and Yahoo. I know where to set the preferred domain to www in Google Webmaster Tools, but I can't seem to find that function in Bing Webmaster Tools. Can someone please point me in the right direction?
Reporting & Analytics | | DentalPlanSEO0