Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Bing vs Google Keyword Research
Hi All, I have carried out keyword research based on Google Analytics, Keyword Planner, competitors and various other toolsets.
Reporting & Analytics | | Mark_Ch
My research has been based on performance metrics favored towards Google.
I'm looking to research keyword traffic metrics and suggestions based on Bing Ads Intelligence. My Logic
I have compiled my master list based on Google metrics.
I will carry out Bing search traffic metrics based on my Google master list.
I will carry out Bing keyword suggestions based on my Google master list. My Questions
#1 Will my logic yield me any favorable benefits based on the high number of Google users vs Bing users
#2 Should Bing favor a particular keyword with high search traffic compared to Google, is there a fear that creating a 'seo user friendly website page' could dilute link juice based on the high Google users. Any other useful advice would be welcome. Regards Mark0 -
Google Tag Manager breaking integration
Using Live Chat's (www.livechatinc.com) Google Analytics integration was populating events and virtual pageviews into my analytics account. I've since added Tag Manager and moved my analytics tracking code into there, but since doing so, the integration no longer seems to work as there is no population of either events or pageviews anymore. Anyone else had any experience of something similar? Any other suggestions (beyond not using GTM for analytics code anymore)? I was considering setting up the event tracking code manually in GTM, but not really sure how to do so seeing as I'm not sure what to fire the different events on. This is the live chat JS code:
Reporting & Analytics | | AdrianCordiner0 -
Google Analytics (not provided)
Hi A greater and greater % of clicks is showing as "not provided" - does any one know of any good external tracking software which will show this information? Thanks
Reporting & Analytics | | jj34340 -
Setting up Google Analytics for Subsites
I currently have one main .com site and am planning on launching geo-location subsites .co.uk, .com.au, .ru, etc... Traffic will flow between both sites and some of the content on the subsites will be duplicate and therefore include a canonical tag to the main site. I want to set up GA to capture who is going to the subsites and vice versa and correctly capture crossover traffic. Any advice on implementing advanced analytics directly (or links to sources that will direct me the right direction for this project)
Reporting & Analytics | | theLotter0 -
Localhost:4444 Showing Up in Google Analytics
Hello All, Lately in my Google Analytics account I have noticed a referral source labelled: localhost:4444 The number of visits is really high from this source, but I have no idea (no clue!) what it actually means. Can anyone shed some light on what this is about? Should I be creating some sort of filter to screen out this as a referral source (assuming it is not legitimate)? Many thanks in advance. Cheers!
Reporting & Analytics | | Robert-B0 -
How to Track Google Local Places in Google Analytics?
I have read many articles on how to track google local places through google analytics. Each article I have read show a different way of setting up google analytics and using tags in google local places. Wondering if anyone as up to date information on this and what would be the best practice to track data from google local lisitngs in google analytics Thanks Arthur
Reporting & Analytics | | VivaArturo0 -
A lot of traffic to one page from Google referral
We recently received a lot of traffic to one page from
Reporting & Analytics | | underthesun808
google.com referral. When I look in analytics it reports that the traffic is
coming from /url that’s not real helpful. Is there a way to get more specific
information as to what the referring url was?0 -
Links and index missing from Google Webmaster Tools!
I've been using a wordpress blog and using google analytics and webmaster tools. I'd never done very much optimizing of my site, my strategy had always been to just keep writing relevant posts and let google do the rest. At some point I installed a SEO plugin to wordpress and did a tiny amount of tweaking, but I noticed that in my webmaster tools page, all my "links to this site" links are gone! My site map says 42 pages submitted, 42 pages indexed, but all the links that I've built over the past few years are missing, and I also notice that I can't find my main landing page in the index anymore. I know there are indexed sites out there that link to me, but they aren't being reflected anymore. Did I anger google by using SEO plugins within wordpress? I'm fairly certain I wasn't doing anything sketchy. Http://brodiegraphics.com RTOH6.png
Reporting & Analytics | | HELPeR0