Google is not indexing all URLs
-
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com
Any help would be appreciated.
-
Hi,
It sounds like something may be getting blocked or not crawled correctly, possibly because of the move from http --> https. I assume this is handled by the .htaccess file?
I would also check to see if there are https page issues on the site. If it's bit, you may have to do some digging around, but you can see this by using Google Chrome --> right click on the page --> inspect element --> console. This will show you the errors on the page.
-Andy
-
Hi Mossa,
Thanks for advice. We have already 2 millions of pages which are crawled. and good number of page views and session/day.
I will go with your point
-
You can see, almost all company and event profile have internal linking for other profile and events
-
We are doing that but we have pages in billion which is literally time taking as it have countries and events from 200 countries
-
It is very difficult to get backlining for billions of URLs, so branding and other activities are there to take care of that.
-
http to https have 301 redirection and all sitemap have been updated.
-
-
Before I start, why your website is using “_blank” even on internal links? This does not make any sense to me and I was literally pissed after three clicks.
Anyways, if you are interested in increasing the indexed pages here are the few things you could do:
- Improve internal linking of the website. Especially when possible link pages that are not indexed to the pages that are indexed.
- Get some social media shares (tweets) to pages that are not indexed.
- Build some quality links to the pages that are not indexed in Google.
Also, make sure that the 301 redirections from http tp https versions are correct and working and xml sitemap is updated.
If you do that I am sure that Google will not only index your pages but will also rank better from the desired key phrases.
Hope this helps!
-
Hi Martijn,
We have various sitemap
www.businessvibes.com/sitemap.companyprofile5.xml
www.businessvibes.com/sitemap.companyprofile6.xml
www.businessvibes.com/sitemap.companyprofile7.xml
And many More
-
Hi,
Do you have sitemaps that could provide some insights into your indexation rate for the data that you've submitted?
M.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google My Business app search report
What is the deal with the search reports in the Google My Business app? I downloaded this app so prospective customers could message my business, and when I look at the search reports on the app, the results seem nonsensical. According to google analytics my business receives pretty steady traffic every day. Why does the report say that I receive zero visitors one day and 400 the next? (See the screenshot below) 4yLRCHG
Reporting & Analytics | | RandyHT1 -
Weird URL Structure in GA
Hey everyone, Thanks in advance for any insight on this. I've been researching it quite a bit on Google and haven't found anything yet. In Analytics, under our pages report, we're getting a lot of pages that look like this: www.execucar.com/https://www.execucar.com or www.execucar.com/https://www.execucar.com/locations/orlando-car-service Any thoughts on how to fix this? These pages don't exist...I'm at such a loss.
Reporting & Analytics | | SuperShuttle0 -
"index.htm" for all url's in google analytics
I don't have this issue with other wordpress websites, only this one website, and I don't know what's causing the issue: Google Analytics is adding an "index.htm" to every single page on the website. So it is tracking the pages, I see no errors - is it tracking the right page? When I click on the page link in a report, I naturally go to a "404 page not found" since the website address isn't "www.example.com/rewards/index.htm" - but instead the actual address would be:
Reporting & Analytics | | cceebar
"www.example.com/rewards/". I have navigated to View Settings in GA to insure "default page" is empty. Although adding anything else to this field does not effect the page url in analytics reports either. Could it be htaccess file - or a plugin effecting the htaccess file?_Cindy0 -
Google Analytics Custom Alert For Goal Completions?
Hi currently setting up custom alerts for a website. I want to do a custom alerts for when goal completions decrease or increase by a certain % e.g. 20% as shown in the third example from this Moz Article back in 2012 - http://moz.com/blog/7-essential-google-intelligence-custom-alerts-that-keep-me-sane However when i go into my GA account the only options seem to be for when the goal completion conversion rate % drops. I could use this, but i rather do just based on completions rather than %. Any ideas on how i can do this? Thanks.
Reporting & Analytics | | MBASydney0 -
Exact Match in Google Search (Not Adwords)
I was going throught the list of keywords that have sent traffic to my site over the last 7 years and cam across one "A516 grade 70" that had hundreds of variants. Now in a lot of cases search volumes were different as were SERPS. We've tested a few variants with reworked pages (70% similar to original but optimised for variant keyword) and see good SERPS and traffic results. Theres obviously some diminishing returns here for us but the interesting question is when to these variants become an exact match and when not? In some cases the variants are unique because of the spacig, periods and hyphens used. there isn't a clear correlation with exact matc though. Insight appreciated. (Sorry for spelling errors. Form doesn't play nicely with iPad)
Reporting & Analytics | | Zippy-Bungle0 -
Cookie tracking in Google Analytics
Hi How do I remove the "/?__utma=...." at the end of my URLs?We have a site http://www.jetonline.co.za/, if you click on one of the menu navigation links i.e. "fashion". A long "/?__utma=...." url appears. I understand this is for tracking as we have separate domains for each page but is there a way to remove this dynamic url and keep it hidden from users?Thanks in advance
Reporting & Analytics | | NeilPursey0 -
Google Analytics session update question
Hello, With reference to seomoz blog post - http://www.seomoz.org/blog/panda-24-and-analytics-session-update-rolled-out-simultaneously#jtc151292 , i would like clarification about the following - User searches Google for "Product Name" and clicks on your AdWords advertisement. User leaves site and searches a few more times, click on competition and comparing prices and features. User ultimately decides to with your product, Googles "Your Brand + Product Name", clicks your organic listing, and buys the product. This whole process takes less than 30 minutes. "Your Brand + Product Name" will appear in your organic keyword report with 1 visit. My question is whether "Product Name" will also appear in organic keyword report with 1 visit if the visitor is not signed in. ( as search won't be encrypted ) Thanks
Reporting & Analytics | | seoug_20100 -
Historical Indexation
Hello, Is there at tool to see how many pages were indexed in google for a particular website historically? Thanks
Reporting & Analytics | | soeren.hofmayer0