Getting pages indexed by Google
-
Hi SEOMoz,
I relaunched a site back in February of this year (www.uniquip.com) with about 1 million URL's.
Right now I'm seeing that Google is not going past 110k indexed URL's (based on sitemaps).
Do you have any tips on what I can do to make the site more likeable by Google and get more indexed URL's?
All the the part pages can be browsed to by going to: http://www.uniquip.com/product-line-card/suppliers/sw-a/p-1
I've tried to make the content as unique as possible by adding random testimonials and random "related part numbers" see here: http://www.uniquip.com/id/246172/electronic-components/infineon/microcontrollers-mcu/sabc161pilfca
Do I need to wait more time and be more patient with Google? It just seems like I'm only getting a few thousand URL's per day at the most.
Would it help me if I implemented a breadcrumb on all part pages?
Thanks,
-Carlos
-
Carlos
Actually, it is in the index but if you do: site:www.uniquip.com as a search in Google it is not the first result, which it should be. If you do "site:www.uniquip.com" (with the quotes) as a search in Google, it is still only the second result, it should be the first, which is a sign that something is weird, which would require some more investigation.
-Dan
-
Thanks for both tips Dan, I will look into them althought number 2 may be a bit tough.... since there are hundreds of suppliers/categories (bit tough to put in a menu).
as for the page not being indexed I'm showing that it is. http://webcache.googleusercontent.com/search?sourceid=navclient-ff&ie=UTF-8&q=cache%3Ahttp%3A%2F%2Fwww.uniquip.com%2F
is this not the case for you?
-
Thanks for the feedback Brent.
I already my sitemaps setup in groups of 30k with a master sitemap file. http://www.uniquip.com/sm/smindex.xml
I will talk to the developer so we can look into regrouping them and perhaps pickup an indexing pattern.
-
Hi Carlos
It appears there are two major issues with your site not getting indexed.
1. Off-Site - Go to opensiteexplorer and type in your URL. You will see there are many suspicious looking links pointing back to your domain. No matter what you do on-site, if you have really low quality backlinks pointing to your domain (and no good ones to balance things out), nothing you do on-site will help. You'll want to do whatever you can to clean these up (get them removed). You'll also want to work on getting some GOOD backlinks in conjunction with cleaning out the bad ones. There are many resources here on SEOmoz on backlinking strategies.
2. On-site - Your product pages are 4 or more clicks away from the homepage. You really want this to be 3 or less (ideally 1 if possible for some products and categories).
I also noticed that your homepage does not appear to be indexed in Google.
We could go WAY more in depth with this, but in general those seem to be the two major issues.
-Dan
-
Carlos,
There is a great thread you should read about this topic: http://www.seomoz.org/q/can-you-push-too-many-urls-via-sitemaps
As for me, I would recommend creating multiple XML sitemaps where you can track which urls are bing indexed and to also control the submission quantity. You don't want to publish a ton of urls at one time. For the urls that aren't in Google's index, I would no index them for now and slowly submit a small group on a monthly basis, along with different XML sitemaps for tracking.
You have to think as if you were Google, is the content you're submitting valuable and new information? There is no way that 1 million new pages is so valuable or new that isn't already out there. A nice steady submission stream will give you the best results in the long run.
I hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Google take to reduce the index size?
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
On-Page Optimization | | cplastiras840 -
Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep. I just wanted to know the possible reasons and has anyone encountered this before?
On-Page Optimization | | MoosaHemani0 -
Google Indexing Wrong Title
Hey guys ! I have a wordpress website and also yoast seo plugin . I've set up a meta title which is : TV Online | Assistir Filmes| Notícias | Futebol |GogsTV . (I checked on some free tools to see , and they also show up this) but .... google is showing this : GogsTV: TV Online | Assistir Filmes| Notícias | Futebol . Seems they are trying to show my brand name first instead of my main keyword . I'm not sure why it doesnt indexes as i want ... Does anybody know how can i fix this . Thanks
On-Page Optimization | | tiagosimk0 -
Will Google Custom Search results on my home page kill it's ranking?
This is probably a dumb question, but here goes anyway. 🙂 On a site I have it would be very useful to the reader to offer a search box that uses a Google Custom Search that I have optimized to search websites that are closely on-topic with my site. I know it sounds bad that I would send people to other sites, but just assume that the reasons are valid for this discussion. My question is, if the search results are set to display on the same page (the home page) as the search box, will the links in the search results to external sites just bleed my page rank to death? I assume it would, but thought I'd check just in case I'm missing something. I have to option to place the results on separate page of my site, and noindex it, but it won't be as powerful as it would be on the home page.
On-Page Optimization | | bizzer0 -
Is it ok to point internal links to index.html home page rather than full www
I thought I saw this somewhere on SEOmoz before but I was so busy by the time I got around to work on my SEO on my site, I realized I have this happening and can't recall if it is a problem which takes away from my ranking. If my www.website.com is ranking well but I have internal menu links pointing to www.website.com/index.html instead of www.website.com will that take away from my www.website.com rankings? Should I change all my menu links that point to /index.html to the full website url path www.website.com ?
On-Page Optimization | | Twinbytes0 -
Remove internal site SERPS from Google Index?
1. Internal Serp pages did not have a robots meta tag 2. As a result, client site has thousands (~4,400) of internal site SERP pages in the Google index. 3. We added the NoIndex, Follow attribute to all internal SERPS 4. We Disallowed: domain.com/internal-search-operator in Robots.txt 5. No new SERP pages are being indexed, but the other 4000 something that were already there are still in the index weeks later. 6. The pages are dynamically created and still work, so I can't use the Remove Content tool from google, because the pages don't 404. Is there any way to get these pages out of the index besides just waiting and hoping google eventuall drops them? Thanks
On-Page Optimization | | delegator.com0 -
Does Too Many On-Page Links on a Page Really Matters?
Does Too Many On-Page Links on a Page Really Matters? Especially if they are pointing to internal page?
On-Page Optimization | | AppleCapitalGroup1 -
Creating New Pages Versus Improving Existing Pages
What are some things to consider or things to evaluate when deciding whether you should focus resources on creating new pages (to cover more related topics) versus improving existing pages (adding more useful information, etc.)?
On-Page Optimization | | SparkplugDigital0