My wepgages aren't crawled by google
-
Most of my webpages aren't crawled by google.
Why is that and what can i do to make google index at least most of my webpages? -
Well, Google does have a crawl budget, they might be using that for your most popular pages. As long as your indexed pages number is going up, that means google is working its way through the backlog.
-
My website is a yellow pages site from Greece www.vreite.gr
It has registered more than 175.000 businesses and every business has 6 profile pages(main page,product page,feed page etc.)
Many visitors engage with these pages and are absolutely dunamic pages.
Is that a problem? -
The only site I can think of that would legitimately have 400,000 pages is Amazon.com. Google probably thinks your site is full of a ton of low quality content. Why in the world do you have that many pages? Are they low quality garbage? Do any visitors actually engage this even a fraction of them?
-
Hi
Yes my site is crawlable.
I checked if robots.txt or noindex tags and canonical urls and everything is fine.
Maybe is it because my website has over 400.000 pages -
Hi
You didn't answer the first part of Zoe's question - are you sure that your site is crawlable and that there are no issues with the robots.txt / noindex tags, ip detection systems, canonicals on all pages pointing to the home and so on. It's not because you can see all pages of your site in a browser that they are accessible/crawlable/indexable by google.
Try a crawl with Screaming Frog and user agent Googlebot to see if your pages can be crawled and indexed.
Backlinks are needed to have your site ranked for keywords - but it's not a prerequisite to have your site crawled. (noticed that a few times when a dev site was indexed by accident)
Without the actual url it's impossible to give a more detailed answer.
Dirk
-
Hi,
Backlinks certainly help, if there's no links at all to your site that could be a reason, but it's hard to say without looking deeper.
Are your internal pages all linked to each other? Does your website have a structured navigation system? This is also really necessary to ensure Google will index your whole site, not just a couple of pages.
Zoe
-
Hi
I added my website to Google Webmaster Tools and i checked my website and i don't have any crawling issues.
I added my website to Dmoz but the backling didn't appear yet.My site is live for about a year and google doesn't crawl most of my webpages yet.
Is it because i don't have quality backlinks?Thank you
-
Hi,
Firstly I'd check that Google can index your website. Have you added your site to Google Webmaster Tools? I'd start there and check for any crawl issues, especially your robots.txt file and any no-indexing of pages.
Secondly, if your website is brand new, I'd add your website to Dmoz & some relevant good quality sites like Yelp, Yell, Yellowpages, Google Plus (where relevant). Make sure the details you add to each match exactly with the details on your website. It will take some time for your site to appear in Google's index- sometimes a few days, sometimes a week or so- you can check by typing site:yourdomainname.com into a Google search to find the pages.
If your website is not new & has been indexed by Google before, I'd investigate whether you have a penalty. This post on penalties from white.net is really useful!
Hope this helps,
Zoe
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Organic search traffic has dropped by 35% since 18 September, we don't know why.
Organic traffic to our website has dropped 35% since 18 September 2017 to date. From 1 January to 18 September 2017 organic traffic was up by just under 1% over all (Google up by 1.32%). Paid search traffic over the same time has remained steady. There is nothing we can think of that we've done that has caused the drop. We had an issue with Google page speed test failing when running a test but we resolved this issue on 20 November and in that time we've seen an even greater drop (44% in the last week). The drop is seen across the 3 main search engines, not just Google, which points toward something we've done, but as mentioned, we can't think of any significant change we made in September that would have such negative effects. There is little difference across devices. Is anyone aware of a significant event in September in the search engine world that may have influenced our organic traffic? Any help gratefully received.
Technical SEO | | imaterus0 -
Looks like keyword stuffing, but it isn't
(There was a similar older question on the forum, but it wasn't really answered so please forgive me if this looks like a repeated question) Looks like keyword stuffing, but it isn't We have a massive web store with 80k "commodity products" (and this amount will only increase) which aren't sold the same way normal products are sold (i.e. by brand and model). Commodity products are sold by specification, therefore their product names are actually descriptions of the product. In our case, industrial fasteners (nuts, bolts, washers, screws, etc) sold in bulk. If you click on the link below, you will see that our catalog involves a tremendous amount of repetition, where the products all appear the same, but are varying by dimensions and/or package quantities. The solutions the web store software offers to solve this problem cause issues for us (i.e. displaying the dimensions and quantities ONLY under a common header) but more importantly, we are concerned that search engines are seeing this as keyword stuffing and penalizing the pages. http://www.aspenfasteners.com/Step-Bolts-Inch-Standard-s/407.htm If we can't change the presentation of the page, should we be concerned and if so, how do we let a search engine know that the repetition is legitimate?
Technical SEO | | AspenFasteners0 -
I can't crawl the archive of this website with Screaming Frog
Hi I'm trying to crawl this website (http://zeri.info/) with Screaming Frog but because of some technical issue with their site (i can't find what is causing it) i'm able to crawl only the first page of each category (ex. http://zeri.info/sport/) and then it will go to crawl each page of their archive (hundreds of thousands of pages) but it won't crawl the links inside these pages. Thanks a lot!
Technical SEO | | gjergjshala0 -
Hreflang Tags - error: 'en' - no return tags
Hello, We have recently implemented Hreflang tags to improve the findability of our content in each specific language. However, Webmaster tool is giving us this error... Does anyone know what it means and how to solve it? Here I attach a screenshot: http://screencast.com/t/a4AsqLNtF6J Thanks for your help!
Technical SEO | | Kilgray0 -
Lots of Pages Dropped Out of Google's Index?
Until yesterday, my website had about 1200 pages indexed in Google. I did lots of changes: removed low quality content, rewrote passable content to make it better, wrote high quality content, got lots of likes and shares on social networks, etc. Now this morning I see that out of 1252 pages submitted, only 691 are indexed. Is that a temporary situation related to the recent updates? Anyone seeing this? What should I interpret about this?
Technical SEO | | sbrault740 -
Why are Google search results different if you are log'd into Google or not?
I get different results when I'm log'd into my Google account associated with my website than if I'm not. The same country is occurring. So how can I rely on the google results I'm seeing? For instance my site is page 1 with the improvements I made based on SEOMOZ if I'm log'd in. Yet I'm not on the first 25 pages if I'm not logged in.
Technical SEO | | Romana0 -
Google Reconsideration Request (Penguin) - Will Google give links to remove?
When Penguin v1 hit, our site took a hit for a single phrase (i.e. "widgets") due to the techniques our SEO company was using (network). We've since had those links cleaned up, and our rankings have not recovered. Our SEO company said they submitted a reconsideration request on our behalf, and that Google denied it and didn't provide which links we needed removed. Does Google list links that need removing if they are still not happy with your link profile?
Technical SEO | | crucialx0 -
Why won't the Moz plug in "Analyze Page" tool read data on a Big Commerce site?
We love our new Big Commerce site, just curious as to what the hang up is.
Technical SEO | | spalmer0