Certain Pages Not Being Indexed - Please Help
-
We are having trouble getting a bulk of our pages indexed in google. Any help would be greatly appreciated!
The Following Page types are being indexed through escaped fragment:
http://www.cbuy.tv/celebrity#!65-Ashley-Tisdale/fashion/4097-Casadei-BLADE-PUMP/Product/175199
<cite>www.cbuy.tv/celebrity/155-Sophia-Bush#!</cite>
However, all our pages that look like this, are not being indexed:
-
Hi Takeshi,
We have a sitemap but also the pages are all interlinked. I didn't know that google puts an upper-bound on indexing based on PR - that's interesting.
Since there is a black and white difference between a set of pages of a certain kind (zero of these pages are being indexed) I suspect there is some other issue. Is it at all possible that google does not like the urls of these pages? :
1. does google not like the parameters?
2. should we reduce the length of our guid id number and move it to the end of the url?
-
Where are these pages being linked from? If you want these pages indexed, you may want to try making them more prominent in your site's navigation and architecture. Listing them in a sitemap can help them get discovered by Google, but actually linking to them from your site will have much more impact.
Also, I notice that the site is only pagerank 2, and already has 5000+ pages indexed in Google. Google limits the number of pages it indexes for sites based on their pagerank, so you may want to consider improving your PR so Google indexes more pages from your site.
-
Hi Mike,
I am sure you've probably already barked up this tree, but do those pages contain 100% substantially unique content?
Also, have you had an SEO developer review your robots.txt and .htaccess files to make sure there isn't something it there preventing crawlers from having access?
Dana
-
Hello Dana,
Thanks for your reply.
We have thousands of #! pages being indexed. Googlebot is sent to our escaped fragment page through a redirect. Our dynamic sitemap helped us get many pages indexed. However there are a subset of pages that google does not like at all and we cannot figure out why. For example when you visit our homepage, http://www.cbuy.tv, then navigate through images in our carousel (each assigned a unique url) none of these pages are being indexed.
Mike
-
Hi Mike,
I am not a developer, but I think the problem is the hashtag in your URL. This is a problem for search engines in that, anything following the "#" is completely ignored by search engines.
Depending on your platform, I would consider re-writing all of your URLs to omit that hashtag completely. Search engines (and humans!) can respond in unpredictable ways to anything other than alpha-neumeric characters. Then I would implement 301 redirects if necessary (depending on how old the site is and how many inbound links there are to each page).
I don't think that sitemap submission is even going to help right now because of the hashtag issue, but I'd love to hear from a developer on this for verification.
I hope this helps!
Dana
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Magento 1.9 SEO. I have product pages with identical On Page SEO score in the 90's. Some pull up Google page 1 some won't pull up at all. I am searching for the exact title on that page.
I have a website built on Magento 1.9. There are approximately 290,000 part numbers on the site. I am sampling Google SERP results. About 20% of the keywords show up on page 1 position 5 thru 10. 80% don't show up at all. When I do a MOZ page score I get high 80's to 90's. A page score of 89 on one part # may show up on page one, An identical page score on a different part # can't be found on Google. I am searching for the exact part # in the page title. Any thoughts on what may be going on? This seems to me like a Magento SEO issue.
Intermediate & Advanced SEO | | CTOPDS0 -
Do uncrawled but indexed pages affect seo?
It's a well known fact that too much thin content can hurt your SEO, but what about when you disallow google to crawl some places and it indexes some of them anyways (No title, no description, just the link) I am building a shopify store and it's imposible to change the robots.txt using shopify, and they disallow for example, the cart. Disallow: /cart But all my pages are linking there, so google has the uncrawled cart in it's index, along with many other uncrawled urls, can this hurt my SEO or trying to remove that from their index is just a waste of time? -I can't change anything from the robots.txt -I could try to nofollow those internal links What do you think?
Intermediate & Advanced SEO | | cuarto7150 -
301 redirects broken - problems - please help!
Hi, I have a bit of an issue... Around a year ago we launched a new company. This company was launched out of a trading style of another company owned by our parent group (the trading style no longer exists). We used a lot of the content from the old trading style website, carefully mapping page-to-page 301 redirects, using the change of address tool in webmaster tools and generally did a good job of it. The reason I know we did a good job is that although we lost some traffic in the month we rebranded, we didn't lose rankings. We have since gained traffic exponentially and have managed to increase our organic traffic by over 200% over the last year. All well and good. However, a mistake has recently occurred whereby the old trading style website domain was deleted from the server for a period of around 2-3 weeks. It has since been reinstated. Since then, although we haven't lost rankings for the keywords we track I can see in webmaster tools that a number of our pages have been deindexed (around 100+). It has been suggested that we put the old homepage back up, and include a link to the XML sitemap to get Google to recrawl the old URLs and reinstate our 301 redirects. I'm OK with this (up to a point - personally I don't think it's an elegant solution) however I always thought you didn't need a link to the xml sitemap from the website and that the crawlers should just find it? Our current plan is not to put the homepage up exactly as it was (I don't believe this would make good business sense given that the company no longer exists), but to make it live with an explanation that the website has moved to a different domain with a big old button pointing to the new site. I'm wondering if we also need a button to the xml sitemap or not? I know I can put a sitemap link in the robots file, but I wonder if that would be enough for Google to find it? Any insights would be greatly appreciated. Thank you, Amelia
Intermediate & Advanced SEO | | CommT0 -
New domain purchase 301 and 404 issues. Please help!
We recently purchased www.carwow.com and 301 redirected the site to www.carwow.co.uk (our main domain). The problem is that carwow.com had URLs indexed like www.carwow.com/a-b-c the 301 sends them to carwow.co.uk/a-b-c which obviously doesn't exist so is a 404! What should be done in this situation? Should it be ignored and not re-directed at all, or is there a way to delete/disavow these dead pages? An SEO has advised we redirect all pages to the homepage, but won't that mess up the link profile? Any advice would be great!
Intermediate & Advanced SEO | | JamesPursey0 -
Ranking slipped to page 6 from page 1 over the weekend?
My site has been on page one for 2 phrases consistently from May onwards this year. The site has fewer than 100 backlinks and the link profile looks fairly even. On Friday we were on page 1, we even had a position 1, however now we are on page 6. Do you think this is Penguin or some strange Google blip? We have no webmaster tools messages at all. Thanks for any help!
Intermediate & Advanced SEO | | onlinechester0 -
Do you add 404 page into robot file or just add no index tag?
Hi, got different opinion on this so i wanted to double check with your comment is. We've got /404.html page and I was wondering if you would add this page to robot text so it wouldn't be indexed or would you just add no index tag? What would be the best approach? Thanks!
Intermediate & Advanced SEO | | Rubix0 -
How Long Does it Take for Rel Canonical to De-Index / Re-Index a Page?
Hi Mozzers, We have 2 e-commerce websites, Website A and Website B, sharing thousands of pages with duplicate product descriptions. Currently only the product pages on Website B are indexing, and we want Website A indexed instead. We added the rel canonical tag on each of Website B's product pages with a link towards the matching product on Page A. How long until Website B gets de-indexed and Website A gets indexed instead? Did we add the rel canonical tag correctly? Thanks!
Intermediate & Advanced SEO | | Travis-W0 -
Can someone please help me understand my sites recent loss of rankings?
My site has been top 3 for 'speed dating' on Google.co.uk since about 2003 and it went to below top 50 for a lot of it's main keywords shortly after 27 Oct 2012. I did a re-submission request and was told there was 'no manual spam action'. I have a Page Authority of 53, a regular blog http://bit.ly/oKyi88, a KLOUT of 40, user reviews and quality content. I did discover that another URL I using was set to a 302 instead of a 301 for some reason. I don't necessarily think this was an issue as Google should know which is the trusted URL and therefore which content to list. I removed this redirect completely about 3 weeks ago, but I've seen no improvement. I'm looking at improving various things, but I'm still not sure why I've been hit and wonder if I'm missing something obvious? Any suggestions greatly appreciated.
Intermediate & Advanced SEO | | benners0