Getting Recrawled by Google
-
I have been updating my site a lot and some of the updates are showing up in Google and some are not. Is there a best practice in getting your site fully recrawled by Google?
-
If your site does not get crawl completely, i will recommend
a) To create a site-map, so that search engines can understand the site hierarchy and can deep crawl the site
b) Look at your site structure, Make your site more organized so as to have better crawl. It should not happen that a user needs to visit - home page > category page > sub category page > article or below to read the article. At best, article snippet should come on home page for a day. If not, should be accessible from home page > category page > article
c) Submit the article posted to Social Media - Reddit, Pinterest, Twitter etc. It will help search engines discover your link faster
By sure - will improve the crawl mechanism to your site
-
PS
#1
http://www.distilled.net/blog/seo/beginners-guide-doing-a-site-audit-using-google-webmaster-tools/
http://www.google.com/submityourcontent/
https://support.google.com/webmasters/answer/1352276?hl=en
Same rules for Google https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages
-
Heres more
Kristina Kledzik from distilled wrights about this a lot
http://www.distilled.net/about/people/kristina-kledzik/
http://www.zemanta.com/blog/technical-seo-audit-for-publishers/
http://www.distilled.net/blog/seo/site-navigation-for-seo/
http://yoast.com/how-to-get-google-to-crawl-your-site-faster/
http://www.stateofsearch.com/a-guide-to-crawling-indexing-and-ranking-your-websit/
http://www.distilled.net/blog/seo/how-to-do-a-5-step-site-audit/
http://www.seerinteractive.com/blog/easy-ways-seo-can-kill-your-site
Use Screaming Frog SEO Spider Use this with http://www.seerinteractive.com/blog/screaming-frog-guide
http://www.portent.com/blog/seo/three-seo-deal-killers.htm
Other good to know
http://www.distilled.net/blog/seo/beginners-guide-doing-a-site-audit-using-google-webmaster-tools/
http://www.portent.com/blog/seo/5-seo-strategies-we-swear-arent-going-anywhere.htm
http://www.mnsearch.org/john-doherty-from-distilled-teaches-mnsearch-about-technical-seo/
Hope this help's,
Thomas
-
Hi,
Yes here is a link to some good info & tips
http://www.distilled.net/blog/seo/advanced-seo-troubleshooting-why-isnt-this-page-indexed/
Absolutely. Make sure you have a fast hosting company hosting your website.
Make sure there's no flash, Ajax, or JavaScript obscuring your site from Google bot
Make sure your robots.txt file is not no following or no indexing any thing that you want to be found by Google
Same goes for your tags
- noindex X-Robots-Tag in HTTP header (X-Robots-Tag: noindex)
- Incorrect mime type declared in HTTP header. For example, for an HTML document, if it was declared as Content-Type: text/javascript
You can use Google Webmaster tools and the selection fetches Google bot and submit all URLs to index if you want to make sure it is being indexed in Google knows that you want the site indexed.
I hope I've been out sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can't get Google to index our site although all seems very good
Hi there, I am having issues getting our new site, https://vintners.co indexed by Google although it seems all technical and content requirements are well in place for it. In the past, I had way poorer websites running with very bad setups and performance indexed faster. What's concerning me, among others, is that the crawler of Google comes from time to time when looking on Google Search Console but does not seem to make progress or to even follow any link and the evolution does not seem to do what google says in GSC help. For instance, our sitemap.xml was submitted, for a few days, it seemed like it had an impact as many pages were then visible in the coverage report, showing them as "detected but not yet indexed" and now, they disappeared from the coverage report, it's like if it was not detected any more. Anybody has any advice to speed up or accelerate the indexing of a new website like ours? It's been launched since now almost two months and I was expected, at least on some core keywords, to quickly get indexed.
Technical SEO | | rolandvintners1 -
Is the Google results serp broken?
Hi everyone! We've been trying to decipher how many of our pages are indexed by google at the moment. If we do the usual "site:https://www.hobbydb.com" search term, the serp says that we have more than 740,000 pages indexed. However, when I do a deep dive and click through to the last page of results, I can only get to page 54, and then there are no more results. This would mean that I only have 540 pages indexed, not 740,000. We have also done other queries for other sub-sections of our website, and the results also truncate at 50 pages. Has anyone run into this problem? Any suggestions are appreciated! Best, Alex
Technical SEO | | mpchobbydb0 -
My Google Author Pic Disappeared
My Google author picture, which had been in place for a couple of years, disappeared from all SERP results recently. I checked, and rel=author attribution is valid on every post, as is the link to to the Google + authorship page (which contains a link back to the web site). When I test URL's in the structured data testing tool the picture appears. I'm out of troubleshooting ideas. Any suggestions welcome.
Technical SEO | | waynekolenchuk0 -
Site being indexed by Google before it has launched
We are currently coming towards the end of migrating one of our retail sites over to magento. To our horror, we find out today that some pages are already being indexed by Google, and we have started receiving orders through new site. Do you have any suggestions for what may have caused this? Or similarly, what the best solution would be to de-index ourselves? We most recently excluded anything with a certain parameter from robots.txt - could this being implemented incorrectly have caused this issue? Thanks
Technical SEO | | Sayers0 -
Pages not indexed by Google
We recently deleted all the nofollow values on our website. (2 weeks ago) The number of pages indexed by google is the same as before? Do you have explanations for this? website : www.probikeshop.fr
Technical SEO | | Probikeshop0 -
How a google bot sees your site
So I have stumbled across various websites like this: http://www.smart-it-consulting.com/internet/google/googlebot-spoofer/ The concept here is to be able to view your site as a googlebot sees it. However, the results are a little puzzling. Google is reading the text on my page but not the title tags according to the results. Are websites like this accurate OR does Google not read title tags and H1 tags anymore? Also on a slighly related note. I noticed the results show the navigation bar is being read first by google, is this bad and should the navigation bar be optimized for keywords as well? If it did, it would read a bit funny and the "humans" would be confused.
Technical SEO | | StreetwiseReports0 -
My site cannot be found by google at all
I don't know why but our company site can not be found by google at all. I have submitted to google webmaster, have social media point to, etc, Is there any reason for this? url for our website is www.bistosamerica.com Thank you
Technical SEO | | BistosAmerica0 -
How to tell google about the changes
Hello All this is my first question here. we have recently changed our website design and backend. so the problem now is in google it is showing the old url's and when ever a client come from google he is prasented with a 404 error page if this happenes many times. will google penalise our site ? if yes. how to prevent this from happen. waiting for some replies.. thanks
Technical SEO | | idreams1