Please help us undertsand the things we need to improve so that google crawler visit us more often to reindex pages from our domain
-
we are currently in the process of a massive project which involves us migrating our domain, we realised that Google crawlwer has not been crawling our pages Quiet often. i have observed some cases where google crawled these pages about 6 months back and then never visited the pages again
and we had to manually submit these pages for reindexing in some geographies. can you please help us undertsand the things we need to improve so that google crawler visit us more often to reindex pages from our domain -
Hi Bhaskaran,
Someone with more knowledge than me may be around soon to give you a more in depth answer, but this should get you going.
Google has a limited amount of resources to crawl each webpage, so to be efficient it schedules pages to be crawled based on a few different things. The one i suspect is affecting your crawl times is how often your content is updated. If google does not believe that your site will be updated frequently it sees no reason to crawl it that often.
If you want google to crawl more often you need to give it a reason to do so, that means ensure your site is easily crawled by bots, good internal linking and make sure your content is fresh and up to date.
On a side note, as you are carrying out a domain migration it might be easier to redirect the pages to the new page on the new domain (301 redirect) and ensure your new domain has the points i raised above.
I hope this helps?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Advice needed on canonical paginated pages
Hi there. I use Genesis and StudioPress themes. I recently noticed that the canonical link for blog pages points to the first page on all paginated pages, which I understand is an SEO no-no. I found some code here that adds a unique canonical link to each paginated page but for categories only. It works fine. I only have one category for my site. My question is: is there a downside (or even upside) to not having a blog page and placing a link to my category page in the navigation bar instead, using the category page as the blog page? It looks good and works. What do you think? I find it odd that this seems to be an issue across the Internet and the only solution that comes up relies on the Yoast plugin, which I don't want to use (don't want to use a plugin for SEO). Thanks in advance.
Intermediate & Advanced SEO | | Nobody16165422281340 -
Google slow to index pages
Hi We've recently had a product launch for one of our clients. Historically speaking Google has been quick to respond, i.e when the page for the product goes live it's indexed and performing for branded terms within 10 minutes (without 'Fetch and Render'). This time however, we found that it took Google over an hour to index the pages. we found initially that press coverage ranked until we were indexed. Nothing major had changed in terms of the page structure, content, internal linking etc; these were brand new pages, with new product content. Has anyone ever experienced Google having an 'off' day or being uncharacteristically slow with indexing? We do have a few ideas what could have caused this, but we were interested to see if anyone else had experienced this sort of change in Google's behaviour, either recently or previously? Thanks.
Intermediate & Advanced SEO | | punchseo0 -
Google Adsbot crawling order confirmation pages?
Hi, We have had roughly 1000+ requests per 24 hours from Google-adsbot to our confirmation pages. This generates an error as the confirmation page cannot be viewed after closing or by anyone who didn't complete the order. How is google-adsbot finding pages to crawl that are not linked to anywhere on the site, in the sitemap or linked to anywhere else? Is there any harm in a google crawler receiving a higher percentage of errors - even though the pages are not supposed to be requested. Is there anything we can do to prevent the errors for the benefit of our network team and what are the possible risks of any measures we can take? This bot seems to be for evaluating the quality of landing pages used in for Adwords so why is it trying to access confirmation pages when they have not been set for any of our adverts? We included "Disallow: /confirmation" in the robots.txt but it has continued to request these pages, generating a 403 page and an error in the log files so it seems Adsbot doesn't follow robots.txt. Thanks in advance for any help, Sam
Intermediate & Advanced SEO | | seoeuroflorist0 -
Homepage is losing rankings...need help!
Hi folks, Our company has been doing SEO for www.3dincites.com since October. Since the website hadn't been optimized at all, one of the first things we did was to add keyword-optimized page title and meta description to the homepage. The person who implemented didn't do it right the first time so we needed to change the page title again after a week or so. We also changed the preferred domain in GWT. Ever since those changes were implemented, the homepage has been decreasing in rankings for the the keywords we optimized it for. This is very surprising as the website has high-quality unique content and pretty solid backlinks. Moreover, the search traffic to other pages is through the roof (2K increase since October) so the client is happy, however, this decrease in rankings for the homepage (from 3d to 6th page for "3d IC technology" which is the main keyword) doesn't let me sleep well at nights. Any ideas why this is happening?
Intermediate & Advanced SEO | | zoomzoom0 -
Need help for improving SEO App?
Please refer our guru99 https://play.google.com/store/apps/details?id=com.vector.guru99&hl=en We have tons of free materials related to SAP in the app. The problem we are facing is we do not rank for terms like SAP or SAP tutorial and discovery is an issue I have searched over the internet and found no concrete solution. Can you experts help ?
Intermediate & Advanced SEO | | Chirag7530 -
Why is this page not being delivered for Google search result?
Hey folks, Figured I would try to get an experts insight on this. On google search result for BLACK TITANIUM RINGS + TITANIUM-JEWELRY.COM the page that I "think" should show up is this one: http://www.titanium-jewelry.com/black-titanium-rings.html However, it does not. Imho, this page is highly relevant. I used Rank Tracker here on seomoz.org and the page is not even in top 50 of search engine results for google. Our 'About Black Titanium Rings' page ranks #2 (http://www.titanium-jewelry.com/about-black-titanium.html) but the /black-titanium-rings.html page doesn't even rank. Any suggestions on what I could look at to figure out why this page is being penalized? We are not under a manual penalty (anymore!). Thanks! Ron
Intermediate & Advanced SEO | | yatesandcojewelers0 -
Merging your google places page with google plus page.
I have a map listing showing for the keyword junk cars for cash nj. I recently created a new g+ page and requested a merge between the places and the + page. now when you do a search you see the following. Junk Cars For Cash NJ LLC
Intermediate & Advanced SEO | | junkcars
junkcarforcashnj.com/
Google+ page - Google+ page the first hyperlink takes me to the about page of the G+ and the second link takes me to the posts section within g+. Is this normal? should i delete the places account where the listing was originally created? Or do i leave it as is? Thanks0 -
Why is my XML sitemap ranking on the first page of google for 100s of key words versus the actual relevant page?
I still need this question answerd and I know it's something I must have changed. But google is ranking my sitemap for 100s of key terms versus the actual page. It's great to be on the first page but not my site map...... Geeeez.....
Intermediate & Advanced SEO | | ursalesguru0