Can I see when SEO Moz has crawled my website?
-
I would like to know if it's possible to see (maybe in my Google Analytics) if SEO Moz has crawled my website.
I'm also curious if and where I can see when the robot of Google visited my website.
Thanks!
-
Thanks Megan and Martijn for your quick reply! They were both helpful
-
Hi Simone! This is Megan from SEOmoz. Your best option to see what bots have crawled your site and when is by checking your server logs. If our bot came through, you'll see our useragent 'rogerbot'.
Hope this helps!
-
Hi Simone,
You're as far as I know not able to see when Rogerbot crawled your site from within SEOMoz. You should be able to see when Googlebot visits your site from within Google Webmaster Tools. They provided crawl stats to see how many pages Googlebot visited.
Hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
Question about SEO and domains
Hello, Over time we have purchased many domain names that relate to our business. Let us say we sell chairs of type A, chairs of type B, Tables of Type C etc. We also have a main web site (main domain) which has been functioning for many years and is ranked well but not great. Since we have domain chairtypeA.com and chairtypeB.com I was thinking of putting them to good use as follows. STRATEGY A a) Create the domain as a parallel domain but is mainly informational - it talks about chair type A, its history, what to look for etc. It has the same IP address as main domain b) Have all the links on the page be directly linked to the MAIN site- so if you are on page for ChairtypeA and want to see it in a set you just click a link and it takes you to the main site. c) The look and feel of the ChairtypeA site is exactly the same as the main site - in fact except for the Heading, title and content the whole framework is from the main site. My hope is that if people are looking for chairtypeA, we would rank and attract customers. Plus it would use up some of the domain names we have purchased. THE OTHER ALTERNATIVE (Strategy B) is a) Create a domain for chairtypeA parallel to main domain (same IP address) b) redirect it to subfolder in main domain c) refer to the pages also as maindomain/chairtypeA (here is where it is no longer purely parallel) I have 3 objectives a) Use my purchased domain names b) Focus on certain products (liek ChairtypeA) c) Overall increase visibility and ranking Please give me any feedback which indicates whether Strategy A or Strategy B is more effective, or if any has negative consequences. Or you could suggest Strategy C - a completely new approach. Thanks Sam
Moz Pro | | samgold0 -
Why is the SEOmoz crawler crawling the old version of our website?
Hello, I'm a new SEOmoz member. On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server. The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet. After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO. For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible. Anyone has a clue? When will the new URLs be indexed by SEOmoz and the major search engines? Thanks for your help!
Moz Pro | | Gestisoft-Qc0 -
SEomoz slow to crawl?
Hello - I am just trying out the trial and it said the next crawl was nov 1st but I see no change in any of the errors since the initial crawl... so just waiting to find out if what I changed was fixed or not. Is this normal ?
Moz Pro | | Bethany_BabyBrowns0 -
Possible to have more then one SEO Moz login / account
We have an SEO Moz subscription and we were wondering if there was a way to setup multiple users attached to that subscription (i.e. separate logins).
Moz Pro | | Panjiva2 -
Question about when new crawls start
Hi everyone, I'm currently using the trial of seomoz and I absolutely love what I'm seeing. However, I have 2 different websites (one has over 10,000 pages and one has about 40 pages). I've noticed that the smaller website is crawled every few days. However, the larger site hasn't been crawled in a few days. Although both campaigns state that the sites won't be crawled until next Monday, is there any way to get the crawl to start sooner on the large site? The reason that I've asked is that I've implemented some changes that will likely decrease the amount of pages that are crawled simply based upon the recommendations on this site. So, I'm excited to see the potential changes. Thanks, Brian
Moz Pro | | beeneeb0