Can I see when SEO Moz has crawled my website?
-
I would like to know if it's possible to see (maybe in my Google Analytics) if SEO Moz has crawled my website.
I'm also curious if and where I can see when the robot of Google visited my website.
Thanks!
-
Thanks Megan and Martijn for your quick reply! They were both helpful
-
Hi Simone! This is Megan from SEOmoz. Your best option to see what bots have crawled your site and when is by checking your server logs. If our bot came through, you'll see our useragent 'rogerbot'.
Hope this helps!
-
Hi Simone,
You're as far as I know not able to see when Rogerbot crawled your site from within SEOMoz. You should be able to see when Googlebot visits your site from within Google Webmaster Tools. They provided crawl stats to see how many pages Googlebot visited.
Hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ahrefs vs Moz
Hi! I noticed the Moz DA en the Ahrefs DA are very different. Where https://www.123opzeggen.nl/ has a DA of 10 at MOZ, the DA at Ahrefs is 26. Where does this big difference come from? Do you measure in different ways? I hope you can answer this question for me. Thank you in advance!
Moz Pro | | NaomiAdivare2 -
Question about SEO and domains
Hello, Over time we have purchased many domain names that relate to our business. Let us say we sell chairs of type A, chairs of type B, Tables of Type C etc. We also have a main web site (main domain) which has been functioning for many years and is ranked well but not great. Since we have domain chairtypeA.com and chairtypeB.com I was thinking of putting them to good use as follows. STRATEGY A a) Create the domain as a parallel domain but is mainly informational - it talks about chair type A, its history, what to look for etc. It has the same IP address as main domain b) Have all the links on the page be directly linked to the MAIN site- so if you are on page for ChairtypeA and want to see it in a set you just click a link and it takes you to the main site. c) The look and feel of the ChairtypeA site is exactly the same as the main site - in fact except for the Heading, title and content the whole framework is from the main site. My hope is that if people are looking for chairtypeA, we would rank and attract customers. Plus it would use up some of the domain names we have purchased. THE OTHER ALTERNATIVE (Strategy B) is a) Create a domain for chairtypeA parallel to main domain (same IP address) b) redirect it to subfolder in main domain c) refer to the pages also as maindomain/chairtypeA (here is where it is no longer purely parallel) I have 3 objectives a) Use my purchased domain names b) Focus on certain products (liek ChairtypeA) c) Overall increase visibility and ranking Please give me any feedback which indicates whether Strategy A or Strategy B is more effective, or if any has negative consequences. Or you could suggest Strategy C - a completely new approach. Thanks Sam
Moz Pro | | samgold0 -
Block Moz (or any other robot) from crawling pages with specific URLs
Hello! Moz reports that my site has around 380 duplicate page content. Most of them come from dynamic generated URLs that have some specific parameters. I have sorted this out for Google in webmaster tools (the new Google Search Console) by blocking the pages with these parameters. However, Moz is still reporting the same amount of duplicate content pages and, to stop it, I know I must use robots.txt. The trick is that, I don't want to block every page, but just the pages with specific parameters. I want to do this because among these 380 pages there are some other pages with no parameters (or different parameters) that I need to take care of. Basically, I need to clean this list to be able to use the feature properly in the future. I have read through Moz forums and found a few topics related to this, but there is no clear answer on how to block only pages with specific URLs. Therefore, I have done my research and come up with these lines for robots.txt: User-agent: dotbot
Moz Pro | | Blacktie
Disallow: /*numberOfStars=0 User-agent: rogerbot
Disallow: /*numberOfStars=0 My questions: 1. Are the above lines correct and would block Moz (dotbot and rogerbot) from crawling only pages that have numberOfStars=0 parameter in their URLs, leaving other pages intact? 2. Do I need to have an empty line between the two groups? (I mean between "Disallow: /*numberOfStars=0" and "User-agent: rogerbot")? (or does it even matter?) I think this would help many people as there is no clear answer on how to block crawling only pages with specific URLs. Moreover, this should be valid for any robot out there. Thank you for your help!0 -
Ignore Moz Notices?
Hi All, I read an old post (2011) about ignoring rel canonical notices from Moz. I wondered why they can be ignored? The thing is, it's showing a bunch of rel canonical errors for my pages but I know I have the tag right. At the risk of being offensive.. can the notices be ignored because the Moz data is erroneous? I'm not sure what to make of this. Thanks!
Moz Pro | | xvpn9020 -
Can Google see all the pages that an seomoz crawl picks up?
Hi there My client's site is showing around 90 pages indexed in Google. The seomoz crawl is returning 1934 pages. Many of the pages in the crawl are duplicates, but there are also pages which are behind the user login. Is it theoretically correct to say that if a seomoz crawl finds all the pages, then Google has the potential to as well, even if they choose not to index? Or would Google not see the pages behind the login? And how come seomoz can see the pages? Many thanks in anticipation! Wendy
Moz Pro | | Chammy0 -
Cannot see Bing rank in Campaign
Hi guys I have included Bing en-GB in my engine list for rankings on one of my websites, however, the actual rank placements are not showing at all in the results, even though a search in Bing shows my site is top on some of the keywords and near top on some of the others. Am I doing something wrong here? Thanks Nick
Moz Pro | | Total_Displays0 -
Do crawl reports see canonical tags?
Greetings, I just redesigned my site, www.funderstanding.com, and have the old site pointing to the new site via canonical URLs. I had a new crawl test run and it showed a large amount of duplicate content. Does the SEO Moz crawl tool validate canonical urls and adjusts the duplicate content count or is this note considered? FYI, I sent from no duplicate content to having 865 errors since the redesign went up so that seems suspicious. I would think though that assuming the canonical tag were used properly, and I hope it is?, that this would not be a problem?? All help with this is most appreciated. Eric
Moz Pro | | Ericc220 -
Where can I view all of the Questions that I have Subscribed to?
I recently asked a question about viewing the questions I have answered already: http://www.seomoz.org/q/where-can-i-view-all-of-the-questions-that-i-have-answered It turns out the link was right under my nose (but it appears a lot of people were having this issue). I'm now having the problem of not being able to see where I can view all the Q&As that I have subscribed to. Again, I may just be missing a link that's right in front of me but it certainly isn't obvious. Any help much appreciated!
Moz Pro | | NickPateman810