Why are only two pages of my site being crawled by MOZ?
-
MOZ is only crawling two pages of my site even though there are links on the homepage, and the robot.txt file allows for crawling. Our site address is www.hrparts.com.
-
I had to remove subdomains from being crawled on the site, and the issue has been resolved. The 301 redirect was an issue, causing MOZ to stop crawling at only two pages, the http and https pages.
-
Thank you for your response, Lauren. Our site only has one major feature that is in Javascript, but the rest of the site is not. We do have a 301 redirect from hrparts.com to www.hrparts.com which is where I believe the issue might be coming from. I am curious whether MOZ has an issue with a 301 redirect in a campaign?
-
Hi there!
Thanks for writing in and sorry about the trouble!
I took a look at your Site Crawl results and your site, and I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
https://www.onely.com/tools/wwjd/ is also a handy tool that makes it super easy to see where links are added or removed by javascript.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Cancel my MOZ Subscription
Cancel my MOZ Subscription. My email id is vtsdesigner106@gmail.com
Product Support | | visiontechnoindia0 -
Why doesn't Moz read the right meta description?
We have a WordPress blog and Moz continues to read our site title as the meta description instead of the nicely crafted and SEO-friendly meta descriptions we are writing on each post in our Yoast plugin. Can someone help use understand why Moz is reading the site title rather than the post meta descriptions? (We did test deleting the site title all together but then this made Moz think that there is no meta description for any of the posts.)
Product Support | | simran-s1 -
Is is possible to revert reporting to a past crawl date?
Site Crawl report defaults to the last crawl. Is there a way to get data from a previous crawl for comparison?
Product Support | | JThibode1 -
I canceled my free trial. Does Moz charge after expiring date although I canceled the free trial?
I canceled my free trial, will Moz charge after expiring date, unless I canceled?
Product Support | | Samantha-Amarakoon0 -
Rogerbot not crawling our site
Has anyone else had issues with Roger crawling your site in the last few weeks? It shows only 2 pages crawled. I was able to crawl the site using Screaming Frog with no problem and we are not specifically blocking Roger via robots.txt or any other method. Has anyone encountered this issue? Any suggestions?
Product Support | | cckapow0 -
Moz Point Swag
So, I've been away from the SEO world for a few years but now I'm back in full swing and I noticed I have enough points for "A special MozPoints t-shirt and a Roger vinyl figurine" but had never gotten those because I assume it must have become an added swag during my hiatus... but I'd absolutely love to get hold of that as my boss is a big Roger fanboy and it would be hilarious to have that on my desk as a friendly mocking. If I can't, I can't. But I figured it couldn't hurt to ask.
Product Support | | MikeRoberts2 -
Remove of No follow when you become a specialist on Moz
Hi It says when you get over 200 moz points that you "Removal of "nofollow" from first custom URL on profile", this isn't the reason I have become more active on the network, but I was wondering do I need to do anything or does it happen automatically. Thanks Andy
Product Support | | Andy-Halliday0 -
MOZ is having a redirect loop issue?
Hey guys, Is anyone else having an issue with this? I leave MOZ analytics open all day and it times out after a while. When you refresh your page when it has timed out, it goes into a redirect loop. It also doesnt let you log back in before you first log out (even though your session has already timed out). Not the greatest UX and I thought I should see if anyone else is having the same issue. Let me know your thoughts!
Product Support | | JoshGaskin0