Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Moz crawler only crawls one page?!
-
Hello there,
I'm using Moz for a while and I'm very pleased with the tool and community. But for the first time I encountered a problem. We are trying to run a crawler for a client's website but only one page (only the homepage) was crawled.
We tried to do a test on a more detailed level (maybe there is something wrong with the homepage). My campaign test's crawl came back for the Producten folder (level deeper than homepage), and it was also only a 1 page crawl with a 200 status. I did look at the robots.txt file now, and it is very restrictive, but there is nothing that I can clearly see that would explain why the crawl isn't working.
Hopefully someone can point us at the right direction.
Thanks in advance,
Jeremy
-
Hey Jeremy,
I took a look at your campaign and it looks like Lynn's suggestion that Javascript may be blocking the crawl is correct. Unfortunately, our crawler isn't sophisticated enough to parse Javascript and all of the links from the homepage are hidden within the Javascript navigation so we aren't able to find them to crawl past the homepage. I'm afraid it looks like our crawler isn't compatible with this particular site.
I apologize for the inconvenience this causes! Please let me know if I can help you with anything further.
-
Thanks!!! I will take a look at the dev blog post
-
Hi,
It can happen for a number of reasons, in my experience usually due to either unintentional blocking of the page from crawling or from being a JavaScript heavy/only site which doesn't allow content to be easily read by the moz crawler. Check out this dev blog post for a rundown on most issues and solutions. Hope it helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is Moz more accurate than Ahrefs ?
When I check my this website https://joinpakarmy.com.pk/ on Moz it gives me DA 3
Moz Bar | Aug 6, 2020, 12:12 PM | joinpakarmy234
But when I check it out on Ahrefs, it tells me there is no authority for this site.
So who is more accurate in the result?
thanks0 -
Site crawl only shows homepage
Hi everyone, A client of us has a quite new website with a lot of URLs. (Google Search Console indicates around 5300.) However, when I execute a site crawl with screaming frog, or a crawl test in MOZ, it only shows me one URL, the homepage. Does somebody have an idea why the other pages of the website are not showing up? Thanks,
Moz Bar | Jan 2, 2018, 7:14 AM | WeAreDigital_BE
Jens0 -
Are we actually getting accurate data on keyword volumes from Moz (or other sources)?
I have a client who does patio furniture repair and restoration. When performing keyword research in Moz for terms like "patio furniture repair" I see that only 11-50 people in the entire US are searching for this term according to the Moz data. However, running an Adwords campaign currently and our top keyword is the phrase match for "patio furniture repair" which has generated over 100 clicks in just a couple of months in ONE county. Is there a better way to research more accurate results on search volume estimates? This makes organic SEO and keyword targeting hard! Thanks, Ricky
Moz Bar | Sep 25, 2017, 2:27 AM | RickyShockley1 -
How to upload the bulk Keywords with Tags in MOZ Rank Tracker Tool?
Trying to upload multiple keywords at a time with their different Tags. But here i can upload the keyword one by one also i am not able to associate tags with the keyword.
Moz Bar | Jun 13, 2016, 8:19 AM | _nitman2 -
What does the Bold/ Strong mean in Moz bar?
Under On-Page Elements in the Moz bar there is a Tag/ Location called Bold/ Strong. What does that mean?
Moz Bar | Sep 10, 2015, 5:47 PM | TiffanyatElite0 -
I want to uninstall the Moz SEO toolbar. How do I do this?
I installed the Moz toolbar and I don't understand it and it covers up important parts of websites and makes them inaccessible. I want to get it off my computer. I installed it in chrome. How do I get it off?
Moz Bar | Sep 29, 2014, 6:28 PM | Bonnie761 -
Duplicate page issue
Hi Guys We were recently having trouble with an excessive amount of duplicate page titles. So I asked our web company, at a reasonable expense, to fix the issue which they did. How ever I since note that the issue has returned (please see the attached graph. Could anyone explain to me why this might have happened? I t would be great to have some insight before i go back to them. Thanks again for your help Regards Pete Capture.jpg
Moz Bar | Oct 13, 2013, 11:27 PM | Hardley1110 -
How do you stop Moz crawling a page?
Hello, I have a contact form which generates thousands of duplicate crawl errors. I'm going to use to block Google indexing these pages. Will this also block MOZ from crawling these pages and displaying the error? Thanks!
Moz Bar | Nov 28, 2016, 7:23 PM | Seaward-Group0