Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Moz crawler only crawls one page?!
-
Hello there,
I'm using Moz for a while and I'm very pleased with the tool and community. But for the first time I encountered a problem. We are trying to run a crawler for a client's website but only one page (only the homepage) was crawled.
We tried to do a test on a more detailed level (maybe there is something wrong with the homepage). My campaign test's crawl came back for the Producten folder (level deeper than homepage), and it was also only a 1 page crawl with a 200 status. I did look at the robots.txt file now, and it is very restrictive, but there is nothing that I can clearly see that would explain why the crawl isn't working.
Hopefully someone can point us at the right direction.
Thanks in advance,
Jeremy
-
Hey Jeremy,
I took a look at your campaign and it looks like Lynn's suggestion that Javascript may be blocking the crawl is correct. Unfortunately, our crawler isn't sophisticated enough to parse Javascript and all of the links from the homepage are hidden within the Javascript navigation so we aren't able to find them to crawl past the homepage. I'm afraid it looks like our crawler isn't compatible with this particular site.
I apologize for the inconvenience this causes! Please let me know if I can help you with anything further.
-
Thanks!!! I will take a look at the dev blog post
-
Hi,
It can happen for a number of reasons, in my experience usually due to either unintentional blocking of the page from crawling or from being a JavaScript heavy/only site which doesn't allow content to be easily read by the moz crawler. Check out this dev blog post for a rundown on most issues and solutions. Hope it helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz DA Factors
Hello Folks, I just wanna know that on which factors does Moz calculates DA ...? and also wanna know my site DA can anyone tell me Helmet Best
Moz Bar | | Alisa8844880 -
Moz not updating the spam score metrics
Hi Experts, I have done everything to come over the following metrics flagged as spam on my website by Moz, few months ago. But, Moz has not updated the spam score yet. ✓Low MozTrust or MozRank Score - improved from 2 to 4.5
Moz Bar | | jamesh.rich01
✓Large Site with Few Links - My website have more than 6K backlinks
✓Small Proportion of Branded Links - My website have a good amount of branded backlinks
✓Thin Content - Every webpage on website has more than 500 words content
✓External Links in Navigation - There is no external link in navigation other than social media links
✓No Contact Info - The proper address has alreay been placed on website footer
✓Low Number of Pages Found - I am wondering if there are any standard score or number of links to reach to remove these flag?
Also, please suggest some ways that will help me improve moz spam score at faster rate. Thanks for your help in advance!"0 -
Need to solve "Oops our crawlers were unable to access" url for new campaign
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
Moz Bar | | alisacromer0 -
How to upload the bulk Keywords with Tags in MOZ Rank Tracker Tool?
Trying to upload multiple keywords at a time with their different Tags. But here i can upload the keyword one by one also i am not able to associate tags with the keyword.
Moz Bar | | _nitman2 -
Should I exclude prepositions in tracked keywords of moz analytics?
I'm new to Moz. Just set up my trial campaign, and it had suggested many keywords. Many of the phrases that were suggested do not contain prepositions. For example, instead of something like "sporting good stores in Chicago" it suggested "sporting good stores Chicago" Today, I looked at the on-page optimization suggestions, which are (of course) suggesting that I remove prepositions from my page to rank well. Well, as you know, that is unnatural to the reader. But I suspect people are searching in higher volume, leaving the prepositions out. I know that if I were to search for a sporting goods store in Chicago, I would probably leave out "in." What should I do? Should I remove all the suggested keywords, and make them readable (which people are less like using in their search?) Do I go back to all my pages and try to optimize it for a keyword that is natural, but does not include a preposition (such as Chicago sporting goods stores) or should I be doing something else?
Moz Bar | | osaka731 -
Moz reporting for C-Blocking
Hey Mozers, I see Moz has a reporting tool for C-blocking and for november I had 330. Does this mean 330 Ip addresses came from the same location in the month of november?
Moz Bar | | rpaiva1 -
Duplicate page issue
Hi Guys We were recently having trouble with an excessive amount of duplicate page titles. So I asked our web company, at a reasonable expense, to fix the issue which they did. How ever I since note that the issue has returned (please see the attached graph. Could anyone explain to me why this might have happened? I t would be great to have some insight before i go back to them. Thanks again for your help Regards Pete Capture.jpg
Moz Bar | | Hardley1110
