Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Moz crawler only crawls one page?!
-
Hello there,
I'm using Moz for a while and I'm very pleased with the tool and community. But for the first time I encountered a problem. We are trying to run a crawler for a client's website but only one page (only the homepage) was crawled.
We tried to do a test on a more detailed level (maybe there is something wrong with the homepage). My campaign test's crawl came back for the Producten folder (level deeper than homepage), and it was also only a 1 page crawl with a 200 status. I did look at the robots.txt file now, and it is very restrictive, but there is nothing that I can clearly see that would explain why the crawl isn't working.
Hopefully someone can point us at the right direction.
Thanks in advance,
Jeremy
-
Hey Jeremy,
I took a look at your campaign and it looks like Lynn's suggestion that Javascript may be blocking the crawl is correct. Unfortunately, our crawler isn't sophisticated enough to parse Javascript and all of the links from the homepage are hidden within the Javascript navigation so we aren't able to find them to crawl past the homepage. I'm afraid it looks like our crawler isn't compatible with this particular site.
I apologize for the inconvenience this causes! Please let me know if I can help you with anything further.
-
Thanks!!! I will take a look at the dev blog post
-
Hi,
It can happen for a number of reasons, in my experience usually due to either unintentional blocking of the page from crawling or from being a JavaScript heavy/only site which doesn't allow content to be easily read by the moz crawler. Check out this dev blog post for a rundown on most issues and solutions. Hope it helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
In MozBar Page Analysis under General Attributes Country is empty or wrong
Some of our clients sites have a top level domain for Mexico, but in the Mozbar they doesn't show any country, is this related to a known bug? https://www.edacom.mx/ https://www.dropbox.com/s/a8jgr7ztk1im15k/Captura de pantalla 2019-08-15 12.41.10.png?dl=0 https://www.cliento.mx/ https://www.dropbox.com/s/a8jgr7ztk1im15k/Captura de pantalla 2019-08-15 12.41.10.png?dl=0 In the search console: "International Targeting report" the target is Ok to Mexico.
Moz Bar | | Andrea_Ugalde1 -
If we put the disavow links in google, does MOZ crawl the same links?
I have put bad or spam links in disavow file, but still showing in MOZ backlinks. So, I want to know that Why is MOZ not removing the spam links from their system?
Moz Bar | | insidewebanalytics0 -
80% Spam Score!! Can Any One Help?
I have a question about the Spam Score for my website .
Moz Bar | | PoppyP
I have run a Spam Score for https://www.poppyporter.co.uk
It has come back with an 80% score because there are 4 links that are apparently from my poppyporter.com domain each with a 69% spam score.
This poppyporter.com domain is parked and I have no idea why there are links from it. There are no pages there and there never have been, I've never used the poppyporter.com domain only poppyporter.co.uk.
I don't understand why there are links there and why they are making my Spam Score so high.
Will they affect my site's SEO? Do I need to worry about them? If so how do I get rid of them?
Can you help?
Thanks very much Poppy1 -
Need to solve "Oops our crawlers were unable to access" url for new campaign
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
Moz Bar | | alisacromer0 -
Error 406 with crawler test
hi to all. I have a big problem with the crawler of moz on this website: www.edilflagiello.it. On july with the old version i have no problem and the crawler give me a csv report with all the url but after we changed the new magento theme and restyled the old version, each time i use the crawler, i receive a csv file with this error: "error 406" Can you help me to understan wich is the problem? I already have disabled .htacces and robots.txt but nothing. Website is working well, i have used also screaming frog as well.
Moz Bar | | ArchieDonnithorne1 -
Moz Bar Not Showing DA?
Hello all, This could be something to do with our site or the Moz bar on Chrome, I just need to know which it is so if it is our website we can look into it further. On certain sections of our website the Moz bar doesn't display any Domain Authority, not even zero, the bar just isn't present. These types of pages are php which pull in data through a feed daily. Speaking to an SEO expert they said it could be where the page is being updated so frequently, or it could be something more sinister and technically not quite right. Does anyone have any ideas? Is the Moz bar just not working for these types os pages or is it more likely something to do with my site? Ironically it's these pages which I'm having trouble with that are not showing in SERPs! Thanks! 3Foorka
Moz Bar | | HB176 -
How Does Moz Shoot Whiteboard Fridays?
Okay, sort of a meta question here... My company, a global test preparation company, is looking into new ways to deliver online course content (both pre-recorded and live streaming). Around the office we have looked at dozens of examples of companies that do video presenting or teaching, and then I realized that I watch a great example of high-quality online video every week: Whiteboard Fridays! They look and sound fantastic! Moz team, what are you willing to share re: the setup you use? I'm looking for specific direction about lighting, sound, and equipment. Anything you share would be VERY much appreciated. Thanks!
Moz Bar | | ScottShrum
Scott 3-methods-fueled-by-data-and-tools-to-earn-more-and-better-links-whiteboard-friday0 -
Is Moz Analytics still available to all MozCon attendees?
I thought all Mozcon attendees were going to get access to Moz Analytics even after the convention was over. I can't seem to find were to access it from my campaigns manager dashboard and the Analytics landing page says im on queue to get access.
Moz Bar | | AmberHanson0