Moz Not Crawling Angular SPA
-
I have a client that just launched a redesigned website using Angular as a single page app. Google appears to be able to crawl the site just fine, but Moz crawl is only finding one page. We have updated the htaccess to allow for Rogerbot and Dotbot, but still unable to crawl any pages other than the home page. Does anyone have experience with this or ideas of why it won't crawl all pages, and how to allow for Moz to crawl all pages? There is a sitemap with approx. 390 pages.
Thanks!
-
I recommended to another Moz agent that it put a clearer warning label to users that Moz Pro's on-page grader and site crawl will not work with single page apps. Alternatively, I'd have a pop-up if the Moz tools run and come up with results like the ones the poster described. I ran into the same issue.
-
Hi!
One way to get around this one is making use of pre-rendering. So if you setup your site/server to pre-render your pages either for all user agents, or specifically for MozBot and other bots not capable of crawling through JavaScript then it might work.
Facing the same issue for us now, but will try this at least.
Best regards,
Anders -
Thank you very much for your quick response. I'm taking a look at the Q&A section and the tools you recommended. I appreciate your help!
-
Hi there! Tawny from Moz's Help Team here.
Thanks for writing in and sorry about the trouble!
I took a look at your Site Crawl results and your site, and I believe this is a result of our crawler's inability to work with Javascript. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. It looks like, due to the Javascript on the page, we weren't able to find any links to keep crawling: https://www.screencast.com/t/mqy7SUb9f
There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but I did find some good blog posts and discussions in the Q&A about this when I searched our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
I'm really sorry I can't be of more help here; I'll definitely be sure to pass this along to our Product team as feedback on your behalf. If you have any follow-up questions or if there's anything else you need help with, please write in to help@moz.com and we'll do our best to sort everything out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz Site Crawl can't index WIX sites
We've been attempting to work on some SEO for a new potential client however they are using a WIX site. We've noticed that Moz SEO tools will not index any WIX sites. e.g. https://www.sharonradisch.com/ (which is one of their case studies). Anyone seen this that can offer any advice? Thanks,
Getting Started | | monkeex
Mark2 -
Moz Pro Warning: Redirect Chain
I have just signed up to a Moz Pro account, and after it finished crawling my website it gave me a warning about a redirection chain. http://elementpaints.com >> https://elementpIaints.com >> https://www.elementpaints.com I'm trying to find some more information about how to fix this problem but I'm not having much luck. This article I found even says it is not a problem: https://really-simple-ssl.com/knowledge-base/avoid-landing-page-redirects So now I'm even more confused, do I still need to fix this? If so, how do I do that? FYI: I have Lets Encrypt SSL cert installed on my server, and I'm using Cloudflare with Full SSL option and HSTS enabled, and "Always Use HTTPS" option is turned on. | http://elementpaints.com |
Getting Started | | elementpaints0 -
Correct Moz Settings for Tracked Keywords Summary
Hi,For some reason, on my tracked keywords summary page, the URL shown to the right of each keyword appears to default to my homepage and not the page I want that keyword to rank on. How do I resolve this? I've tried looking around but can't see how to change it so that the correct URL is shown against the correct keyword. Many thanks! Bob
Getting Started | | SushiUK0 -
901 error code showing url back to back in crawl
Hi Everyone, I'm absolutely dumbfounded about this 901 issue (showing pages with our url back to back). Our site is hosted on Big Commerce: https://www.santabarbarachocolate.com When I look for these pages being crawled I don't find them. I've called BC for help and I can't seem to find a solution or where to turn as to how to fix the issue at hand or even if it matters. Please see below what the Moz crawl shows. Could this be related to Yotpo or some app we have running? Or does this even matter and does it have any influence on rank? Do you have recommendations or ideas? Thanks so much. Pages with Crawl Attempt Error as of Mar 3 URL Page Authority Linking Root Domains Status Code | Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/100-percent-pure-cacao-unsweetened-baking-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/buy-wholesale-bulk-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/organic-chocolate-wholesale | -- | -- | 901 |
Getting Started | | santabarbarachocolate0 -
MOZ Guru Needed for Freelance Work
Hello -- I'm new to Moz and wanted to know if any community members were available for freelance work to help me fine-tune the setup of the Moz account for one of my clients. This could lead to additional work for other clients of my advertising agency.
Getting Started | | KRT0 -
I am new to MOZ, I set up one tracking campaign two weeks ago, I have tracked no keywords, I have done some keyword research for ranking difficulty and in two weeks I have already hit 50K pages crawled, I'm maxed out, is this common?
I am a startup and can't afford the higher plans yet. And even their highest plan is 600K pages crawled, which seems really low considering how lightly I used the tool and how quickly I hit 50K. Does anyone have any advice or information on how they use the tool on lower packages? Did I do something wrong to hit 50K pages crawled that fast? Does this pricing make any sense, it seems like an incredibly high price, I love the tool, any help is appreciated.
Getting Started | | Daedilus1 -
Moz Staff: History + Watch List
Hi All, I have been going crazy trying to find a thread I read last week ... I think!
Getting Started | | Mark_Ch
Several search attempts later, no closer to finding the thread in question. A genius moment or simple frustration got me thinking. How cool would it be to have the following features on MOZ Q&A Forum. Watch List
1] Check off any threads that you would like to monitor over time. Should you stumble upon a thread of value then simply add to watch list and view it's progress over time. No searching for the thread in question. History
2] Keep a history off all threads viewed over a period of time. Thanks Mark0 -
How do I interpret Duplicate Content in a Crawl Report, when it only gives me a URL? How do I know what is duplicated on that page somewhere else?
I need help interpreting the Crawl Report for Duplicate Content. It gives me the URLs of pages that have duplicate content, but how do I know what content exactly is duplicated elsewhere? And how do I figure out where it is duplicated? Also, are there Moz Analytics articles or videos teaching you how to use each component of the analytics programs? Thanks!
Getting Started | | NancyBryan0