Angular SPA & MOZ Crawl Issues
-
Website: https://www.exambazaar.com/
Issue: Domain Authority & Page Authority 1/100
I am using Prerender to cache/render static pages to crawl agents but MOZ is not able to crawl through my website (https://www.exambazaar.com/). Hence I think it has a domain authority of 1/100. I have been in touch with Prerender support to find a fix for the same and have also added dotbot to the list of crawler agents in addition to Prerender default list which includes rogerbot. Do you have any suggestions to fix this?
List: https://github.com/prerender/prerender-node/commit/5e9044e3f5c7a3bad536d86d26666c0d868bdfff
Adding dotbot to Express Server:
prerender.crawlerUserAgents.push('dotbot'); -
Cheers Gaurav
I will try that technique. We use something similar for the title but in the Angular app.js file, however have struggled to do the same for the description, canonical tags etc.
Actually right now we add the tags we need in Javascript, using the DOM and prerender and google seems to like it. We can;t run a crawl any longer in the std Moz package for some reason we can only do one per week now. Moz doesn't seem to cope too well with the Javascript though. When we tried other tools it seemed to c heck out fine.
I'm not sure if it is Moz crawlers that are causing us the issue here, anyway!
We're also still using Angular 1.6 so we really need to upgrade to v4 or at least 2.
-
Hi Dave
Thanks for asking. You can set the pageDescription variable (similarly for other meta tags) in each route/state of the SPA which gets cached by Prerender. If you are using prerender you can see the list of default crawler agents supported here: https://github.com/prerender/prerender-node/commit/5e9044e3f5c7a3bad536d86d26666c0d868bdfff
If you wish you can enable prerender for the crawler dotbot too!
-
Hi Gaurav
We're also launching a new MEAN SPA, We've gotten our site looking quite good for SEO now, other than the meta description and canonical tags.
However I believe the MOZ crawler still doesn't understand javascript, we use AngularJS or JQEURY to change and add the meat tags for each page, which I am confident is OK with Google but MOZ and SEMRush don't like it.
That could be something to do with the lower score on Moz?
Do you mind me asking how do you set your page meta descriptions and canonical tags?
Dave
-
Hey, Dave from the Help Team here!
I just ran your site through our crawler and didn't have any issues crawling your site. I think the Domain Authority score of 1/100 is actually caused by us not having your site in our Mozscape Index. To find out more about how we Index the web, you can read about it here
https://moz.com/help/guides/research-tools/open-site-explorer/updates/index-process
Hopefully that helps to clarify, let us know if you need anything else!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl depth seems off?
I'm reviewing my site crawl data and am seeing some very strange things such as: The homepage URL has a listed crawl depth of 2. Pages that are featured in the main site navigation (which is present on all pages, including homepage) are ranking at a crawl depth of 3. What am I missing here? Shouldn't my homepage have a crawl depth of 0 or 1? Why would pages linked directly from my homepage have a crawl depth other than 1? (Single click from homepage to that page)? Thank you!
Link Explorer | | LianaLewis0 -
Moz crawling doesn't show all of my pages
Hello, I'm trying to make an SEO on-pages audit of my website. When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ? How can i check and correct this. My website is discoolver.com
Link Explorer | | MK-Discoolver1 -
Can MOZ keyword explorer output sentences instead keywords?
Can MOZ keyword explorer output sentences instead keywords? We need to optimize our website for the Knowledge Graph (OG). There was an old tool that does it, but I can't remember its name.
Link Explorer | | Elchanan1 -
Sudden Spike in 404 Pages Not Found in Moz Crawl But No Errors in WMT
Recently I received a spike in errors from the Moz crawler. When I looked into the matter I noticed that all the URI's looked right but then I looked a little closer and there was a /page/2 and /page/3 in front of the URI's. I'm running a WordPress website. Immediately I thought to myself this must be some kind of caching or permalinks error. So I disabled all my plugins including W3 Total Cache and ran the Integrity Link Crawler for the Mac and found that the errors were still popping up. 404-errors-ncworkercomp.png?dl=0
Link Explorer | | NCCompLawyer0 -
Compare Metrics in Moz
Wondering if there are any webinars or websites available to help me better understand and break down the different categories and what they specifically mean when you run a website through the Compare Metrics in Moz?
Link Explorer | | MainstreamMktg0 -
Moz's Spam Score - WWW v Non-WWW
Given that going with a www or non-www TLD is a coin-toss in Google's eyes, I'm curious as to how this relates to Moz's spam score (if at all)? Specifically, we've got a client who has several sites across a number of industry verticals, and for some of them, the www and the non-www versions of their websites are both in play (sigh). Obviously we'll sort that out shortly, but my question is why, when we put both TLDs into Open Site Explorer does the www and the non-www version of the sites get different spam scores? With some of their sites (www v non-www) the difference in the spam score might be a 2 for the www v a 3 for the non-www. But in others it might be 0 for the www v 8 for the non-www! And it's unclear which one is the 'real' spam score, as there seems no rhyme or reason to the results. Anyway, any thoughts from the room would be appreciated.
Link Explorer | | SEO-NS0 -
I have 1/100 PA & DA with a gTLD ".website" ?
Hello all, We launched a website builder with a ".website" domain and our Domain & Page authority is currently 1/100 . I would like to know if we'r making a mistake by using a ".website" gTLD ? Our site is lyl.website Thanks for your help !
Link Explorer | | LylConcepts0 -
Issues Checking Backlinks To YouTube Videos
Hi, Hope someone might be able to help here. I have been using OSE to check backlinks to sites without an issue. However, I do seem to have problems checking links to videos. For examplpe, I took a video url and checked backlinks and OSE reports 33 million just discovered, majestic SEO reports none and Ahrefs seems about right with 17. I was wondering if there is something I need to do to ensure OSE returns correct results for individual youtube videos. FYI, I am using the url that appears in the Google search. Thanks Mark
Link Explorer | | mitonmark0