Moz bot has trouble crawling Angular JS - I believe it's seeing the SPA (Single Page Application) before Universal. Anyone else have this issue? What is the fix?
-
The Moz bot user agent detection settings are able to read Universal, but the Single Page Application (SPA) version partially loads on the website before Universal. Because of this, Moz (and possibly search engines) think we have massive duplicate content issues. For example, the crawl report said a particular product page (which has about 1,000 words) has 33,000 words and has duplicate content with over 300 other pages. This makes me believe it's only picking up the SPA version. Has anyone come across this, and what would be the fix?
-
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
https://www.onely.com/tools/wwjd/ is also a handy tool that makes it super easy to see where links are added or removed by javascript.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz keyword mention on-page counting errors
Hi. Moz is showing 18 mentions of the keyword 'street furniture' on this landing page https://www.broxap.com/street-furniture.html But I can only count 6 in total in the body copy and 13 if you include navigation links. This is the same on other pages too for that keyword. Does anyone know where it's counting these extra keywords from? I don't want to fall foul of keyword stuffing but as far as I can see we're not! Could Moz be miscalculating? Any help appreciated! Thanks Joe
Moz Bar | | iweb_agency0 -
GA snippets for subdomains and best tracking in MOZ
Hey there, I have a blog running at blog.URL.com. I need to add Google Analytics and I want to track the value of this subdomain as separate from the main URL. Can I do that? If I do a unique snippet in the header of the blog is that the right way to go? The main site is on an antiquated CMS system and will be converted over the next few months to a full on WP site, does that change anything long-term? Thanks in advance for your help. Cheers, Lisa
Moz Bar | | LisaBOS0 -
Unusual "internal links" causing SEO issues?
Hi all, I'm working on an ecommerce site which has been around for almost 20 years. Over the years it has started to suffer in Google's search results and the decision was recently made to completely overhaul the site. We're now very happy with the website's design, and care was taken to maintain page rank via 301s, etc. However, the site has just fallen off the bottom of Google's first search result page (for the first time in years) for our main keyword. I signed up here in the hope of using Moz's SEO tools to help us return to our former glory, but I'm seeing some confusing results: I've run a crawl test on our site, as well as on our two biggest competitors. One thing that really stood out was that we have over 1000 "internal links" to our homepage, whereas our competitors both have around 20-30 (both of which appear at the top of the first SR page). Since the rest of the "on-page SEO" looks OK, I suspect that this could be causing our problems, but I don't understand where this "internal links" number is coming from. Links to our competitor's homepage appear in the navigation bar on every single one of their product pages (which they have about 500 of), yet your report only claims that they have 30 links. The only link to the homepage appears in the site's main navigation bar (which obviously appears on every product page - exactly as it does on our competitors' sites). Additionally, almost every other page on our site apparently has 0 "internal links" and 0 page authority. Is this a problem with Moz's crawl test tool, or is our site actually at fault? The above has been asked directly to Moz staff, but I haven't had a reply. I'd hugely appreciate any words of wisdom from the community. Many thanks in advance. Nick
Moz Bar | | nick45010 -
Should I exclude prepositions in tracked keywords of moz analytics?
I'm new to Moz. Just set up my trial campaign, and it had suggested many keywords. Many of the phrases that were suggested do not contain prepositions. For example, instead of something like "sporting good stores in Chicago" it suggested "sporting good stores Chicago" Today, I looked at the on-page optimization suggestions, which are (of course) suggesting that I remove prepositions from my page to rank well. Well, as you know, that is unnatural to the reader. But I suspect people are searching in higher volume, leaving the prepositions out. I know that if I were to search for a sporting goods store in Chicago, I would probably leave out "in." What should I do? Should I remove all the suggested keywords, and make them readable (which people are less like using in their search?) Do I go back to all my pages and try to optimize it for a keyword that is natural, but does not include a preposition (such as Chicago sporting goods stores) or should I be doing something else?
Moz Bar | | osaka731 -
URL inaccessible for On Page Grader
I am trying to use the on page grader however it is not working with my website. The URL is as follows: https://capbeast.com. I have been trying to read in older posts to see if https is now supported or not but have not found anything. I know there is no robots.txt issue as I am able to run the crawl test on our website fine. Is the issue on my end in regards to configuration or is due to DDos attacks? Any help would be appreciated. Thanks
Moz Bar | | MisterStitches0 -
Moz Local | Download Template
Dear Moz I've received your email about Moz Local. A fantastic tool but it does not allow you to download a template. Clicking 'Download this template' simply reloads the page. I am testing it under incognito mode of Chrome with no add-ons Thank you!
Moz Bar | | Bio-RadAbs0 -
Duplicate page content/page titles on tages
Hello everyone. New to the community and loving it already. Question, I am receiving an error of 6 pages with duplicate content and page titles. A majority of these are tag pages. Should I be worried about these? IN the column listed duplicate urls it is listing 0 ( screen shot: http://screencast.com/t/azvuVk0ucWt) Are these tags a problem? Will SEO be hurt because of this? What are TAG pages? Actually pages, categories, should I eliminate these?
Moz Bar | | Jasonalanmagic0 -
Moz "Crawl Diagnostics" doesn't respect robots.txt
Hello, I've just had a new website crawled by the Moz bot. It's come back with thousands of errors saying things like: Duplicate content Overly dynamic URLs Duplicate Page Titles The duplicate content & URLs it's found are all blocked in the robots.txt so why am I seeing these errors?
Moz Bar | | Vitalized
Here's an example of some of the robots.txt that blocks things like dynamic URLs and directories (which Moz bot ignored): Disallow: /?mode=
Disallow: /?limit=
Disallow: /?dir=
Disallow: /?p=*&
Disallow: /?SID=
Disallow: /reviews/
Disallow: /home/ Many thanks for any info on this issue.0