URLS appearing twice in Moz crawl
-
I have asked this question before and got a Moz response to which i replied but no reply after that.
Hi,
We have noticed in our moz crawl that urls are appearing twice so urls like this - http://www.recyclingbins.co.uk/about/ www.recyclingbins.co.uk/about/
Thought it may be possible rel=canonical issue as can find URL's but no linking URL's to the pages.
Does anyone have any ideas?
Thank you
Jon
I did the crawl test and they were not there
-
Hey Jon!
There are only two ways for these URLs to be picked. Relative links in the source code or malformed sitemaps if the <loc>tags are not on the same line as the URL</loc>
Incorrect:
<loc>www.domain.com</loc>
correct:
<loc>www.domain.com</loc>
If the linking page or referral URL is the sitemap, look in the source for the tags. If they are from another linking page, look for hyperlinks missing http://
-
Hi Jon,
Just to clarify is your report showing the domains with http and without, or with www. and without as the duplicate content?
i have seen this issue on a couple of my sites where it is reporting with and without www. and have just implemented the rel canonical to point towards one or the other as i am very confident that this will resolve the issue.
I'll let you know the outcome.
Regards,
Ben
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz can't crawl my new website?
We had a new website go live at the end of April - I keep requesting crawl tests but I get this in the excel copy... URL Title Tag
Moz Bar | | RayflexGroup
http://www.pvc-strip.co.uk 602 : Page redirects to a URL outside the scope of this campaign. I always list the website as https://... but the crawl always returns the http:// version. Not sure what I can do to make sure the website can be crawled?0 -
External Links not showing in MOZ?
Hello:) I have a ton of external links on my site, but there not showing on MOZ? The reason i'm slightly concerned is because my top competitor has 80 and I show as only having 3? Does any one know why this would be? Also, my traffic on MOZ is a lot lower then my traffic report on Google Analytics? Is that normal, & which should I trust more? Here's my Site: Www.ThumannAgency.com Thank you in advance:)
Moz Bar | | MissThumann0 -
Crawl test csv has lost its formatting??
All the columns/heading merged into column A. Anyone else noticed this over the past few days?
Moz Bar | | Moving-Web-SEO-Auckland0 -
Why does the Moz Tool Bar show code as HTML text for my site?
When I run my site in the Moz Tool bar, all of the page elements are read correctly except for HTML text. Sample page: http://www.lifeionizers.com/blog/alkaline-water/electrolyzed-reduced-water Instead of the text on that page, the tool shows Javascript code: /* With a very high character count for text (14,247).
Moz Bar | | karasd0 -
Moz On-page is not working
My on-page is not working....I do have 3 keywords in the 1º position of Google.pt and the Moz is not reporting nothing....bug?
Moz Bar | | Popbox0 -
Is Moz Analytics still available to all MozCon attendees?
I thought all Mozcon attendees were going to get access to Moz Analytics even after the convention was over. I can't seem to find were to access it from my campaigns manager dashboard and the Analytics landing page says im on queue to get access.
Moz Bar | | AmberHanson0 -
New website launched 3 weeks ago but not sure what Moz Analysis is telling me?
Hi 3 weeks ago I re-branded and completely changed the proposition on www.over50choices.co.uk as the URL is appropriate to the proposition. And whilst it looks good I am unsure what the Moz Tools are telling me, other than it doesnt look good? There should be about 163 pages, but Moz reports over 500 due to i think so many 301's? Plus most pages under the HTTP Status reports No Data! Most of the the 404's are from the old site. Webmaster Tools say 163 pages submitted but only 2 indexed, but on site:search it says about 117 but 40+ of these are the old site. It looks a mess...! Sleepless nights have started - any thoughts!? Thanks Ash
Moz Bar | | AshShep10 -
Screaming Frog, Moz and other crawlers
Hi Ignorant question, but is it possible to use Screaming Frog or the Moz crawler or any other reputable crawler for a site still in development i.e. it is yet to be indexed? If so, could someone provide some quick instructions on how this can be done. Thanks in advance for any support. Neil
Moz Bar | | mccormackmorrison0