Crawl Diagnostics finding pages that dont exist. Will Rel Canon Help?
-
I have recently set up a campaign for www.completeoffice.co.uk. Im the in-house developer there. When the crawl diagnostics completed, i went to check the results, and to my surprise, it had well over 100 missing or empty title tags. I then clicked it to see what pages, and nearly all the pages it say have missing or empty title tags, DO NOT EXIST.
This has really confused me and need help figuring out how to solve this. Can anyone help?
Attached image is a screen shot of some of the links it showed me on crawl diagnostics, nearly all of these do not exist.
Will the relation Canonical tag in the head section of the actual pages help?
For example,
The actual page that exist is: www.completeoffice.co.uk/Products.php
Whereas, when crawled it actually showed www.completeoffice.co.uk/Products/Products.php
Will have the rel can tag in the header of the real products.php solve this?
-
Wow, what a great answer. Thanks bunch, actually helped alot
-
I performed a quick crawl of your website and immediately found part of the problem - it looks like your HTML sitemaps is still linking to the pages in question.
With that in mind, I would NOT recommend using the canonical tag here. Instead, I would check whether or not these pages have links pointing to them. If they don't, then just change the URL in your sitemaps to the correct location. However, if you do have inbound links to these pages then implement a 301 Permanent Redirect to the appropriate page.
I didn't conduct a comprehensive crawl but if you'd like your own data then please check out Xenu Link Sleuth. Also, I think inside the SEOmoz PRO campaign you can click on each individual URL and it'll show you the referring links as well.
Hope that helps and good luck!
UPDATE: Actually, the problem looks like it's coming from your source code. You have your old URLs still located on the page under
. A parallel issue is 'products.php' and 'Products.php' both resolve completely different pages so you might want to look into 301 redirecting them.
-
Hi,
Yes, canonical tag will word great.
But if people share this page and currently there are small amount of links to www.completeoffice.co.uk/Products/Products.php, i recommend to create 301 redirect for better URL look.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate pages coming from links from the login page - what should we do about them?
This is a follow on to an earlier question which was well answered by Dirk Ceuppens regarding abnormal crawl issues. We are seeing that the issues relating to Duplicate Pages are coming from links from the login page which shows information about where the user was redirected from. For example, if the visitor is not logged on and wishes to wish-list an item, they will be redirected to the login page, with the item code and intended action in the url; which can then continue on to the desired page once logged on. The MOZ crawler is seeing these pages as having Duplicated Content whilst they are all the same apart from a piece of information in the URL. Should we be blocking these duplications? Are they a risk to us? What should we be doing? Many thanks, Sarah
Moz Pro | | Mutatio_Digital0 -
Crawl Diagnostics
My site was crawled last night and found 10,000 errors due to a Robot.txt change implemented last week in between Moz crawls. This is obviously very bad so we have corrected it this morning. We do not want to wait until next Monday (6 days) to see if the fix has worked. How do we force a Moz crawl now? Thanks
Moz Pro | | Studio330 -
Campaign Crawl
I have a site with 8036 pages in my sitemap index. But the MozBot only Crawled 2169 pages. It's been several months and each week it crawls roughly the same number of pages. Any idea why I'm not getting fully crawled?
Moz Pro | | JMFieldMarketing0 -
Unable to crawl pages
Hi, I am trying to set up a campaign for our website - www.salvationarmy.org.au however, I can't seem to get a scan of more than three pages. I have tried the following: www.salvationarmy.org.au (only 2 pages) www.salvationarmy.org.au/home (only 1 page) salvationarmy.org.au (only 3 pages) There is a geo IP redirect on www.salvationarmy.org.au but the second domain listed above should resolve the full site. I'm a newbie to SEOmoz so any help would be appreciated! Thanks, Mel
Moz Pro | | KingPings0 -
Missing Page Titles On The Comptetive Link Comparison Page
Hello, When I do a Link Analysis using the SEOmoz tools I have noticed that most of the pages listed on the Top Pages tab show [No Data] for page title. Any idea why that could be? The page source of those pages have one and only one <title>tag.</p> <p>Thanks!</p></title>
Moz Pro | | andersvin0 -
Where can I find a list of all Page Authority Metrics?
Where can I find a list of all Page Authority Metrics? I'm looking for an exhaustive list of all link metrics that calculate Page Authority? Thanks
Moz Pro | | poolguy0 -
Crawling One Page
I set up a profile for a site with many pages, opting for setting up as a root directory. When SEOMoz crawled, they only found one page. Any ideas for why this would be? Thanks!
Moz Pro | | Group160 -
Crawl test. Bot crawled only 200 or so links when it should have crawled thousands
Hi everyone, I just recieved my crawl test report and its only given me 200 or so URL's when my site has thousands, any thoughts?
Moz Pro | | Ev840