How to handle crawl diagnostic errors for the same url. /products & /products/
-
I have copied on of the errors out of the crawl diagnostics report. Both /products and /products/ are returning an error, and both have pretty good domain authority so I feel like its hurting my site that these show up this way. Both urls create the same page, should I just setup a 301 on the /products with no slash or will that cause more harm... I am using the MODx cms system and that could have something to do with it.
|
Products | Datalight
http://www.datalight.com/products 1 37 5 Products | Datalight
http://www.datalight.com/products/ | 1 | 30 | 1 |
-
Hey TJ,
For your example there would be an easy solution:
- 301 redirect (it will take up a lot of time but, then you will be redirecting both users and search engines to one of the version)
- Canonical (it will not redirect the user, instead it will tell Search Engines that the page that they are looking for is the one specified in the canonical link)
A more visual explanation of this issue can be found on a great post of Dr. Pete. Check it out: http://www.seomoz.org/blog/an-seos-guide-to-http-status-codes
I hope that helped,
Istvan
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Before Migration/after(www/non-www/http/https) - Good concentration needed :p
Hi all, Im confusing between those www's and http's. If i go to searchbar (chrome) and ENTER: www.mywebsite.nl, It changes to https://www.mywebsite.nl
Moz Pro | | Dreamgame2016
( with www, and https:// not used) / Its OK next: typing in searchbar and enter: mywebsite.nl, It changes to https://mywebsite.nl (without www and https:// ) / OK Next: www.mywebsite.nl, it stay the same, just https:// added: https://mywebsite.nl (used with https://) / OK Now its comes: If I do it again without http**(s)://mywebsite.nl, **It changes to https://www.mywebsite.nl/?SID=bccbuhvi1cf53r188bpvskn597 / NOT OK 😛 In google search console (webmastertool) I gave property for the https://mywebsite.nl and https://www.mywebsite.nl Each of the website, Im seeying data clicks/ volume keywords etc, so both of them functionating By search console: https://www.mywebsite.nl (With www) I see crawlfaults/errors: 1633 (the url has not linked existing page) I see again: "?SID=..." after urls, example: mywebsite.nl/blabla/?SID=m07ev6lliefbf0tfhe4kf0ih54 By search console - other website: https://mywebsite.nl **(none-www) **you see two crawlfaults/errors! Bad influance for my SEO, because of no existed pages, bad urls and dubble content. Bye bye keywords! Lets analyze/crawl with Moz tool ofcourse ^^: Pages with High Priority Issues: | 2646 | Duplicate Page Content |
| 14 | 4XX Client Error |
| 3 | Crawl Attempt Error |
| 1 | Title Missing or Empty | Medium priority: | 9618 | Temporary Redirect |
| 2688 | Duplicate Page Title |
| 13 | Title Element is Too Long |
| 1 | Missing Meta Description Tag | After seeying this results what is the best option (no losing link-juice)? redirect 301? www to none-www (https://) ? Shortly I am going to change my domain provider and the website template in magento. After that I am going to focus on the SEO implementation. First, I have to solve this problem. Who can give me an advice for this situation? Regarding, Newbee0 -
Multiple Countries, Same Language: Receiving Duplicate Page & Content Errors
Hello! I have a site that serves three English-speaking countries, and is using subfolders for each country version: United Kingdom: https://site.com/uk/ Canada: https://site.com/ca/ United States & other English-speaking countries: https://site.com/en/ The site displayed is dependent on where the user is located, and users can also change the country version by using a drop-down flag navigation element in the navigation bar. If a user switches versions using the flag, the first URL of the new language version includes a language parameter in the URL, like: https://site.com/uk/blog?language=en-gb In the Moz crawl diagnostics report, this site is getting dinged for lots of duplicate content because the crawler is finding both versions of each country's site, with and without the language parameter. However, the site has rel="canonical" tags set up on both URL versions and none of the URLs containing the "?language=" parameter are getting indexed. So...my questions: 1. Are the Duplicate Title and Content errors found by the Moz crawl diagnostic really an issue? 2. If they are, how can I best clean this up? Additional notes: the site currently has no sitemaps (XML or HTML), and is not yet using the hreflang tag. I intend to create sitemaps for each country version, like: .com/en/sitemap.xml .com/ca/sitemap.xml .com/uk/sitemap.xml I thought about putting a 'nofollow' tag on the flag navigation element, but since no sitemaps are in place I didn't want to accidentally cut off crawler access to alternate versions. Thanks for your help!
Moz Pro | | Allie_Williams0 -
Ajax4SEO and rogerbot crawling
Has anyone had any experience with seo4ajax.com and moz? The idea is that it points a bot to a html version of an ajax page (sounds good) without the need for ugly urls. However, I don't know how this will work with rogerbot and whether moz can crawl this. There's a section to add in specific user agents and I've added "rogerbot". Does anyone know if this will work or not? Otherwise, it's going to create some complications. I can't currently check as the site is in development and the dev version is noindexed currently. Thanks!
Moz Pro | | LeahHutcheon0 -
Crawl Diagnostics Shows thousands of 302's from a single url. I'm confused
Hi guys I just ran my first campaign and the crawl diagnostics are showing some results I'm unfamiliar with.. In the warnings section it shows 2,838 redirects.. this is where I want to focus. When I click here it shows 5 redirects per page. When I go to click on page 2, or next page, or any other page than page 1 for that matter... this is where things get confusing. Nothing shows. Downloading the csv reveals that 2,834 of these are all showing: URL: http://www.mydomain.com/401/login.php url: http://www.mydomain.com/401/login.php referrer: http://www.mydomain.com/401/login.php location_header: http://www.mydomain.com/401/login.php I guess I'm just looking for an explanation as to why it's showing so many to the same page and what possible actions can be taken on my part to correct it (if needed). Thanks in advance
Moz Pro | | sethwb0 -
Duplicate page errors
I have 102 duplicate page title errors and 64 duplicate page content errors. They are almost all from the email a friend forms that are on each product of my online store. I looked and the pages are identical except for the product name. Is this a real problem and if so is there a work around or should I see if I can turn off the email a friend option? Thanks for any information you can give me. Cingin Gifts
Moz Pro | | cingingifts0 -
Wild fluctuation in number of pages crawled
I am seeing huge fluctuations in the number of pages discovered the crawl each week. Some weeks the crawl discovers > 10,000 pages and other weeks I am seeing 4-500. So, this week for example I was hoping to see some changes reflected for warnings from last weeks report (which discovered > 10,000 pages). However, the entire crawl this week was 448 pages. The number of pages discovered each week seems to go back and forth between these two extremes. The more accurate count would be nearer the 10,000 mark than the 400 range. Thanks. Mark
Moz Pro | | MarkWill0 -
Campaign 4XX error gives duplicate page URL
I ran the report for my site and had many more 4xx errors than I've had in the past month. I updated my .htaccess to include 301 statements based on Google Webmaster Tools Crawl Errors. Google has been reporting a positive downward trend in my errors, but my SEOmoz campaign has shown a dramatic increase in the 4xx pages. Here is an example of an 4xx URL page: http://www.maximphotostudio.net/engagements/266/inniswood_park_engagements/http:%2F%2Fwww.maximphotostudio.net%2Fengagements%2F266%2Finniswood_park_engagements%2F This is strange because URL: http://www.maximphotostudio.net/engagements/266/inniswood_park_engagements/ is valid and works great, but then there is a duplicate entry with %2F representing forward slashes and 2 http statements in each link. What is the reason for this?
Moz Pro | | maximphotostudio1 -
Crawl Diagnostics and missing meta tags on noindex blog pages
Hi Guys/Gals We do love the Crawl Diagnostics, but do find the missing meta tags ("Missing Meta Description" Tag in this case) somewhat spammy. We use the "All in One SEO Pack" for our blog and it does stick in noindex,follow (as it should) on the pages that is of no use to us. "2008/04/page/2/" and the likes. Maybe I'm wrong but should the Diagnostics tool not respect the noindex tag and just ignore any warnings, since it should really mean that these pages are NOT included in the search index. Meaning that the other meta tags are really useless. Any thoughts?
Moz Pro | | sfseo0