How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect Chain Issues - Cannot Fix
Hi All My Moz Crawl for my domain brings up 7 redirect chain errors. In each case, the original source is our old website. Strangely, only 7 of the pages show up as having redirect chains, as there are many more examples. We have added redirects from the old pages via the redirection tool in wordpress. The redirects themselves are working as expected. The redirects are from the old website, so the address http://www.oursite.co.uk/example.html to https:www.oursite.co.uk/new-page/ The chain seems to go from the original site and address to https and then onto the new domain. Screenshot link to the Moz Report https://imgur.com/a/eXRSFcD Any assistance here would be greatly appreciated. The hosts says there is nothing wrong with the.htaccess file, but I'm thinking this must be the issue. Many Thanks
Technical SEO | | ruislip180 -
1000 Pages on old website. What to do with the 301 redirects for this domain?
Hi Moz Community, I have a 301 redirect question... I just acquired an old domain: Totally in my niche Domain is 14 years old Website exists of 1000 pages Great amount of backlinks Website is offline since about 2 weeks Will place a new website online asap with new url structure For the 50 best scoring pages I wrote a new, but fully comparable/related article. I will put a 301 redirect from those old to the new pages. My question: What to do with the 950 other url's? Should I put a 301 redirect to the homepage? Should I forward those pages to the 404 page? Should I divide the 950 url's with a 301 redirect to the 50 new ones? Another solution maybe? Any idea what would be the best solution so we can save as much Google juice as possible? Thanks in advance!
Technical SEO | | snorkel0 -
Remove html file extension and 301 redirects
Hi Recently I ask for some work done on my website from a company, but I am not sure what they've done is right.
Technical SEO | | ulefos
What I wanted was html file extensions to be removed like
/ash-logs.html to /ash-logs
also the index.html to www.timports.co.uk
I have done a crawl diagnostics and have duplicate page content and 32 page title duplicates. This is so doing my head in please help This is what is in the .htaccess file <ifmodule pagespeed_module="">ModPagespeed on
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule> <ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule> <ifmodule mod_rewrite.c="">Options +FollowSymLinks -MultiViews</ifmodule> DirectoryIndex index.html RewriteEngine On
# Rewrite valid requests on .html files RewriteCond %{REQUEST_FILENAME}.html -f RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]
# Return 404 on direct requests against .html files RewriteCond %{REQUEST_URI} .html$
RewriteCond %{QUERY_STRING} !rw=1 [NC]
RewriteRule ^ - [R=404] AddCharset UTF-8 .html # <filesmatch “.(js|css|html|htm|php|xml|swf|flv|ashx)$”="">#SetOutputFilter DEFLATE #</filesmatch> <ifmodule mod_expires.c="">ExpiresActive On
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years"
ExpiresByType image/x-icon "access plus 1 years"
ExpiresByType image/jpg "access plus 1 years"
ExpiresByType text/css "access 1 years"
ExpiresByType text/x-javascript "access 1 years"
ExpiresByType application/javascript "access 1 years"
ExpiresByType image/x-icon "access 1 years"</ifmodule> <files 403.shtml="">order allow,deny allow from all</files> redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html
redirect 301 /about_us.html http://www.timports.co.uk/about-us.html
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery.html redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery.html
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards.html
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs.html redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy.html redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed.html redirect 301 /payment_info.html http://www.timports.co.uk/payment-info.html1 -
Static site to wordpress - avoiding 301 redirects
Moving our static website to wordpress, pages currently end in the .htm extension and for reasons of me having to do all the moving myself and wanting to preserve link equity is there any way I can run the pages with a .htm extension in Wordpress? Tried using a plug-in by Daddy Design but it seems a bit hit and miss at times. I basically need to keep the url's the same as I will not be able to get the vast majority of my links altered to the new pages, plus I am doing this by myself!
Technical SEO | | Jon-C0 -
Where to put 301 redirects in Magento?
I will be changing the URL structure in a magento store from "base/category/subcategory/product" to "base/product" which means i have to make over 1000 URL 301 redirects so our old links still work. Should i put the redirects in a .htaccess file so they stay intact no matter what or should i just put them in with all the other rewrites (in Magento 's URL rewrite manager)? Thanks
Technical SEO | | tilenkrivec0 -
Should I change a 301 redirect?
I recently moved all the content from an old site to a new site on a new domain. I lost a significant amount of traffic as a result. There are 301 redirects for every page on the old site. Generally, these point to the same content as was on the relevant page of the old site. However, the 301 redirect for the homepage on the old site points to the homepage on the new site, not to the content from the old site homepage. I'm wondering whether to change the 301 to point at the content from the old site homepage. Any advice would be much appreciated.
Technical SEO | | seqal0 -
Setup 301 Redirects
I have been asked to transfer a clients old domain over to a new domain with a new site. All of the inbound links basically go to the homepage, and the few links that dont go to the homepage on the old site, might as well be redirected to the homepage on the new site. I'm wondering is there a "catch all" sort of redirect such as www.oldsite.com/* redirects to newsite.com. So any redirects we havent set up will automatically go to the new site homepage? And secondly, whats the easiest way to the redirects up? Can I just add it as a parked domain or addon domain in cpanel, and do the redirects in there? Or does it needs its own hosting for the old domain with its own htaccess file? Any help appreciated! 🙂
Technical SEO | | timscullin0 -
301 Redirects Change?
Hi, Mozzers- I've noticed pages that are getting 301 redirected are staying out there longer. It used to be that you would implement a 301 redirect and then after a couple of months the old page would disappear out of Google's index. Over the last couple of months I've noticed pages lingering, popping up.... What gives? Thanks in advance! LHC
Technical SEO | | lhc670