How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
.htaccess Redirect 301 issues
I have completely rewritten my web site, adding structure to the file directories. Subsequently added was Redirect information within the .htaccess file. The following example ...
Technical SEO | | Cyberace
Redirect 301 /armaflex.html http://www.just-insulation.com/002-brands/armaflex.html
Returns this response in the URL bar of ...
http://www.just-insulation.com/002-brands/armaflex.html?file=armaflex
I am at a loss to understand why the suffix "?file=armaflex" is added The following code is inserted at the top of the file ...
RewriteEngine On redirect html pages to the root domain RewriteRule ^index.html$ / [NC,R,L] Force www. prefix in URLs and redirect non-www to www RewriteCond %{http_host} ^just-insulation.com [NC]
RewriteRule ^(.*)$ http://www.just-insulation.com/ [R=301,NC] Any advice would be most welcome.0 -
What is the best way to change tons of 302 for 301...
I'm doing an audit for a new client and their website has tons of 302... (more than 1000)... What is the best/fast way to change all the 302 for 301...?
Technical SEO | | Felip30 -
How to know what pages are 301 redirecting to me?
Hi! It is easy to know if somebody is spam linking your website, looking i.e., looking at open site explorer to analyse the links profile. But, is it possible to know if a competitor of mine is redirecting a bad domain to main with a 301 redirect, thus transfering any bad SEO reputation to me? Best Regards, Daniel
Technical SEO | | te_c0 -
Is there ever a time when 301 redirects aren't possible?
I have been told that 301 redirects are always possible. I've been told that it's a very time consuming process so developers at times will say that it's not possible. Is there ever a time when it is not impossible? Perhaps using a specific server? I know it's do-able in Apache which is the server that is in question. Would it be impossible if someone were using a templated type set of websites & if they made changes on one website it would make changes across all websites? *Edit "due to a server configuration 301 redirects aren't possible" Thanks so much for any help or answers you can provide.
Technical SEO | | DCochrane0 -
What is the best way to fix legacy overly-nested URLs?
Hi everyone, Due to some really poor decisions I made back when I started my site several years ago, I'm lumbered with several hundred pages that have overly-nested URLs. For example: /theme-parks/uk-theme-parks/alton-towers/attractions/enterprise I'd prefer these to feature at most three layers of nesting, for example: /reviews/alton-towers/enterprise Is there a good approach for achieving this, or is it best just to accept the legacy URLs as an unfixable problem, and make sure that future content follows the new structure? I can easily knock together a script to update the aliases for the existing content, but I'm concerned about having hundreds of 301 redirects (could this be achieved with a single regular express in .htaccess, for example?). Any guidance appreciated. Thanks, Nick
Technical SEO | | ThemeParkTourist0 -
Setup 301 Redirects
I have been asked to transfer a clients old domain over to a new domain with a new site. All of the inbound links basically go to the homepage, and the few links that dont go to the homepage on the old site, might as well be redirected to the homepage on the new site. I'm wondering is there a "catch all" sort of redirect such as www.oldsite.com/* redirects to newsite.com. So any redirects we havent set up will automatically go to the new site homepage? And secondly, whats the easiest way to the redirects up? Can I just add it as a parked domain or addon domain in cpanel, and do the redirects in there? Or does it needs its own hosting for the old domain with its own htaccess file? Any help appreciated! 🙂
Technical SEO | | timscullin0 -
Redirect or not to redirect
We are rebuilding a website and try to get rid of errors. The content remains exactly the same but we correct the code and make it load faster. The site has quite many backlinks and I can't decide whether to remove .html endings from the urls and 301 redirect to the new ones or leave them with the older ending. If I remove the endings how much of the link juice will be passed? Anyone any idea?
Technical SEO | | sesertin0 -
Any issues with lots of pages issuing 301 redirects?
Hi all, I'm working on a site redesign and it is possible the new site could issue a lot of 301 redirects as we may migrate from one forum solution to another. Is there any issue with crawlers getting a lot of 301 redirects from a site? Thanks Nick
Technical SEO | | nickswan0