How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 Redirects Showing As 307 Redirects
Hi, Our clients are adamant that they have set up 301 permanent redirects on their websites, but when we check using Screaming Frog and various online HTTP status code checkers they are showing as 307 temporary redirects. Examples;
Technical SEO | | Webpresence
http://www.lifestylelifts.co.uk/home-lifts/
http://www.terrylifts.co.uk/ Again, the client says they are seeing 301 redirects. Why are we seeing 307's? Who is right? Very puzzling, any theories would be very much appreciated 🙂 Thanks in advance. Lee.0 -
301 redirect syntax for htaccess
I'm working on some htaccess redirects for a few stray pages and have come across a few different varieties of 301s that are confusing me a bit....Most sources suggest: Redirect 301 /pageA.html http://www.site.com/pageB.html or using some combination of: RewriteRule + RewriteCond + RegEx I've also found examples of: RedirectPermanent /pageA.html http://www.site.com/pageB.html I'm confused because our current htaccess file has quite a few (working) redirects that look like this: Redirect permanent /pageA.html http://www.site.com/pageB.html This syntax seems to work, but I'm yet to find another Redirect permanent in the wild, only examples of Redirect 301 or RedirectPermanent Is there any difference between these? Would I benefit at all from replacing Redirect permanent with Redirect 301?
Technical SEO | | SamKlep1 -
Console: Change Of Address, 301 Redirects Step Not Working
Hi everyone, We just made a switch to a new domain based on a rebrand, and we are currently directing users who navigate to the homepage of the old site to an interim page on our new site that reads companyname.com/companyname, that then redirects to the actual homepage of the new site, companyname.com. The redirect page's purpose is to tell users about the brand name change before showing them the new site. The challenge is that in the second step of the verification process for the address change, it reads that our domain name request does not correspond because the homepage is directing to this interim page. Is there a way for us to continue to direct people to companyname.com/companyname and have GSC verify that the 301 redirects work properly? Ideally we would not want to direct people from the homepage of the old site to the homepage of the new site. Thank you in advance for your help. Sincerely, Chase VHSpF
Technical SEO | | commcreative0 -
Best & easiest way to 301 redirect on IIS
Hi all, What is the best and easiest way to 301 redirect URLs on IIS server? I got access to the FTP and WordPress back office, but no access to the server admin. Is there an easy way to create 301 redirect without having to always annoy the tech in charge of the server? Thanks!
Technical SEO | | 2MSens0 -
301 Redirect - Technical Question
I have recently updated a site and for the url's that had changed or were not transferring I set up 301 redirects in the htaccess file as follows This one works - Redirect 301 /industry-sectors http://www.tornadowire.co.uk/fencing But this one doesn't - Redirect 301 /industry-sectors/equine http://www.tornadowire.co.uk/fencing/application/equestrian/ What it does is change the url to this instead http://www.tornadowire.co.uk/fencing/equine ..... which returns a 404 page not found error The server is nginx based server and we have moved from a joomal platform to a wordpress platform I would be grateful for any ideas
Technical SEO | | paulie650 -
What should be use 301 or 302 redirection for 404 pages
Please suggest which redirection we should use for 404 pages- 301 or 302. If you can elaborate it with reason then it will be highly appreciated.
Technical SEO | | koamit0 -
Setting up a 301 redirect from expired webpages
Hi Guys, We have recently created a new website for one of our clients and replaced their old website on the same domain. One problem that we are having is that all of the old pages are indexed within Google (1000s) and are just getting sent to our custom 404 page. We are finding that there is an large bounce rate from this and also, I am worried from an SEO point of view that the site could lose rank positioning through the number of crawl errors that Google is getting. Want I want is to set up a 301 redirect from these pages to go to the 'our brands' page. The reason for this is that the majority of the old URLs linked to individual product pages, and one thing to note is that they are all .asp pages. Is there a way of setting up a rule in the htaccess file (or another way) to say that all webpages that end with the suffix of .asp will be 301 redirected to the our brands' page? (there is no .asp pages on the new site as it is all done in php). If so, I would love it if someone could post the code snippet. Thanks in advance guys and if you have any other ideas then be my guest to suggest 🙂 Matt.
Technical SEO | | MatthewBarby0 -
301 Redirect Properly To Keep the Juice
I have a bunch of WP Blogs and was thinking of taking all linkjuice from these to my main money site. The most of the other WP Blogs is hosted at godaddy.com (domain and site) and I know they have a URL Redirects page in site manager but I`m not sure this is the right way to go. Also I wonder some of these sites have hundreds of blogposts there is no way I can "re-create" those on the money site but I am sure that is not a must-thing to do in order to keep the "juice" right or wrong? Last but not least, I was wondering if you think it would be best to redirect the sites to relevant pages on money sites. For instance if i had a domain called cheap-ties.com with 100 blogposts about this and on money site a webshop with a category called ties, should redirect to this or to main domain or doesnt it matter?
Technical SEO | | fAgBxa8b0