Remove html file extension and 301 redirects

[ulefos]

Hi

Recently I ask for some work done on my website from a company,  but I am not sure what they've done is right.
What I wanted was html file extensions to be removed like
/ash-logs.html to /ash-logs
also the index.html to www.timports.co.uk
I have done a crawl diagnostics and have duplicate page content and 32 page title duplicates. This is so doing my head in please help

This is what is in the .htaccess file

<ifmodule pagespeed_module="">ModPagespeed on
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule>

<ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule>

<ifmodule mod_rewrite.c="">Options +FollowSymLinks -MultiViews</ifmodule>

DirectoryIndex index.html

RewriteEngine On 
 #

Rewrite valid requests on .html files  RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA] 
 #

Return 404 on direct requests against .html files

RewriteCond %{REQUEST_URI} .html$  
RewriteCond %{QUERY_STRING} !rw=1 [NC]
 RewriteRule ^ - [R=404]

AddCharset UTF-8 .html # <filesmatch “.(js|css|html|htm|php|xml|swf|flv|ashx)$”="">#SetOutputFilter DEFLATE #</filesmatch>

<ifmodule mod_expires.c="">ExpiresActive On
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years"
ExpiresByType image/x-icon "access plus 1 years"
ExpiresByType image/jpg "access plus 1 years"
ExpiresByType text/css "access 1 years"
ExpiresByType text/x-javascript "access 1 years"
ExpiresByType application/javascript "access 1 years"
ExpiresByType image/x-icon "access 1 years"</ifmodule>

<files 403.shtml="">order allow,deny allow from all</files>

redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html
redirect 301 /about_us.html http://www.timports.co.uk/about-us.html
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery.html redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery.html
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards.html
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs.html redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy.html redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed.html redirect 301 /payment_info.html http://www.timports.co.uk/payment-info.html

[Tom-Anthony]

This looks good to me, the html pages are 301ing to the non .html versions.

[ulefos]

I think I've done it this is what I have found and added to my htaccess code.

<ifmodule mod_rewrite.c="">   
 Options +FollowSymLinks -MultiViews</ifmodule>

DirectoryIndex index.html

RewriteEngine On    
RewriteBase /

#removing trailing slash
RewriteCond %{REQUEST_FILENAME} !-d    
RewriteRule ^(.*)/$ $1 [R=301,L]

#non www to www
RewriteCond %{HTTP_HOST} !^www.    
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

#html
RewriteCond %{REQUEST_FILENAME} !-f    
RewriteCond %{REQUEST_FILENAME} !-d    
RewriteRule ^([^.]+)$ $1.html [NC,L]

#index redirect 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index.html\ HTTP/    
RewriteRule ^index.html$ http://www.timports.co.uk/ [R=301,L]
RewriteCond %{THE_REQUEST} .html    
RewriteRule ^(.*).html$ /$1 [R=301,L]

[ulefos]

I still have the internal error, thank you for your time in looking at this I will keep trying

[LynnPatchett]

Hi,

htaccess can be a pain and I will admit I usually manage what I am after with a bit of trial and error. Try the following, and if you have problems concentrate on the lines:

RewriteCond %{THE_REQUEST} .html
RewriteRule ^(.*).html$ /$1 [R=301,L]

I have added a redirect for index.html to root, and from non www to www and removed the last .html from the last list of _ to - redirects. Give it a shot, and keep that backup handy just in case. If no go, maybe one of the htaccess experts around can step in and have a look, I am not 100% sure what some of those other rules are doing to be honest!

<ifmodule pagespeed_module="">ModPagespeed on 
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule>

<ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule>

AddCharset UTF-8 .html

<filesmatch ".(js|css|html|htm|php|xml|swf|flv|ashx)$"="">

#SetOutputFilter DEFLATE 
#</filesmatch>

<ifmodule mod_expires.c="">ExpiresActive On 
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years" 
ExpiresByType image/x-icon "access plus 1 years" 
ExpiresByType image/jpg "access plus 1 years" 
ExpiresByType text/css "access 1 years" 
ExpiresByType text/x-javascript "access 1 years" 
ExpiresByType application/javascript "access 1 years" 
ExpiresByType image/x-icon "access 1 years"</ifmodule>

<files 403.shtml="">order allow,deny allow from all</files>

# mod_rewrite On only needed once
RewriteEngine On

301 permanent redirect old underscore.html to new dash urls

redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs 
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood
redirect 301 /about_us.html http://www.timports.co.uk/about-us
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery
redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs
redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy
redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed
redirect 301 /payment_info.html http://www.timports.co.uk/payment-info

301 permanent redirect index.html to folder

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.html?\ HTTP/
RewriteRule ^(([^/]+/))index.html?$ http://www.timports.co.uk/$1 [R=301,L]

301 permanent redirect non-www to www

RewriteCond %{HTTP_HOST} !^(www.timports.co.uk)?$
RewriteRule (.*) http://www.timports.co.uk/$1 [R=301,L]

301 permanent redirect all .html to non .html

RewriteCond %{THE_REQUEST} .html
RewriteRule ^(.*).html$ /$1 [R=301,L]

[ulefos]

thanks Lyn, but that gave an 500 internal error, back up worked though

[LynnPatchett]

Hi,

I think you will only need this bit:

#301 from example.com/page.html to example.com/page
RewriteCond%{THE_REQUEST}^[A-Z]{3,9}\ /..html\ HTTP/
RewriteRule^(.).html$ /$1 [R=301,L]

And you would replace this bit below with the above:

Rewrite valid requests on .html files  RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA] 
 #

Return 404 on direct requests against .html files

RewriteCond %{REQUEST_URI} .html$  
RewriteCond %{QUERY_STRING} !rw=1 [NC] 
 RewriteRule ^ - [R=404]

But leave the  at the end of that section.

htaccess files can be a bit picky, so be sure to keep a backup so you can quickly undo something if it is not working!

[ulefos]

Ok have got links to work again with old code, going to try this

#example.com/page will display the contents of example.com/page.html RewriteCond%{REQUEST_FILENAME}!-f RewriteCond%{REQUEST_FILENAME}!-d RewriteCond%{REQUEST_FILENAME}.html -f RewriteRule^(.+)$ $1.html [L,QSA] #301 from example.com/page.html to example.com/page RewriteCond%{THE_REQUEST}^[A-Z]{3,9}\ /..html\ HTTP/ RewriteRule^(.).html$ /$1 [R=301,L]

where would I put this code in relation to what I already have in my htaccess file

[ulefos]

Thanks you for your reply, I have looked at the links you provided and tried replacing this RewriteEngine On #

Rewrite valid requests on .html files RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]

Return 404 on direct requests against .html files

RewriteCond %{REQUEST_URI} .html$  
RewriteCond %{QUERY_STRING} !rw=1 [NC]
RewriteRule ^ - [R=404]

with this, but it didn't work or I did something wrong. #example.com/page will display the contents of example.com/page.html RewriteCond%{REQUEST_FILENAME}!-f RewriteCond%{REQUEST_FILENAME}!-d RewriteCond%{REQUEST_FILENAME}.html -f RewriteRule^(.+)$ $1.html [L,QSA] #301 from example.com/page.html to example.com/page RewriteCond%{THE_REQUEST}^[A-Z]{3,9}\ /..html\ HTTP/ RewriteRule^(.).html$ /$1 [R=301,L]

Now www.timports.co.uk  says this page cant be displayed so I tried to put it back to the previous  .htaccess and still no links working

I am so stuck

[LynnPatchett]

Hi,

Indeed there seems to be an issue with your redirects since the .html versions are still available on your site. Two things to check in the first instance:

1. The redirect line for the .html to non .html versions:

Rewrite valid requests on .html files  RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]

I am not sure if this will work the way you want it. First of all a # at the beginning of this line means it is a comment and not processed so you seem to have the RewriteCond part of the statement as a comment (maybe this is just the forum formatting it wrong, but good to check).
You can check some other solutions for redirecting .html to non .html here: http://stackoverflow.com/questions/5730092/how-to-remove-html-from-url

2. At the bottom of the file you have a bunch of 301 redirects like this:

redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html

Which are working as expected redirecting underscored urls to urls with dashes. But they are also redirecting to the .html version which means you will be getting into double redirects which is pointless in your case. Once you have the non .html redirects working as expected you should adjust these 301s to go to the non .html version like so:

redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood

Hope that helps!