There isn't a direct penalty for having rel="canonical" tags on every page, no, as long as you are correctly utilizing them (i.e. don't set the href of the tag to an invalid or non-existent URL). If there is even the possibility of duplicate content on your website, it is best to use canonical tags.
For websites serving straight HTML files, both _http://www.example.com/index.html_ and _http://www.example.com/_ likely serve the same content.
If you use a framework like ASP.NET MVC, it would by default return duplicate content for both _http://www.example.com/_ and _http://www.example.com/Home/Index_.
Choose one or the other and set your canonical tag to that:
(note: the trailing slash is optional - just be consistent with including it or not)
