HariesDesign.com - The Joomla site has a checklist on security that is excellent. It covers installation, setup, and ongoing administrative matters. I would say it is required reading for anyone who has a Joomla website.
A lot of the suggestions on this page may seem a bit technical for some people, but they are worth persisting with. However, there are some suggestions even the least technically-minded person can use.
These include:
use the latest, stable version of Joomla - keeping in mind that it
sometimes takes a little while for the plugins you may need to catch up
(for example, this could be an issue with Joomla 1.5).
be careful in your choice of web host. Web hosts themselves can
introduce security vulnerabilities, that can be easily avoided. There
is link to a list of recommended hosts on the page above.
delete all the left over installation files - particularly if you’re not using fantastico via cpanel to install joomla
move your configuration file outside the Public_html folder
change the default user name for your admin user - simple but effective!
PHP5 is more secure, but the ability to use this will depend on whether your host is using it.
Don’t use PHP safe mode
Set the Register Globals Emulation off
Delete all design templates you’re not using for your site.