I have a very small, simple WordPress installation with virtually no customisations, which is automatically backed up nightly (both files & mysql DB).  This is what i do to get it upgraded.  It's mostly here for my reference, but i figured others might find it helpful.

1. Take a MySQL backup just in case and save it on my laptop (using phpMyAdmin, adding the drop table option).
2. Link existing files (saves disk space for files that aren't changed):
   cd /var/www
   cp -al wordpress wordpress-YYYYMMDD
3. Remove wp-admin & wp-includes:
   rm -rf wordpress/wp-admin wordpress/wp-includes
4. Download:
   wget http://wordpress.org/latest.tar.gz -O wordpress.tar.gz
5. Extract:
   tar -xzvf wordpress.tar.gz
6. Run upgrade script:
   http://my-site/wp-admin/upgrade.php