I’ve used a thumbdrive before and it works very nicely! The only downside is you have physically hand it to them (or send it in the mail) and that’s only practical if you live close.
Since then I’ve started using Dropbox and that is my favorite way to deliver photos to clients. You just upload your photos to a folder then send a link to your clients and they can download the photos without having to create an account or anything.