Here are a dozen tasks you should consider before going "Live" with your website. These tasks will increase security and speed-up performance of your store.
Enable HTTP Compression
Optimize your bandwidth by GZipping static and dynamic content. See this blog for more information.
Add an SSL Certificate
Customers look for the "closed lock" on your site when they want to checkout. A SSL Certificate is not only good business, for many Gateways it's required. The brand of certificate rarely matters anymore, so don't get talked into a $300/yr certificate when a $30 one will do the same thing.
Turn off Debug Mode
Edit your web.config file, and search for "debug". Make sure you set it to false (debug="false"). This will improve your website load times, and overall performance.
Rename Admin folder
"Admin" is easy to guess, so rename your Administration site. Then set the AdminDir AppConfig to the name of the new folder, so the store will know where to find it.
Lock-down your Administration site
Require a Windows Authenticated login for your admin folder. You can request this via your hosting company.
Re-compress product images
You can maximize your bandwidth, and even double your throughput by heavily compressing your product images. This blog has more information.
Set-up redirects for non-www requests
If you want to make sure *all* requests go to your www-site instead of your non-www site, setup a redirect in IIS. Here's how.
Make sure that you're not excessively-blocking search engine spiders. Consider the images folder... do you want your product images to be searchable? Have you added any custom pages that you don't want indexed?
Open the web.config file, and search for "MachineKey". Every site should have a unique set of validation and decryption keys. You can generate your own here.
Set a memory limit for your AppPool
Here's a good whitepaper (somewhat dated, but still accurate) on how to configure the AppPool for your web application.
Remove unused files from the site
Any files that your site doesn't use should be removed for your site (don't delete the file, just move it off the site in case you need it later). After removing any file, be sure to test your site to ensure you didn't break any functionality. Here's a list of files that are often removed:
Set a custom error page and enable custom errors
In the web.config file there is a "customErrors" element. When you are convinced that your site is working properly you should set customErrors to On and create a static .htm page to be shown to your customers when an error occurs on the site. This will A) prevent your customers from seeing an ugly .NET exception if an error does occur, and B) will prevent your site from disclosing potentially sensitive information about your hosting environment such as the database name (in the case of a SQL error) or disk path of your site.
Hopefully, your site will become better performing and be more secure by performing these 13 simple tasks.
Note By Dan: I added custom errors to Jon's list. This is an important, but often missed step when going live.