Until yesterday I wasn't aware of the fact that it is possible to delete a team project. It's not possible using the UI though, but through a command-line utility. The util is called TFSDeleteProject.exe and is located in <drive >\Program Files\Microsoft Visual Studio 8\Common7\IDE.
Usage: TFSDeleteproject [/q] [/force] [/server:servername] team project name (Use quotation marks if the team project name contains spaces)
On the internet I read an article that you should not delete a team project when there are still files checked out...but I didn't test that yet.
You can expect (in a happy scenario) output like this:
Deleting from Build ...
Done
Deleting from Work Item Tracking ...
Done
Deleting from Version Control ...
Done
Deleting Report Server files ...
Done
Deleting SharePoint site ...
Done
Deleting from Team Foundation Core ...
Done
Microsofts documentation about the tool can be found here.