Information, tips and myths - No.14 - Closing Windows windows
You can close all windows at once by pressing and keeping down alt and tab at the same time, then pressing and keeping down delete.
You can close individual ones by pressing and releasing the Windows and tab keys at the same time, then click the windows you wish to close.