Ripped straight from their website:

What is Visual Task Tips?

Visual Task Tips is a lightweight shell enhancement utility. It provides thumbnail preview image for each task in the Windows Taskbar, as seen in the upcoming Microsoft Windows Vista.

NOTE: Windows XP is required to run Visual Task Tips.

Simple

Visual Task Tips is a simple utility. Just download and install it.

Useful

By using Visual Task Tips you can significantly increase your productivity.

Free

Visual Task Tips is absolutely free for non-commercial use.

I just installed it.  Works great, however some apps aren’t showing thumbnails for some reason.