You can make it display an image representing the video (like an album art).
If you're asking about actually playing the video on there -- this feature currently does not exist, but could be added .. the main reasons of it not being implemented (yet) are:
-There's normally a video playing in the background of audio_browser.skin
-There's no skin made with anything like that right now.
-Most slower machines will suffer from it because of the time it takes to preload the video each time you click on something -- regardless of the video player you use.