Well, I may be slow of the mark here, but Java rocks. I have finally taken the plung and learnt it, and taking a University course in it, as well as using it at work, so its kind of been fun being thrown in the deep end.
I find that its great for high level stuff, but the common critisism is that performance can suck. I guess this is true since the language encourages you to not worry about the lower level implementation, but with a bit of careful tinkering, you can get java to process live video at 640x480x32 at about 30fps, and throw thousands of meta circles around at the same time for good measure. Although, that is, if you do not use Graphics2D, which is horrendously slow.
A good article on the different methods for rendering graphics, and the basis of pixel buffer rendering is [woops, lost this bookmark!] and here is another. I used the VXP package (broken, try the google cache) that wraps the QuickTime VDIG and JMF and presents a common PixelSource interface. Nice.
Given that, its just a matter of plugging it together to get per-pixel access to a live video stream (or indeed any video source supported by JMF or QT).
Unfortunately, I have been dragged kicking and screaming back to the world of “real” work and studying, so no more playing for me for a while.