Ogre, pinch and TAFFI: Gesture Now!

[tuan kuranes]

Hot topic is definitely Human Interface in computer today, as we all saw Nintendo smart move. How Wiimote make low 3D Wii shine over overpowered PS3 is really the best proof.

We tech people often forget how limited Human Interfaces are, how long it takes to master mouse, 105 keys, or even joystik/pad. (but long user cannot forget chronic pain that goes with.)

Wiimote and other acceleration based input are nice and cheap, and are today best choice... but let's have a look at future :

Brain based interfaces are really too premature to deliver real input (EEGG hardly give up to 5 boolean input...)

On the other side, Camera based seems to become the most important research topic... Natural, Human, fast and cheap (30$ webcam) and really ubquitous (all cellphones has camera)

Here's a quick and Ogrish glimpse at it, with papers, code and amazing videos on a Camera based Interface : Thumb And ForeFinger Interface (TAFFI)

Animation for Novice using Ogre,Opencv and Pinching
student webpage
youtube video 1
youtube video 2 (most self-explaining one.)

Another TAFFI implementation paper

All based on the paper "Robust Vision-Based Detection of Pinching for One and Two-Handed" listed here :
Andrew D. Wilson

Still on gesture recognition and 3D :
While on Wilson's page, check the Micromotocross demoms video

On mouse gesture recognition, a really nice code:
Double check also his $1 gesture recognition that does what I did once (oisaddons gesture) but faster, better and cheaper... (source code in javascript and c#)

opencv is beginning the reference, but don't forget GpuCV which gives GPU power to image operators.

Now, really image and video analysis are integrating programming capabilities must-have, like text parsing is, as semantic can be extracted from video/images now. (Sound is also, but really lacks an openSound library... now has to rely on ms speechSDK which is painful... and only works on vista without user install.)

[cybereality]

Cool links. I am very much interested in this field. I think Nintendo made a good move with the motion-controls, but the progress has been marred by lazy devs pumping shovel-ware onto the wii with tacked on controls. This is one of the main reasons alternative HIDs haven't taken any real hold of the market is because buggy or inaccurate software. The hardware and the algorithms are clearly there and have been for years. I guess the iPhone could be considered the one true success-story in that respect. It has both the hardware and the software in an easy to use package.

I agree that current EEG technology may be premature, but I still kinda want the OCZ Neural Impulse Actuator. Who wouldn't?

I think the near-future of alternative HIDs are going to be the depth sensing web-cams like what 3DV and others are releasing later this year. These depth-cams have a real chance of becoming mainstream, regular users are not interested in wearing data-gloves or friggin' EEG helmets. A simple web-cam with no need for controllers is the right option.

[Vectrex]

I love this stuff. So much me and a friend made a camera based music system (poor mans audiopad ) here http://www.similarmusic.com
We actually made an ogreised version but all the fancy 3d graphics and particle effects kind of missed the point