WiiLi.org

mik · Joined: 10 Jan 2007 Posts: 26

Is anyone currently investigating possibilities to support gesture recognition? Anyone looking into HMM, ANN etc. or even Frameworks? I would really like to see this feature and can share my findings here.

Mik

SwedishFrog · Posted: Mon Apr 23, 2007 4:32 pm Post subject:

yeah, I've been interested in this for a while now. I want to help make it happen.

I found a general purpose java library called iGesture that looks promising. Its supposed to work with a variety of input devices, so that's a plus. I guess we need to just tie it in with chaos' WiiRemoteJ and hopefully it will work.

I'd also like to see if we can use software like this with a glovepie script.

mik · Joined: 10 Jan 2007 Posts: 26

Nice!
The only promising thing i found is The Gesture and Activity Recognition Toolkit (GART) based on GTK which again is based HTK3. But GART seems to be under rework at the moment. I will have a closer look at iGestures.

mik · Joined: 10 Jan 2007 Posts: 26

Hmmm, at first glance iGestures seems to work for desktop applications.
So it shouldn't be a problem to let it run together with the SensorBar for 2D gestures.

For 3D gestures based on the accelometers GART looks like the better option. One of there samples even works with a bluetooth 3D accelometer. So I will continue to investigate in that direction.

SwedishFrog · Posted: Tue Apr 24, 2007 12:50 pm Post subject:

Good shit bro. I'm gonna start screwing around with these libraries once my pc is running again. (my mobo became one with the force)

TiagoTiago · Joined: 20 Jan 2007 Posts: 710 Location: Brasil

wic · Joined: 26 Mar 2007 Posts: 57 Location: Sweden

I've been looking into using a neural net for gesture recognition with the wiimote. I'm not sure if it's possible to simply input the raw acceleration data into the ANN and expect any sane output. I mean, its not even possible for a human to decipher the acceleration graphs into a motion, let alone a computer...

I have done some tests today using a neural net and, well, the results are so-so. At least I'm very far from LiveMotion quality. Training the net is easy, but recognition is much harder and there are lots of false positives unfortunately.

I'm thinking we need to post-process the acceleration data somehow, but I'm not sure how. Best bet would be to use position data, but that is not possible (as well all know by now). I'm gonna try a moving average to reduce the noise and see if that makes any difference. I read somewhere about using "acceleration deltas", whatever that means.

Any and all tips are welcomed!