From: daver%sunspot.ssl.Berkeley.EDU@ucbvax.Berkeley.EDU (David Ray)
Subject: Re: APPS: Gesture recognition using simple statistics (DataGlove app.)
Date: 2 Dec 1992 22:36:29 GMT
Organization: Space Sciences Lab, UC Berkeley



In article <1992Dec2.082956.24747@u.washington.edu>
gbnewby@uxh.cso.uiuc.edu (Gregory B. Newby) writes:

>I wanted to share the results with my virtual friends.  This is a
>program that uses a simple statistical method for match a given hand
>position against a 'library collection" of known positions, using the
>VPL DataGlove.
>Someone please tell me if *everyone* already knows this.

I have never heard of it done with the VPL glove, but there are people
who have done similar things with Nintendo Power Gloves.

>The program has a "train" mode and a "recognize" mode. 
>
>I tested with about 12 gestures....  point, fist, flat, bird, horns,
>pinkie, a-ok, thumb-up, snap, etc. THE PROGRAM NEVER MISSED!

Sounds a lot like Mike Lee's neural network program for Max.  I have
done a lot of work using the Power Glove with Max, and I recently
combined that with the neural network program. It can be programmed to
recognize gestures -- i.e. non-time-dependent combinations of finger
bend and hand position. The recognition can then be programmed to
trigger various things with MIDI.

>In a real-world application, we would be trying to pull gestures from
>an ongoing stream.  There's no clearly-defined point at which a
>"gesture" exists (as opposed to just hanging around, or starting to
>make a gesture).  Furthermore, gestures happen across time.  I've only
>taken a slice, and not looked at more time-wise gestures, such as
>"jabbing finger" or "hitching a ride."

Yes, it seems to me that this could be done in principle but you would
need an extremely fast CPU to process the whole neural network
sufficiently fast to resolve the temporal changes.  

>This week, I'll see how my program does with the American Sign 
>Language Manual Alphabet.

There is a guy named Mark Coniglio who has developed his own glove and
software for recognizing finger positions, and which outputs MIDI
data.  He then programmed a synthesizer/sampler with about 100 words
which can each be triggered by a specific MIDI message. Basically he
can make a hand position to trigger the sound of spoken words from the
sampler. He gave a demonstration where he could"speak" whole sentences
by using hand gestures.

>Will keep y'all posted.  BTW: this program runs on the SGI SkyWriter
>system at NCSA.  The part described above is written in Fortran.  The
>DataGlove interface stuff is in C.

I do my PowerGlove/Max/Neural-network programming on a stock Mac LCII.
:-)

Dave