By Janet Fang
Posting in Design
Lined with a dozen sensors, EnableTalk senses the movements of the wearer's hands and fingers and translate the signs into spoken words.
There are an estimated 40 million deaf, mute, or deaf-mute people in the world today. And while many of them can communicate with each other through sign language, there's a linguistic wall between them and people who can speak and write but don't know signs.
A team of Ukrainian researchers called QuadSquad is trying to bring down that barrier. They've designed a system called EnableTalk, which includes a glove that can sense the movements of the wearer's hands and fingers and translate the signs into spoken words.
- Each glove is lined with more than a dozen flex sensors thin strips that detect changes in resistance and can tell when a finger is bent and touch sensors.
- Built-in accelerometers can tell when the hand gestures, and in which direction.
- On the back of the hand lies a controller, the heart of the system that analyzes all these incoming signals and transmits them via Bluetooth to a mobile device.
- A lithium-ion battery charged via USB powers the device (though QuadSquad built in solar cells to provide extra juice).
- Finally, Windows 8 software that the team developed translates the signals into an audio signal, and spoken words emerge.
EnableTalk recently became one of 6 finalists at Microsoft's Imagine Cup. The researchers say the cost of the prototype adds up to only about $75. Plus, it can learn.
Besides the cost, though, another feature that makes this project so interesting is that users can teach the system new gestures and modify those that the team plans to ship in a library of standard gestures, TechCrunch reports. Given the high degree of variation among sign languages, which also has regional dialects just like spoken language, this will be a welcome feature for users.
Image: EnableTalk gallery
Jul 12, 2012
This is another minor modification of Dr. James Kramers attempt to make a talking glove some 20 years ago. There are alot of other considerations to be taken into account to translate "sign language" into the spoken language. Dr. Kramer attempted to construct a glove that could translate "Finger spelling" into "English" language speech using analog strain gagues and could not interperate all hand movements. I have a method to construct a Digital glove that uses "flexible" linear encoders with a resolution down to NANO and can measure all hand movements. But no one is interrested. The deft and dumb do not desire to play on our fieldd when they can force us to play their game! GOOD LUCK!
These guys should get together with the guys over at Leap Motion. These gloves have all the hardware, but still "just miss" the full signing experience by being unable to include (detect and categorize) the body language OTHER than the hand motions (hand placement relative to the head, facial expressions, mouth movements, etc). On the other hand they already have the accompanying software package which can take inputs from a system and capture a new gesture, while recognizing an existing gesture, and linking to an output mechanism to generate speech (I would assume a straightforward "output text" option probably exists or could easily exist, for those times when you would like to receive a text msg translation). Really, the only thing missing from the software side is the ability to recognize a text or voice input and "playback" a corresponding gesture (on a smartphone screen, say). What Leap Motion has is the Gesture Detection sensor to beat all sensors -- using infrared light, within a volume of eight cubic feet, they can measure the position of EVERYTHING within that volume, 160 times per second, with a precision of 0.01 millimeter (ten micrometers, one wavelength of infrared light). So whether the gloves are redundant or not (I can't say -- maybe, maybe not -- the sensor device costs slightly less than the gloves) that "recognition space" can encompass hands, head and upper torso, and the Leap sensor can detect ALL visible changes in ALL of those elements (including everything the gloves miss). With the ability to fully and completely detect any sign, to recognize known signs and capture new signs, to interact with text AND voice input/output devices (pipe incoming ambient "voice" to a voice recognition device, generate outgoing speech from a library of recorded speech phrases) AND with a smartphone video display screen able to PLAYBACK any archived gesture -- we are within reach of an inexpensive full two-way translation device (as my wife says "if you were allergic to it, you'd be sneezing right now"). If you Google "sign language" there are more than two hundred different recognized languages; this device should work with them all. The combined EnableTalk/Leap device would ease acquisition of signing skills (practice in front of it, get immediate feedback about "what you just signed" and coaching on what still needs practice) and make those skills, essentially, readily available to everyone.
Gee, I hate people like you. Have you ever said anything good about an article you've seen on this site? You, are a real a hole. I can't say words strong enough, because I do want you and others to read what I have to say. My grandparents haven't been able to hear since birth and therefore aren't able to speak. They are deaf-mutes. They call themselves that by using sign language. If the gloves are perfected it would be a wonderful thing. I sincerely hope they are!!
I'm sorry, but what century are we in? Deaf-Mute? really? How about Deaf-and-Dumb, too, while we're at it? http://www.eamo.org/SNA/deaf%20PC%20terminology.pdf" What is Wrong With the Use of The Terms: 'Deaf-mute', 'Deaf and dumb', or 'Hearing-impaired'?" by Nancy Creighton, Pub. Coordinator Natl Assoc. of the Deaf Deaf and hard of hearing people have the right to choose what they wish to be called, either as a group or on an individual basis. Overwhelmingly, deaf and hard of hearing people prefer to be called "deaf" or "hard of hearing". Nearly all organizations of the deaf use the term "deaf and hard of hearing", and the National Association of the Deaf (NAD) is no exception. The World Federation of the Deaf (WFD) voted in 1991 to use "deaf and hard of hearing" as an official designation. Yet there are many people who persist in using terms other than "deaf" and "hard of hearing". The alternative terms are often seen in print, heard on radio and television, and picked up in casual conversations all over. Let us take a look at the three most-used alternative terms used today:
Now someone needs to build a system that converts spoken words into sign language, perhaps on a small screen on the gloves.
I hope they can make this succeed. I know about the wall you speak of, having deaf in-laws. It's a very lonely and confusing world for them. Hopefully, the gloves will be comfortable. Those in the photo don't look it. Better yet, a kit that one could deploy on the gloves of their choice, though that would be a significant challenge for both inventor and user Interesting how this does not completely remove the barrier, as the receiver needs to sign back or respond somehow. Still, it helps the deaf to interact with the world, it's a great step forward. Learn sign, everyone!
I think there is another use that wasn't mentioned in the article. If non-deaf people use these as well when talking to deaf people, it would probably make it easier to learn sign. I have never had to learn sign, and it sounds like you do, so what do you think of that idea?