The Bulletin

Man hacks Kinect to assist stroke victim

Posting in Design

Chad Ruble has hacked Kinect software to help his disabled mother, who can't use a keyboard after suffering from a stroke 12 years ago.

Ruble's mother suffered a stroke which caused her to develop Aphasia -- a condition that irrevocably damages the language functions in the brain and makes it difficult to communicate both thoughts and emotion.

Worried for his mom and pondering ways that technology may improve her quality of life, Ruble wanted to create a new type of keyboard that she could use to express how she was feeling without needing to read individual, standard keys.

The first stage was to design a keyboard with visual icons -- representing emotions including happiness, sadness, and feeling tired. In order to hone this further, Ruble included a series of bars that his mother could use to track how intensely she felt, such as 'very tired' or 'very happy'.

He then hacked his $109.99 Kinect using the SimpleOpenNI library to process the input from the Kinect through gesture recognition code to track the position of his mom's hand. To complete the hack, Ruble used a "sample processing sketch" provided by Daniel Shiffman to generate and send a complete message via email by 'clicking' the green button. Once complete, by waving her hand the message can be reset by crossing the screen's red "X" mark.

Ruble plans to add more refined features which will allow more variety in messages but maintain "a super simple interface" for his mum. He writes on his blog:

"It would also be fun to snap and attach a jpeg from the RGB Kinect camera to the email. Still, its clear that mom is happy with the result. I might also send the emails to a Posterous account so that she can have her own blog."

Someone else has to launch the program for her at the moment, but as Ruble writes, it's certainly a great start.

Image credit: Chad Ruble

— By on September 12, 2012, 1:11 AM PST

Charlie Osborne

Contributing Editor

Charlie Osborne is a freelance journalist and photographer based in London. In addition to SmartPlanet, she also writes for business technology website ZDNet and consumer technology site CNET. She holds a degree in medical anthropology from the University of Kent. Follow her on Twitter. Disclosure