By Laura Shin
Posting in Design
Microsoft and Carnegie Mellon researchers unveil OmniTouch, a system that turns hands, walls, pads of paper and more into touchscreens.
This week, Microsoft and Carnegie Mellon researchers are unveiling OmniTouch, a system that turns any surface into a touchscreen.
Composed of a depth-sensing camera and laser-based pico projector (more on that in a second), the contraption looks like a flattened-pancake version of R2D2 and sits on the user's shoulder.
The user first creates a screen wherever he or she likes -- such as a hand, arm, wall, table or pad of paper -- and then, just like with any other touchscreen, uses fingers to navigate it.
The screen is created with the laser-based pico projector, which is an electronic system placed on a tiny chip. It transforms the image into an electronic signal, which then sparks laser lights of different colors. Those in turn get copied and projected by mirrors, pixel-by-pixel, onto the selected surface.
The depth-sensing camera then detects when a user's fingers are touching the "screen" (the surface the user has designated) and "pressing" a button on it.
In order to get the technology to work, the researchers needed to teach the system how to recognize fingers, which are crucial to the technology because they create screens, select buttons, zoom in and so on. They dissected the qualities of fingers and developed a method for determining when a finger was "clicking" on something.
Researcher Hrvoje Benko said:
In this case, we're detecting proximity at a very fine level. The system decides the finger is touching the surface if it’s close enough to constitute making contact. This was fairly tricky, and we used a depth map to determine proximity. In practice, a finger is seen as “clicked” when its hover distance drops to one centimeter or less above a surface, and we even manage to maintain the clicked state for dragging operations.
The technology was unveiled at the 2011 UIST, a conference on innovations in the ways humans interact computer in Santa Barbara, Calif., from October 16-19. Benko and Carnegie Mellon Ph.D. student Chris Harrison and Andrew D. Wilson of Microsoft wrote about the technology in this paper.
Watch the video to see how users can create a screen, move an image on the screen, zoom in and out, paint using a palette of colors, select buttons on a menu, and more.
This video explains how the system learned to identify and track the movement of fingers, in order to detect commands on the system.
Oct 17, 2011
Siri shows the way. Voice is the interface of the future. Touch is old news. BTW similar technology was show in TED some time back (also developed in MIT) at this link http://www.pranavmistry.com/projects/sixthsense/
So you still have a cell phone, but now it's hanging on your shoulder. Maybe this will be a piece of a better idea, but like it is, I kinda hate it.
...except I guess my parrot will have to sit on the other shoulder. At least it's not a Univac. Seriously, thanks to someone for pointing out its use for the handicapped. Dopey me was trying to figure out how this was an advance.
Pranav Mistry is the actual man behind this technology (which he named SixthSense) and I am surprised why his name is not mentioned anywhere !
I am hoping that this means that "monitors" will shortly be a thing of the past. This has to be exceptionally environmentally friendly! Ahhh, technology getting smaller as our universe expands.
Add SSL access to your home network, a pair of those glasses with a built-in display - to view things others shouldn't see (think bank account info) - phone functionality and then you'll really have something.
I don't care if it is old I want 1. On as serious note this is a near production ready product. The product footprint will shrink and the final appearance will be cleaned up. Marketing will then stick their nose into it, turn it lime green and price it out of reality, and everyone will say it could have been such a great thing. But I still want one.
The Japanese have had this kind of projector/touch technology for several years - so it's neither new nor news. This is a refinement of existing technology and should be reported as that.
I'm not so surprised. Carnegie-Mellon is a very socially/politically/ambition-driven university. (Even more than most!)
Give it a minute, it have to be reduced to the size of a pendant and powered as such before it's worth mas production....and that's where the motivation is.