Brain waves used for faster image sorting
Computers are fast for many tasks, but humans are faster for identifying objects or people in images. But is it possible to combine the speed of a computer with the sensitivity of the human brain? According to a IEEE Spectrum Online article, 'A Brainy Approach to Image Sorting,' several teams at Honeywell, Teledyne Scientific and Imaging, and Columbia University think so. They're working on a Defense Advanced Research Projects Agency's program called 'Neurotechnology for Intelligence Analysts' (NIA). One of the teams said intelligence analysts can sort images six times faster than before. But there is a culprit: they'll need to carry for hours a 32-electrode EEG cap which detects their brain activity. But read more...
The Defense Advanced Research Projects Agency (DARPA)'s program, Neurotechnology for Intelligence Analysts (NIA), in its third and final stage. At the end of this stage, one of the teams mentioned above will be selected. OGI Deniz Erdogmus, an assistant professor of computer science and engineering at the OGI School of Science & Engineering of the Oregon Health & Science University (OHSU) is working with the Honeywell team on the NIA project. You can see above some pictures used by Erdogmus in his lab to speed up sorting. (Credit: Unknown photographs, via IEEE Spectrum Online)
According to Erdogmus, "it takes humans about 300 milliseconds to consciously recognize specific information in a picture -- an adult face among children, for example. It takes another 200 ms for the person to react physically, say, by pushing a button as an analyst would do. But even before a person is conscious of what he or she is seeing -- about 150 ms after being shown an image -- the electrical activity in the brain's visual cortex has already spiked. The activity is called an event related potential, or ERP."
With these facts in mind, Erdogmus designed his experiments. "Six professional image analysts watched as aerial photographs flashed on a computer screen, more than five of them per second. The analysts were told to search the terrain for large targets, such as golf courses. Meanwhile, a 32-electrode EEG cap, plastered to the analysts' heads, detected brain activity that was then recorded in a separate computer. After the experiment, Erdogmus ran the recordings through a program that flagged any pictures whose appearance coincided with an ERP. While his analysis pulled out many false targets, it rarely missed a real one."
The results look good, but Erdogmus's team is still facing several challenges. First, "the brain continues to respond electrically even after the image disappears, which makes it difficult to match signals with the pictures that evoked them." This can be solved by calibrating the system for each new user, but this might be expensive. Then, there are users' issues. "The question remains whether watching images in rapid sequence will tire analysts out faster and ultimately make them less efficient." And the analysts will have to have a conductive gel covering their heads and electrodes hooked to their heads for long times. This is not something I would like to do...
Anyway, the research that Erdogmus is doing for DARPA is not widely available. Nevertheless, he co-authored a large number of papers. For example, during the 2006 Conference on Human Factors in Computing Systems (CHI '06), held in Montréal, Québec, Canada, he presented a paper named "Neurophysiologically driven image triage."
Here is a link to the abstract. "Effective analysis of complex imagery is a vital aspect of important domains such as intelligence image analysis. As technological developments lower the cost of gathering and storing imagery, the cost of searching through large image sets for important information has been growing substantially. This paper demonstrates the feasibility of using neurophysiological signals associated with early perceptual processing to identify critical information within large image sets efficiently. Brain signals called evoked response potentials, detected in conjunction with rapid serial presentation of images, show promise as a human computer interaction modality for screening high volumes of imagery accurately and efficiently."
He also presented a paper at the 3rd International IEEE/EMBS Conference on Neural Engineering held in Hawaii in May 2007 named "A Fusion Approach for Image Triage using Single Trial ERP Detection." Here is a link to the abstract. "This paper addresses the problem of conducting visual target search on a large set of images. We use electroencephalography to detect targets and apply a fusion approach combining neurophysiologic signals and overt physical responses to achieve high target detection accuracy. We conducted an experimental evaluation of the method using trained human experts to find target objects in broad area satellite images. Based on the fusion results, we applied spatial target likelihood maps to present the estimated target locations in the images. The results demonstrate the efficacy of the method on multiple subjects."
Sources: Morgen E. Peck, IEEE Spectrum Online, April 2008; and various websites
You'll find related stories by following the links below.