System matches suspect sketches to mug shots

With so many security cameras in public places, it's easier than ever for criminals to be caught in the act. But some of the most serious crimes are committed outside the view of a camera's lens, leaving police to rely on a sketch of the suspect.

Now, researchers from Michigan State University are aiming to make those sketches as powerful as photographs. Doctoral student Brendan Klare and Anil Jain, a professor of computer science and engineering, have developed a system that matches artistic renderings of crime suspects to mug shot databases. I spoke recently with Klare and Jain about how the system works -- and how it could change the way we catch criminals.

You developed a system that matches mug shots with police sketches. How does that technology work?

Klare: If you're given a photograph of a person's face and a sketch of a face, we'd like the system to [determine whether they] are the same two people. Unfortunately, that's not quite possible given the difficulty of the problem. These sketches are not perfect. They're from someone's memory. We have to use new techniques because one image is a sketch and the other is a photograph. We take a sketch and a photo. We measure the similarities of different regions of the face and we're given a score. The way a forensic investigator would use this is the system would have access to all the face images in the mug shot database. When they submit a sketch, it would be measured for similarities to all the photographs in the database. It will return an ordered list, showing the most similar face first.

How does this advance the technology that's already available?

Klare: It's seen as a different system. There are about 10 commercial face recognition systems that are extremely good at matching photos to photos. But they weren't designed to match a sketch to a photo. We designed a system specific for this problem.

Jain: When you capture a face photo by a digital camera, the images consist of pixels. If you match a photo to a photo, the two images are composed in the same way. A sketch is essentially a line drawing. It does not contain the same level of information as a digital photo. That's why the commercial systems primarily relied on matching a digital image of the face to another digital image of the face. The challenge was to come up with some features we could extract from the mug shot and the sketch which could be effectively matched in order to improve the accuracy.

Why is it important to be able to match sketches to photos?

Klare: It's because these are such heinous acts. There's been a lot of publicity about the East Coast Rapist. The only thing they had on him was a sketch. They just caught the guy. This is the only visual information we have on the person when they've done something terrible. Usually people who have done these horrible crimes are repeat offenders, so they're in these mug shot databases.

Jain: One of the main advantages of a system like this is the speed at which you can apprehend the suspect. The current method is you post the sketch in public places. Nobody may come forward. If you have an automatic system, the law enforcement agencies already have millions of these images of people who have been convicted before. A matching system like this can address this problem in a few hours.

What results have you seen in your testing so far?

Jain: We have a database of 10,000 mug shots. We took 150 mug shots [which are paired with sketches] and added them to the database. We saw how often the system correctly found the sketch's mate in the mug shot database.

Klare: We separated the sketches into good sketches and bad sketches based on visual similarities. There are a lot of sketches that don't look anything like the suspect. For the ones that we labeled as good, the system could identify those about 45 percent of the time. For the ones that were poorer, identification was about 10 to 15 percent. We had higher matching accuracy on the poor sketches than the commercial systems had on the good sketches.

These results are about a year old. Since then, we've done a lot more research. We have an improved algorithm now. We didn't separate them into good and poor sketches. We had about 28 percent accuracy without factoring out race and gender.

What's the next step in this research?

Klare: We submitted a proposal to the National Institute of Justice to receive additional funding. We're going to continue to improve the accuracy of the system. We need to continue to build our database. Any artists who have sketches they could send would be enormously helpful. We're deploying this system at a [law enforcement] agency in Florida. As they receive forensic sketches, they'll be using this in real scenarios.

Photo, top: Brendan Klare

Photo, bottom: Anil Jain

This post was originally published on Smartplanet.com