People of ACM - Anil K. Jain

April 5, 2016

How did you first become interested in pattern recognition, computer vision, biometrics and analysis of random fields? What kinds of projects were you working on when you entered the field in the early 1970s?

I joined the EE graduate program at Ohio State in 1969. During the spring quarter, 1970, a new faculty member in computer science at that time, B. Chandrasekaran, offered a course on "pattern recognition" that I attended. Chandra used a draft of the classic pattern recognition book by Duda and Hart (first published in 1972) for us to read, and additionally covered some papers from the literature. One topic that I really liked was the "curse of dimensionality," which essentially suggests that an arbitrary increase in the number of features does not necessarily correspond to an increase in recognition accuracy when the number of training examples is small. As part of the course, I extended the results of one of the papers we were reading, and published it in the IEEE Transactions on Computers. After taking the pattern recognition course, I got involved in a project funded by the Air Force Office of Scientific Research that my advisor Robert McGhee was working on. The goal of the project was to design a pattern recognition system that could recognize different types of planes (MIG, Mirage, and Phantom) based on their two-dimensional images. We didn't have any real images of these planes, so we hung model airplanes from the ceiling and took photos at different viewpoints to generate training and test patterns.

One of your most recent projects has garnered a lot of attention. What was a key challenge in developing the large-scale face-search system that helped catch the Boston Marathon bomber, and how did you and your team overcome it?

Cameras are everywhere, from smartphones to surveillance posts on streetcorners. At the same time, the size of face databases maintained by law enforcement and various government agencies (e.g., motor vehicle bureau and passport office) and posted on social media sites is huge and continues to grow. So, naturally, it is desirable to search for a photo, captured by a smartphone or a surveillance camera, in real-time against these databases with millions of images. While tremendous progress has been made in face recognition over the past 25 years, most of the work in research labs has focused on face verification, i.e., 1:1 comparison. But, many applications need the capability of large-scale face search, i.e., 1:N comparisons, where N could be in the millions. In order for the search to be effective, it needs to be extremely fast and extremely accurate. This will allow law enforcement to identify and apprehend a suspect quickly without many false alarms.

This large-scale face search has been our focus for the past four years or so. We now have a database of 120 million face images. Depending on the quality of a query image, we can locate the person of interest with sufficient accuracy, provided that he has a photo in the database. We have shown this capability with the surveillance photo of Dzhokhar Tsarnaev, the younger brother involved in the 2013 Boston Marathon bombings. His social media photo could be retrieved from an 80 million-face database at rank eight in seven seconds.

Do you think biometric gatekeepers such as fingerprint and iris recognizers are more effective than traditional gatekeepers such as login credentials or passwords in securing cell phones, PCs and ATM machines? In 15 years, will biometric gatekeepers be universal? Could a person's biometric information be stolen?

Every security mechanism, whether based on credentials or biometrics, has strengths and limitations. If the correct password is entered, access will be assured, but the system does not know who entered the password, genuine user or a malicious impostor. Passwords or credentials, in general, could be stolen, shared or guessed; this is why biometric traits, linked to the human body, have received so much traction. Who would have thought just five years ago that hundreds of millions of citizens will be unlocking their phones using a tiny area of their fingertips (just 90 x 90 pixels)? Over the next few years, smartphones from every major vendor would come embedded with biometric authentication schemes, whether fingerprint, face or iris or some other attribute. The cost of the biometric sensor has dropped to just a few dollars, and the matching algorithms are now robust and accurate. So, then, what's the limitation of biometrics? Well, the user has to be cooperative. You will not get authenticated if you place a wet finger on a fingerprint reader. The growth and popularity of biometrics does not mean that credentials are obsolete. They still have a role to play in multi-factor authentication (e.g., to guard against biometric spoofs) and as a "backup" if biometric authentication does not work, as in the case when you have a deep cut on your fingertip(s).

An important challenge is how do we safeguard our biometrics? Our face can be captured covertly or taken from social media sites. We touch various surfaces during the day so fingerprints could be lifted from there to build a spoof. Or, an insider attack could steal the biometric traits (or the derived templates) from the biometric database and then target that specific individual. Hence, there is an urgent need for research on detection of spoofs, also known as biometric presentation attacks, and stronger template protection beyond just encryption.

What advice would you give younger colleagues who are just getting started in these areas of research?

Choice of the research problem is the key. There is no recipe for it, but in my experience, listening and talking to domain experts helps. When I first started working on fingerprints 25 years back, the typical reaction from my colleagues in the computer vision and pattern recognition community was "Isn't the problem solved?" Good thing I didn't listen to them and persisted. I tend to avoid working on problems where too many players have already driven their stake. This is reflected in our ongoing work on large-scale face search and large-scale face clustering involving over 100 million faces. Another exciting problem we are working on is recognizing newborns and toddlers using fingerprints. Applications of this are numerous, from vaccination tracking and healthcare to national registries, particularly in less developed countries.