This article proposes the use of demographic information (e.g., gender and ethnicity) and facial marks (e.g., scars, moles, and freckles) for improving face image matching and retrieval performance.
Soft biometric traits embedded in a face (e.g., gender and facial marks) are ancillary information and are not fully distinctive by themselves in face-recognition tasks; however, this information can be explicitly combined with face matching score to improve the overall face-recognition accuracy. Moreover, in certain application domains, e.g., visual surveillance, where a face image is occluded or is captured in off-frontal pose, soft biometric traits can provide even more valuable information for face matching or retrieval. Facial marks can also be useful in differentiating identical twins whose global facial appearances are similar. The similarities found from soft biometrics can also be useful as a source of evidence in courts of law, because they are more descriptive than the numerical matching scores generated by a traditional face matcher. An automatic facial mark detection method has been developed that uses (1) the active appearance model for locating primary facial features (e.g., eyes, nose, and mouth), (2) the Laplacian-of-Gaussian blob detection, and (3) morphological operators. Experimental results based on the FERET database (426 images of 213 subjects) and two mugshot databases from the forensic domain (1,225 images of 671 subjects and 10,000 images of 10,000 subjects, respectively) show that the use of soft biometric traits can improve the face-recognition performance of a state-of-the-art commercial matcher. (publisher abstract modified)