Facial Recognition Software Use on Surgically Altered Faces: A Systematic Review

Objective: Facial recognition software (FRS) is becoming pervasive in society for commercial use, security systems, and entertainment. Alteration of the facial appearance with surgery poses a challenge to these algorithms, but several methods are being studied to overcome this issue. This study systematically reviews methods used in facial recognition of surgically altered faces. Materials and Methods: A systematic review was performed by searching PubMed and Institute of Electrical and Electronics Engineers (IEEE) databases to identify studies addressing FRS and surgery. On initial review, 178 manuscripts were identified relating to FRS and surgery and allowed division into multiple subgroups. The decision was made to focus on the recognition of surgically altered faces. Results: Eligible studies included those reports in English on FRS of surgically altered faces, and 39 papers were included. Surgical procedures range from affecting skin surface, such as skin peeling, to altering facial features, such as rhinoplasty, mentoplasty, malar augmentation, brow lift, facelift, orthognathic surgery, facial reanimation, and facial feminization. Methods were classified into appearance-based, feature-based, and texture-based. Descriptive versus experimental protocols were characterized by different reporting outcomes and controls. Accuracy ranged from 19.1% to 85.35% using various analysis methods. Conclusions: Knowledge of available limitations and advantages can aid in counseling patients regarding personal technology use, security, and quell fears about surgery to evade authorities. Surgical knowledge can be utilized to improve FRS algorithms for postsurgical recognition.