Feb 24, 2021
Trueface Achieves Top 3 Ranking in the West on NIST 1:N
We are proud to report our performance on Trueface’s first submission to the National Institute of Standards and Technology (NIST) Face Recognition Vendor Test (FRVT) 1:N Identification. Out of 262 entries, we achieved the third-highest ranking by a western firm in the Visa Kiosk category.
As an extension of the NIST FRVT 1:1 competition, the 1:N competition brings a greater challenge to developers. Whereas the 1:1 competition measures the effectiveness of an algorithm at comparing two provided face images to obtain a similarity score, the 1:N competition requires algorithms to enroll up to 12 million face recognition templates into a searchable database. Given a probe image, algorithms are evaluated on their speed and ability to search the database and find any potential identity matches.
When one considers which NIST test to reference for their business the following example may be helpful: As referenced above, NIST 1:1 FRVT measures an algorithm’s ability to compare two provided face images, in order to verify the identity of an individual. 1:1 can be used to tell if a person presenting an ID or passport is who they claim to be. On the other hand, NIST 1:N is measuring an algorithm’s ability to find potential face matches from a reference database, thereby identifying one person out of many. Identification can be used to authenticate a person for use cases like granting access to a building or secure area.
Below is a summary of the performance of all algorithms listed in the main NIST FRVT 1:N Identification leaderboard. The box plots show the distribution of False Negative Identification Rates (FNIR) at a threshold used to achieve a False Positive Identification Rate (FPIR) of 0.003, with a lower FNIR indicating better performance.
FNIR is the proportion of mated searches failing to return the mate above threshold. FPIR is the proportion of non-mated searches producing one or more candidates above threshold…The use of thresholding supports use of face recognition in making mostly automated decisions e.g. for access into facility.
The Trueface algorithm performance is shown by the magenta dotted lines, and the rank below the plots refers to the Trueface algorithm rank in that category.
We performed particularly well on the Mugshot Profile and Visa Kiosk categories, where we placed 16th out of 189 submissions and 15th out of 133 submissions, respectively. When submissions are scrubbed to reflect those based in Western countries only, Trueface ranks third in both these categories.
As you can see in the reference images above, these two categories are particularly challenging because they focus on face images where the subject is not looking directly at the camera, resulting in each face having extreme yaw and pitch angles. These image categories directly reflect real-world environments where perfect front-facing image data are the rare exception to the norm.
Whereas many of the other algorithms chose to ignore the Mugshot profile category entirely and thus performed very poorly in that category, Trueface has trained its face recognition model with face images at various pitch and yaw angles to ensure we have a very robust model. Thus, we are confident in our algorithm’s ability to excel in real-world applications where poor angles, sub-adequate lighting conditions, and other factors can diminish the performance of other models.
We’re proud of achieving a top-three ranking amongst western competitors with our very first submission to NIST FRVT 1:N. However, we’re not here to rest on our laurels. We are throwing this proverbial wreath in the backseat and continuing to blaze the trail to achieving fast, accurate, secure, and above all, unbiased facial recognition for all. We expect to improve with our subsequent submissions and continue to drive this industry forward.
Disclaimer: Results shown from NIST do not constitute an endorsement of any particular system, product, service, or company by NIST. For more information, visit https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-ongoing
