ISO/IEC 19795-10:2024 PDF

Overview

ISO/IEC 19795-10:2024 - "Information technology - Biometric performance testing and reporting - Part 10: Quantifying biometric system performance variation across demographic groups" defines requirements for measuring and reporting biometric performance variation across different demographic groups. The standard covers metrics such as failure-to-enrol (FTE), failure-to-acquire (FTA), shifts in comparison scores, recognition error rates (e.g., false match/false non-match), and response/processing time (throughput). It applies to technology evaluations, scenario evaluations, and operational (fielded) system assessments and provides terms, definitions and reporting conventions to promote consistent, transparent assessment of demographic differentials.

Key topics and technical requirements

• Demographic variables and ground truth: guidance on categorical (sex, ethnicity, gender) and continuous (age, skin lightness) demographic measures and required ground-truth data.
• Trial generation: methods for producing mated and non-mated comparison and identification trials for analysis.
• Differential performance metrics (DPM): standardized measures including mathematical difference and ratio approaches, false positive differential (FPD), false negative differential (FND), and comparison score differential measures.
• Aggregate equitability measures: procedures such as geometric-mean-based comparisons and Gini coefficient approaches for multi-group assessments.
• Operational thresholds & policy: how threshold selection affects demographic differentials and reporting of threshold-management policies.
• Statistical uncertainty & sample size: requirements for reporting uncertainty estimates, sampling the target population, and practical sample-size considerations (including an illustrative annex).
• Reporting requirements: clear templates for documenting experimental design, target application, test population, differential performance results, comparison-score analyses, and exception handling.

Practical applications

ISO/IEC 19795-10:2024 is designed to support:

• Vendors and system integrators conducting development and pre-deployment testing to detect demographic performance variation.
• Procurement agencies requiring objective, comparable evidence of biometric system equitability during vendor evaluation.
• Operational program managers and auditors assessing fielded systems for unequal performance across demographics.
• Researchers and regulators quantifying demographic differentials and reporting results with statistical rigor.

By standardizing metrics and reporting, the document helps stakeholders identify, communicate and mitigate potential demographic biases in biometric systems.

Related standards

• ISO/IEC 19795 series (biometric performance testing & reporting)
• ISO/IEC 2382-37 (biometrics vocabulary) - normative reference used for terminology

Keywords: ISO/IEC 19795-10:2024, biometric performance, demographic groups, differential performance, failure-to-enrol, failure-to-acquire, comparison score, equitability, statistical uncertainty, biometric system testing.