ISO/IEC 30137-4:2021

Overview

ISO/IEC 30137-4:2021 - part of the ISO/IEC 30137 series - defines a standardized ground truth and video annotation procedure for the use of biometrics in video surveillance systems (VSS). It establishes requirements for annotating humans, faces, body parts and arbitrary objects in imagery, specifying the metadata to embed, how to encode spatial and temporal ground truth (including presence and confirmed absence), and procedures for annotating known vs unknown subjects. The document does not specify video encoding formats.

Key topics and technical requirements

• Annotation scope: Humans, human faces, body parts and arbitrary objects appearing in video; excludes encoding of video data.
• Metadata model: Standardized metadata elements to insert in a video stream (regionData root element and related types) and an XSD schema provided in Annex A.
• Spatial annotation:
  • Bounding box encoding (axis-aligned rectangles).
  • Bounding polygon encoding for complex or rotated regions.
  • Face landmarks (anthropometric landmarks) may be used instead of or in addition to bounding regions.
• Temporal annotation:
  • Frame timestamps, frame intervals and track encodings to represent when subjects appear, move and leave.
  • Encoding of tracks through single or multiple camera sequences.
• Presence and absence: Procedures and encodings for objects present in video and for explicitly documented absences.
• Object and subject metadata:
  • Object class, generic object information, and human subject metadata (appearance descriptors, pose, occlusion, estimated attributes).
  • Procedures for annotating known (identified/enrolled) versus unknown subjects.
• Conformance: Records must meet semantic, structural and encoding requirements specified across Clauses 6–7 and Annex A.
• Interoperability: Schema references ISO/IEC metadata standards (XSD import patterns), enabling machine-readable annotations.

Applications and users

Practical uses include:

• Training and evaluation of automated face recognition (AFR), detection and tracking algorithms.
• People counting, crowd density and behavior analysis.
• Re-identification and multi-camera tracking across camera networks.
• Operational deployments such as watchlist matching, law-enforcement review, and access control testing. Key users:
• System integrators and VSS vendors
• Data annotators and lab managers building ground-truth datasets
• Researchers and algorithm developers benchmarking performance
• Compliance officers and evaluators assessing biometric system accuracy

Related standards

• Part of the ISO/IEC 30137 series (guidance on biometrics in VSS). A list of related parts is available on the ISO and IEC websites.
• Annex A of ISO/IEC 30137-4 provides the normative XSD schema for producing conformant annotation records.

Keywords: ISO/IEC 30137-4:2021, video annotation, ground truth, biometrics, video surveillance, face recognition, bounding box, polygon, metadata, VSS, annotation procedure.