Overview
EN ISO/IEC 5259-4:2025 (ISO/IEC 5259-4:2024) defines a Data Quality Process Framework (DQPF) for analytics and machine learning (ML). It establishes common, organization‑agnostic approaches to ensure the quality of training and evaluation data used across supervised, unsupervised, semi‑supervised and reinforcement learning, as well as general analytics. The standard covers the full data lifecycle - from acquisition, composition and preparation to labelling, evaluation, provisioning and decommissioning - without mandating specific tools or platforms.
Key topics and requirements
- Data quality process principles: foundational concepts that guide consistent data quality practices for ML and analytics.
- DQPF structure: five main process areas - planning, evaluation, improvement, process validation and guidance on using the framework.
- Data lifecycle coverage: requirements and guidance for
- Data requirements and planning (scoping, data needs, provenance)
- Data acquisition and composition (source selection, integration)
- Data preparation (cleaning, encoding, de‑identification)
- Data labelling and annotation (specifications, roles, task management)
- Data quality assessment and improvement (evaluation criteria, remediation)
- Data provisioning and decommissioning (safe use and retirement of datasets)
- ML‑specific guidance: tailored process steps for supervised, unsupervised, semi‑supervised and reinforcement learning, including dataset composition and training/evaluation considerations.
- Roles and responsibilities: defined participant roles such as data planner, originator, collector, engineer, holder and user, clarifying accountability across the data quality process.
- Labelling methods & process controls: recommended practices for labelling specifications, task assignment, quality checking and revision workflows.
Practical applications and who uses it
This standard is intended for organizations of any size or sector implementing ML or analytics projects who need repeatable, auditable data quality processes. Typical users include:
- Data governance teams and data engineers establishing ML data pipelines
- ML/AI practitioners and data scientists focused on training/evaluation data integrity
- Quality assurance, compliance and risk management professionals assessing ML readiness
- Vendors and system integrators designing data workflows, annotation platforms or data services (note: the standard does not prescribe specific tools)
Practical benefits include improved model performance, reduced bias and rework, clearer audit trails for datasets, and stronger alignment between data operations and ML objectives.
Related standards
- Part of the ISO/IEC 5259 series on AI data quality; developed by ISO/IEC JTC 1 and adopted as EN ISO/IEC 5259‑4:2025 by CEN. Consider integrating this DQPF with organizational data governance and AI risk management standards for comprehensive ML governance.