ISO/TS 8000-81:2021

Overview

ISO/TS 8000-81:2021 - "Data quality - Part 81: Data quality assessment: Profiling" defines a standardized procedure for data profiling as the foundation for data quality assessment. The technical specification applies to data sets organized as tables and columns (or produced by transformations into that form) across all database technologies. The aim is to produce a data profile that helps organizations identify data quality improvement opportunities and support subsequent rule creation and governance.

Key topics and requirements

• Scope of profiling
  • Structure analysis, column analysis and relationship analysis are the three mandatory processes.
  • Applicable to tabular data and outputs of transformations into tables/columns.
• Structure analysis
  • Inputs: data set and optional column metadata (names, descriptions).
  • Output: a data element concept that captures the conceptual domain for subsequent analysis.
• Column analysis
  • Inputs: data set + data element concept.
  • Activities: extract data elements, compare elements to actual values, determine the value domain.
  • Outputs: constraints of value domain including cardinalities (row counts, distinct counts, nulls), storage characteristics (data types, lengths, decimals), and valid-value definitions (lists, ranges, patterns). Methods include discovery, assertion testing and visual inspection; automation tools can assist.
• Relationship analysis
  • Inputs: data set + data elements from column analysis.
  • Activities: identify dependencies and correspondence between data structure and real-world items (requires collaboration with domain experts).
  • Outputs: list of dependencies - primary/foreign keys, functional dependencies, derived columns, and synonym relationships (redundant or domain synonyms).
• Explicit exclusions
  • The specification does not cover data extraction/sampling methods, deriving data rules, or measuring the extent of nonconformities.

Practical applications

• Establishing a repeatable data profiling baseline for data governance, data quality management, and data stewardship.
• Informing data migration, master data management (MDM), analytics, BI, and regulatory compliance efforts by revealing structural issues, value-domain constraints and inter-column dependencies.
• Supporting the design of validation rules and remediation plans (profiling provides inputs for rule derivation, though rule creation itself is outside the spec).
• Useful for automated tooling adoption: selection, configuration and evaluation of data profiling tools.

Who should use this standard

• Data quality managers, data stewards, data engineers, DBAs, business analysts, BI teams, auditors and architects responsible for data governance, MDM, ETL/ELT and analytics.

Related standards

• ISO 8000 series (context for data quality)
• ISO 8000-2 (Vocabulary)
• ISO 8000-8 (Measuring data and information quality)
• ISO 8000-61 and ISO/TS 8000-1 (data quality management and series overview)

Keywords: ISO/TS 8000-81, data profiling, data quality assessment, value domain, column analysis, relationship analysis, data governance, data stewardship.