Artificial intelligence is revolutionizing healthcare — from diagnostics to personalized treatment — but concerns remain about the safety and fairness of AI systems trained on complex and often opaque datasets.

At the BIPM’s Scientific Symposium held during the 150th Anniversary of the Metre Convention, Dr Daniel Schwabe of the Physikalisch-Technische Bundesanstalt (PTB), presented a poster entitled “On the path to quality assurance of trustworthy AI in medicine: the METRIC-framework for systematically assessing medical training and test data.”

His poster explores how metrology can help assess the quality of data used in machine learning for medical applications — a critical concern as Europe moves towards regulating high-risk AI under the EU AI Act.

“One of the biggest levers we have to assess and improve AI performance lies in the data — especially training and test data.” – Dr Schwabe explained.

Machine learning models are only as good as the information they learn from, and how datasets are collected is equally important from a traceability, transparency and standardization standpoint.

Dr Schwabe’s poster outlines how the METRIC-framework can support conformity assessment procedures and provides a pathway toward regulatory readiness. It also highlights future directions, including the development of automated tools to assess dataset quality and to quantify how shortcomings in data may affect the behaviour of AI systems — such as their predictive accuracy or fairness across patient populations.

Poster

The METRIC-framework was co-authored by Katinka Becker, Martin Seyferth, Andreas Klaß and Tobias Schäffter, in collaboration with partners from PTB, the Technical University of Berlin, and the Einstein Centre for Digital Future.

Listen to a 3 minute audio recording of Dr Schwabe explaining their poster, here.

Watch Dr Schwabe’s Presentation

Watch all 9 poster presentations at the BIPM 150th Anniversary Scientific Conference on the BIPM YouTube channel.