Academic

Quantifying differences between probability distributions is central to statistics and machine learning, primarily for comparing statistical uncertainty. However, epistemic uncertainty—stemming from incomplete knowledge—requires richer representations than classical probabilities. Imprecise probability (IP) theory provides such models, capturing ambiguity and partial belief.

As interest in Imprecise Probabilistic Machine Learning (IPML) grows, so does the need for metrics that extend beyond precise probabilities. This talk introduces the Integral Imprecise Probability Metric (IIPM) framework—a Choquet integral-based generalisation of classical Integral Probability Metrics (IPMs) to capacities, covering IP models like lower probabilities, belief functions, probability intervals, and many more.

We show that IIPM satisfies key metric properties and metrises a natural form of weak convergence for capacities. It also offers a unified framework for comparing IP models and quantifying epistemic uncertainty within one. By contrasting a model with its conjugate, we derive a new class of epistemic uncertainty measures—Maximal Mean Imprecision (MMI)—that satisfy core axioms in the uncertainty quantification literature. Empirical results on selective classification tasks demonstrate MMI’s effectiveness, particularly in high-class regimes where standard measures fail to scale. We believe this work lays a foundation for principled comparison and quantification of epistemic uncertainty under imprecision.