Date: Tuesday, 2 February 2026, at 11.00am (CEST)

Dual format: ESSEC Singapore Campus, Room TBA
and via Zoom, please click here

This study evaluates binary classifier performance with a focus on calibration, which is often overlooked by traditional metrics like accuracy. In high-stakes domains such as finance and healthcare, well-calibrated probabilities are crucial. We highlight the limitations of standard calibration metrics, particularly under score distortions and heterogeneous distributions. To address this, we introduce the Local Calibration Score and advocate optimizing models using Kullback-Leibler (KL) divergence to better align predicted scores with true probabilities. Our approach emphasizes balancing global and local calibration, ensuring overall distributional alignment while maintaining reliability across different score ranges. Using Random Forest and XGBoost across diverse datasets, we show that KL-based tuning improves calibration without sacrificing performance. Our results reveal that relying solely on traditional metrics can mislead model assessment, especially in sensitive decision-making scenarios. This is some joint work with Agathe Fernandes Machado and Ewen Gallic.

Kind regards,
Pierre Alquier, Marie Kratz, Roberto Reno, and Riada Djebbar (Singapore Actuarial Society - ERM)