Evaluating risk prediction models: the predictiveness curve and its geometric summaries
American Journal of Epidemiology · 2026-11-04 · 195 (11) · 2899-2906
DOI: 10.1093/aje/kwag123 ↗Abstract
Abstract Risk prediction models are widely used, but their evaluation is dominated by discrimination-based measures, such as the receiver operating characteristic (ROC) curve and AUC, which provide limited insight into how risks are distributed across a population. We extend the predictiveness curve to risk prediction models, characterize its geometric properties, and derive 3 geometric summaries—the Pietra index, Gini index, and scaled Brier score. To obtain well-calibrated and stable risk estimates, we implement a 3-step prediction post-processing procedure, consisting of cross-validation, isotonic recalibration, and bootstrap averaging. We illustrate the approach using toy examples and a nationwide Taiwan cohort of 23 839 lung cancer patients, in whom a Cox model was used to estimate the 5-year risk of fatality. The 3 indices captured complementary aspects of performance and supported model choice for specific purposes: the Pietra index for minimizing the gray zone, the Gini index for maximizing separation, and the scaled Brier score for increasing certainty. In the lung cancer application, the final post-processed model showed clear risk stratification: 25.2% of patients were very low risk (<10% fatality), and 50.2% were high risk (>75% fatality), whereas only 4.8% were near the average risk. Overall, the predictiveness curve and its geometric summaries provide a population-oriented framework for transparent, purpose-specific evaluation of risk prediction models.
EUVIMED