When stacking works: it depends on which features your models look at
Stacking TabPFN3, TabICL, and XGBoost provides at most +0.5 pp AUC on most tabular datasets. But on heavily imbalanced fraud detection, the ensemble is dramatically more robust. The reason is not model diversity in the abstract—it is concrete feature disagreement. XGBoost and TabPFN disagree strongly on which features matter for fraud (Spearman ρ = 0.24), while they agree closely on every other dataset (ρ = 0.67–0.95). When models look at different features, stacking hedges correlated failure modes. When they look at the same features, stacking is just expensive averaging.