The DOI displayed above is for this specific version of this dataset, which is currently the latest. Newer versions may be published in the future. For a link that will always point to the latest version, please use
DOI: 10.4121/3bd18108-fad0-4e4c-affd-4341fba99306

Yan, Cheng; Mohr, Felix; Viering, Tom (2025): Learning Curves Database 1.1. Version 1. 4TU.ResearchData. dataset. https://doi.org/10.4121/3bd18108-fad0-4e4c-affd-4341fba99306.v1

Categories

• Artificial Intelligence and Image Processing
• Information and Computing Sciences

Keywords

AutoML Data-Driven Modeling Learning Curve Machine Learning Meta-Learning Scaling Law

Licence

CC BY 4.0

Interoperability

• RO-Crate Metadata

Export as...

RefWorks BibTeX Reference Manager Endnote DataCite NLM DC CFF

by Cheng Yan , Felix Mohr , Tom Viering

Sample-wise learning curves plot performance versus training set size. They are useful for studying scaling laws and speeding up hyperparameter tuning and model selection. Learning curves are often assumed to be well-behaved: monotone (i.e. improving with more data) and convex. By constructing the Learning Curves Database 1.1 (LCDB 1.1), a large-scale database with high-resolution learning curves including more modern learners (CatBoost, TabNet, RealMLP and TabPFN), we show that learning curves are less often well-behaved than previously thought. Using statistically rigorous methods, we observe significant ill-behavior in approximately 15% of the learning curves, almost twice as much as in previous estimates. We also identify which learners are to blame and show that specific learners are more ill-behaved than others. Additionally, we demonstrate that different feature scalings rarely resolve ill-behavior. We evaluate the impact of ill-behavior on downstream tasks, such as learning curve fitting and model selection, and find it poses significant challenges, underscoring the relevance and potential of LCDB 1.1 as a challenging benchmark for future research.

History

• 2025-05-08 first online

• 2025-10-17 published, posted

Publisher

4TU.ResearchData

Format

*.hdf5

References

• https://arxiv.org/abs/2505.15657

Organizations

TU Delft, Faculty of Electrical Engineering, Mathematics and Computer Science,
Department of Intelligent Systems, Pattern Recognition & Bioinformatics