3.1. Cross-validation: evaluating estimator performance
A guide to model-evaluation splits, including grouped and time-series data, nested model selection and permutation-based assessment.
External resource. No execution or independent verification is claimed here.
How much optimism arises when repeated observations from the same group are split across train and test?
Generate evaluator-owned grouped data, audit split disjointness by entity and compare estimated errors against an independent held-out population.
What you could produce
- A versioned minimal protocol, evaluator-owned test cases, and a comparison report with numerical/statistical uncertainty and failures.
Before you use it
- scikit-learn 1.9.1
- Compatible NumPy/SciPy and compiled dependencies; exact environment not prepared
Limits to keep in view
- No upstream example, source package, build hook or submitted code was executed.
- The current qualified pilot accepts only self-contained Python 3.13 with a 90-second deadline. This reference's package/runtime is not qualified for that path.
- The proposed protocol requires bounded resource estimates and an independently controlled evaluator before any scientific execution claim.
Source and permission context
scikit-learn documentation. 3.1. Cross-validation: evaluating estimator performance. https://scikit-learn.org/stable/modules/cross_validation.html; observed version 1.9.1 (observed documentation version; stable URL is mutable).
Rights need review. Review the scope and upstream conditions before reuse.
documentation · BSD-3-Clause
This documentation page credits the developers and explicitly labels its footer BSD License; the matching 1.9.1 COPYING file identifies the three-clause terms. Referenced datasets and separately licensed material are not cleared.
Inspect the license evidence ↗Still unresolved
- Third-party figures, linked papers, datasets and dependency licenses have not been assessed; no external content is copied into this catalog.
Put this resource to work.
Test an analysis against data with known generating truth
Can the analysis distinguish finite-sample variation from leakage, missingness bias, or a covariance bug?
Open the brief RESEARCH BRIEF · 6 SOURCESBuild a baseline whose preprocessing can be audited
Does a more complicated classifier improve on a simple baseline under the same frozen split and training-only normalization?
Open the brief