Test an analysis against data with known generating truth
Can the analysis distinguish finite-sample variation from leakage, missingness bias, or a covariance bug?
Make this plan your own ↓Read, edit and export without an account. This is preparation; no run or result is claimed.
What to compare
Use the fixed intercept/slope/noise model, the declared normal missingness mechanism, and the known covariance matrix. Do not require a finite realization to match population moments exactly.
Useful outputs
- Generated rows, model parameters, seeds, masks and a frozen split.
- Training-only fit/preprocessing and separate held-out metrics.
- Population comparison quantities alongside finite-sample observations.
Inputs and prerequisites
- Three original generators are prepared for the qualified standard-library runtime.
- Scikit-learn comparisons require a separately qualified environment.
What this work would not establish
- Synthetic rows are not real-world measurements.
- No held-out score for a named external model has been measured.
Start with these sources.
Synthetic regression data with declared coefficients and split
Generate 128 regression observations with known coefficients, a declared Gaussian noise process, and a fixed train/test split, together with a small reference fit.
Synthetic missingness data with complete ground truth
Generate paired complete, independently missing, and value-dependent missing observations from a declared standard-normal process.
Synthetic correlated measurements with a known covariance
Generate 256 paired synthetic measurements through a declared linear transformation of independent standard-normal draws, with known population covariance.
3.1. Cross-validation: evaluating estimator performance
A guide to model-evaluation splits, including grouped and time-series data, nested model selection and permutation-based assessment.
3.4. Metrics and scoring: quantifying the quality of predictions
A broad scoring reference that distinguishes classification, regression, ranking and clustering metrics and their scorer interfaces.
Make the question your own.
Edit freely. Export a copy before reloading or leaving. Your text stays in this tab until you choose to copy, download or save it privately.