ML Reference Values

cds.ml estimators are written from scratch in pure Python. To show they are correct, not just runnable, scripts/verify_ml_reference.py runs each estimator on a fixed seeded dataset and compares against reference outputs computed with scikit-learn 1.9.0:

Check	CDS	Reference (sklearn 1.9.0)	|diff|	Tol	Status
LogisticRegression accuracy (scaled features)	0.930000	0.930000	0.00e+00	2e-02	PASS
DecisionTreeClassifier accuracy (max_depth=5)	0.980000	0.980000	0.00e+00	2e-02	PASS
KMeans(k=2) adjusted Rand index vs true labels	0.738285	0.738293	8.15e-06	3e-02	PASS
PCA cumulative explained variance (2 components)	0.712253	0.712253	2.80e-07	1e-06	PASS
LinearRegression r2 on noisy linear target	0.981702	0.981702	3.21e-10	1e-09	PASS

Reproduce with one command (no dependencies beyond CDS itself):

python scripts/verify_ml_reference.py

The dataset is generated in-process from seed=42 (200 samples, 4 features: two Gaussian blobs with binary labels, plus a noisy linear regression target), so every run is bit-for-bit deterministic.

These figures were last regenerated against scikit-learn 1.9.1 and reproduce the table above. scripts/verify_ml_reference.py itself needs no third-party dependency — the sklearn numbers are frozen constants, not something it recomputes.

How the reference values were made

Each metric was computed once with scikit-learn 1.9.0 on the identical generated data and hard-coded into the script, keeping the verification dependency-free:

Run this from the repository root with scikit-learn installed (pip install scikit-learn). scripts/ is not an importable package, so load the dataset generator by path:

import importlib.util
import pathlib

# Reference derivation (requires scikit-learn) — run once, then freeze the
# numbers into scripts/verify_ml_reference.py:REFERENCE.
_spec = importlib.util.spec_from_file_location(
    "verify_ml_reference", pathlib.Path("scripts/verify_ml_reference.py")
)
_mod = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(_mod)
make_dataset = _mod.make_dataset

X, y, y_reg = make_dataset()  # seed=42

from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.metrics import accuracy_score, adjusted_rand_score, r2_score
from sklearn.preprocessing import StandardScaler
from sklearn.tree import DecisionTreeClassifier

Xs = StandardScaler().fit_transform(X)

logreg = LogisticRegression(max_iter=2000).fit(Xs, y)
tree = DecisionTreeClassifier(max_depth=5, random_state=42).fit(X, y)
km = KMeans(n_clusters=2, n_init=10, random_state=42).fit(X)
pca = PCA(n_components=2).fit(Xs)
lin = LinearRegression().fit(X, y_reg)

print(accuracy_score(y, logreg.predict(Xs)))
print(accuracy_score(y, tree.predict(X)))
print(adjusted_rand_score(y, km.labels_))
print(pca.explained_variance_ratio_.sum())
print(r2_score(y_reg, lin.predict(X)))

The printed values reproduce the "Reference (sklearn 1.9.0)" column above.

Honest differences between CDS and sklearn

The numbers agree, but the implementations get there differently. Where behavior can legitimately differ:

• LogisticRegression: CDS uses fixed-learning-rate full-batch gradient descent (lr=0.5, epochs=2000); sklearn uses LBFGS. Both land on the same decision boundary here, but on barely-separable data the optima (and therefore a handful of predictions) can differ. Tolerance: ±0.02 accuracy.
• DecisionTreeClassifier: both are greedy Gini CART. Candidates tied on Gini improvement are broken by scan order, which differs between the two implementations; accuracy-level agreement is expected, identical trees are not.
• KMeans: CDS uses k-means++ seeding from a seeded RNG, assigns distance ties to the lowest cluster index, and keeps the previous centroid for an empty cluster. sklearn runs n_init=10 restarts and breaks ties differently. Both converge to the same basin on this dataset (ARI difference ≈ 8×10⁻⁶), but label indices are arbitrary, compare with ARI, never raw labels.
• PCA: CDS diagonalizes the covariance matrix with cyclic Jacobi rotations; sklearn uses LAPACK SVD. Explained-variance ratios are sign-invariant and match to ~10⁻⁷; individual component signs may be flipped between the two (a valid PCA convention difference).
• LinearRegression: both solve the normal equations in closed form; agreement is at floating-point rounding level (~10⁻¹⁰).

When the table needs refreshing

Update the frozen numbers only when the dataset generator, an estimator's algorithm, or the pinned sklearn version changes, and always re-run the full script afterwards. If a check starts failing, treat it as a regression signal first, not a tolerance problem.