OpenReview 5nNNVY8NW4fixtureaudited 2026-08-15
To Grok Grokking: Provable Grokking in Ridge Regression
Theory paper on provable grokking in ridge regression. Hand-driven reproduction that passed the ICML 2026 Agent Reproductions judge end to end — now the pipeline's regression fixture.
- supportedC1 Theorem 4.1 — end-to-end grokking (zero teacher)
- supportedC2 Theorem 4.2 — grokking with realizable teacher
- supportedC3 Theorems 4.4-4.6 — phase decomposition
- supportedC4 Figure 2 — hyperparameter dependence of grokking time
- supportedC5 Figures 3-4 — two-layer ReLU grokking beyond the linear setting
- supportedFALS
arXiv 1903.08560corpusaudited 2026-10-04
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
Exact asymptotic risk formulas for the min-norm interpolator in both regimes — the canonical companion to the double-descent paper. The lab verified its underparameterized risk blow-up (289x at gamma near 1) and its optimal-ridge comparison.
- inconclusiveC1 Isotropic Min-Norm Risk Formula
- not_auditedC2 Double Descent in Misspecified Model
- not_auditedC3 Latent Space Model Monotonicity
- not_auditedC4 Nonlinear Model Universality
- supportedC5 Ridge Dominates Min-Norm
- supportedC6 Underparametrized Risk Divergence
arXiv 1906.11300corpusaudited 2026-10-04
Benign Overfitting in Linear Regression
Effective-rank characterization of when interpolation generalizes. The lab found boundary cases its eigenvalue-rank inequality chain does not cover — two falsifications with passing controls.
- falsifiedC1 Benign Overfitting Condition for Power-Law Eigenvalues
- not_auditedC2 Benign Overfitting in Finite-Dimensional Space with Isotropic Noise
- not_auditedC3 Upper Bound on Excess Risk
- not_auditedC4 Lower Bound on Excess Risk
- not_auditedC5 Necessity of Overparameterization
- falsifiedC6 Relationship Between Effective Ranks
arXiv 1912.07242auditaudited 2026-10-03
More Data Can Hurt for Linear Regression: Sample-wise Double Descent
First fully agent-orchestrated audit: Omnigent lab director + specialist agents verified the double-descent peak at n=d, found the risk valley at gamma=0.90 via an agent-generated hypothesis, and routed an unverified follow-up through a human approval gate.
- supportedC1 Sample-wise non-monotonic test MSE peaks at n=d
- supportedC2 Overparameterized risk approximation (Claim 1)
- supportedC3 Underparameterized risk approximation (Claim 2)
- falsifiedC4 Exact finite-sample bias-variance decomposition (Lemma 1)
- supportedC5 Exact trace update when adding one sample (Claim 3)
- inconclusiveC6 Bias is non-increasing with more samples
- supportedC7 Finite-d risk valley: the overparameterized double-descent dip at gamma*=0.90 (generated variant of C2)
- falsifiedC8 Finite-d bias exactness: E[beta_hat] = gamma*beta at finite d (repairs C2's circular bias check)
- inconclusiveC9 Finite-d correction of the C2 overparam risk approximation: power-law decay d^-alpha at fixed gamma=0.5 (generated variant of C2)
arXiv 2510.10981auditaudited 2026-08-16
In-Context Learning Is Provably Bayesian Inference
ICL-as-Bayesian-inference theory: the risk-decomposition identity and coupled p-N scaling of the Bayes gap verified numerically; the posterior-variance bound honestly inconclusive under finite precision.
- supportedC1 Risk Decomposition Identity
- falsifiedC2 Coupled p-N Scaling of Bayes Gap
- supportedC3 Rapid Task-Type Identification (Posterior Concentration)
- supportedC4 Bayes Gap Upper Bound Rate
- inconclusiveC5 Wasserstein Stability of Bayes Gap
- inconclusiveC6 Posterior Variance Bound by Minimax Risk
- falsifiedC7 Boundary test: pN invariance of the Bayes Gap
arXiv 2607.08077auditaudited 2026-08-17
Modular Pretraining Enables Access Control (GRAM)
GRAM adds gradient-routed auxiliary MLPs to a base model so capabilities can be isolated and selectively disabled. Reproduced the 26M-param Simple Stories experiment on Modal A10G; compute-ratio evaluation follows the paper's Appendix-M power-law inversion.
- supportedC1 GRAM approximates multiple data-filtered models in a single run (Simple Stories)
- not_auditedC2 GRAM isolates realistic dual-use capabilities and matches data filtering at 800M scale
- not_auditedC3 Capability isolation improves with scale and GRAM tracks data filtering from 50M to 5B
- not_auditedC4 GRAM composes arbitrary capability subsets without degradation, unlike FT-LoRA
- not_auditedC5 GRAM outperforms filtering and FT-LoRA on capability removal under partial labeling
- supportedC6 GRAM's training cost is independent of the number of capability profiles, yielding 5× savings over filtering
JMLR 22-1228auditaudited 2026-08-16
Grokking phase transitions in learning local rules with gradient descent
Grokking in cellular-automaton rule learning: critical exponents, grokking-probability trends, and a bimodal grokking-time distribution. Audited end to end on CPU.
- supportedC1 Critical exponent in 1D exponential model
- supportedC2 Critical exponent in D-dimensional uniform ball model
- supportedC3 Grokking probability increases with L1 regularisation
- supportedC4 Grokking probability decreases with dimension D
- inconclusiveC5 Rule-30 CA learning exhibits grokking
- supportedC6 Grokking time distribution bimodality