training-dynamics

Fast and Easy Infinite Neural Networks in Python

CVPR 2024-Improved Implicit Neural Representation with Fourier Reparameterized Training

ICML2025-Inductive Gradient Adjustment for Spectral Bias in Implicit Neural Representations

Existing literature about training-data analysis.

A unified framework for attributing model behavior to model components, training data, and training dynamics.

Code for "What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformers" (NeurIPS 2025)

Official repository for "FOCUS: First Order Concentrated Updating Scheme"

Code for 'Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics'

Code for "Effect of equivariance on training dynamics"

Source code for <Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies>

Official repository for the EMNLP 2024 paper "How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics"

Rewind a training run to the instant its loss spiked, localize the poisoned optimizer state, repair just that slice, and prove it via bit-identical replay forks. Instrument built + CUDA-verified on Kaggle T4; restarting on AMD MI300X. PyTorch research.

Two-parameter Weibull lens on transformer weights (shape k, scale λ): 7-family benchmark, AdamW training dynamics (three-force λ evolution), a data-predictability law for λ growth, and a 76-run grid showing weight-scale growth tracks training effort, not learning quality. npm-weibull-py + database + code, arXiv:2605.18898/2606.19367/2608.23573

TMLR 2026 | Mechanistic interpretability: attention-head binding (EB*) as a marker of concept emergence. 7 models, 5 architectures (Pythia 160M–2.8B, OLMo-1B, CRFM GPT-2, SmolLM3-3B, Qwen2.5-1.5B), 41 terms.

A plug-in debugger and visualizer for RL reward functions. Detects reward hacking, tracks training health, and renders a live terminal dashboard.

Code and data for: Three Phases of Expert Routing — How Load Balance Evolves During MoE Training

A Training-Time Sign Flip in IOI Circuit Formation : Code, data and analyses for paper accepted at the ICML 2026 Mechanistic Interpretability Workshop.

Cross-Family Convergence of Neural Network Weight Skeletons. Companion to Zenodo paper (10.5281/zenodo.19652706).

Git-bisect for neural networks: trace behavioral changes across checkpoints to internal mechanisms, influential training data, and counterfactual causal evidence.

Code, results, and Pythia-160M training checkpoints for "Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training?" (Findings of EMNLP 2026).