Effective dimensionality, the thing ndt tracks, is precisely how people detect representation collapse, a named and feared failure in self-supervised, contrastive, and JEPA-style training, in ViTs, and in fine-tuning. A collapsing or stagnant representation is a failing run you usually cannot see in the loss curve; the dimensionality curve shows it. That is a decision: keep training, stop, or fix.
🔍 ndt monitor how a neural network's representational dimensionality evolves during training and lets you watch the representation and catch collapse, and it verifies the signal. What ndt shows, in one number: roughly, "how many independent directions is the representation actually using?" Many directions, held steady or growing, means healthy and spread out. That number sliding toward a few means it is collapsing. So the health check is simply: is that number holding up, or falling? ndt draws you that curve while the model trains.
Unlike any other tracker, ndt tells you whether a change it detects is real or an artifact of your measurement.
| The situation | What ndt gives you |
|---|---|
| A jump detector reports a clean transition in your training curve. | ndt runs that same detector on trajectories whose transitions you control, and reports its recall and its false-positive rate on pure noise. |
| Two detectors on the same run disagree almost completely. | One verdict on ground truth, in milliseconds, before you commit to a claim. No model, no GPU, no training. |
| You want to see representation geometry, not just loss. | Four dimensionality metrics per layer, logged on any PyTorch model in three lines. |
| A reviewer asks how you know the transition is not an artifact. | A rendered report with a positive control, three null controls, and a plain PASS or FAIL. |
| Goal | Start here |
|---|---|
| Check a detector before you trust it | Tutorial 1 |
| Track dimensionality during training | Tutorial 2 |
| Validate your own detector or your own data | Tutorial 3 |
| Understand the result behind the tool | The finding |
👉 Read the full API | Documentation
pip install ndtrackerThis pulls PyTorch and NumPy, everything the tracker and the harness need. The validity harness itself is pure NumPy, so its tests run without a GPU.
This is the reason ndt exists. Take any transition detector, run it against a battery of ground-truth fixtures, and read a verdict. The built-in JumpDetector is included as one detector to be validated, not as a truth oracle.
from ndt import JumpDetector
from ndt.validity import validate_detector, jump_detector_as_callable
detector = jump_detector_as_callable(JumpDetector(z_threshold=3.0))
report = validate_detector(detector, name="JumpDetector(z=3.0)")
print(report.render())Validity report for: JumpDetector(z=3.0)
planted_transition recall: 1.00 false positives: 5
planted_multi recall: 1.00 false positives: 14
pure_noise null control false positives: 2 (5.0/1000 steps)
drift_no_jump null control false positives: 4 (10.0/1000 steps)
mean recall on planted transitions : 1.00
false positives on null controls : 6
VERDICT: NOT VALID on these fixtures
Fires on pure noise and continuous drift. It manufactures transitions where
there are none, so a detection on real data cannot be trusted without this check.
Read this carefully: the detector recovers every planted transition (recall 1.00) and fires on pure noise and on a smooth drift. A bare detection count says nothing until you know that false-positive rate. The harness gives you the number, and the verdict is a function of it:
report.valid # False
report.mean_recall # 1.0
report.false_positives_on_null # 6Add representational-dimensionality tracking to any PyTorch model in three lines. The tracker attaches hooks, samples activations at a fixed frequency, and records four metrics per layer.
import torch.nn as nn
from ndt import HighFrequencyTracker, plot_phases
model = nn.Sequential(
nn.Linear(784, 512), nn.ReLU(),
nn.Linear(512, 256), nn.ReLU(),
nn.Linear(256, 10),
)
tracker = HighFrequencyTracker(model, sampling_frequency=10)
for step, (x, y) in enumerate(dataloader):
output = model(x)
loss = criterion(output, y)
loss.backward()
optimizer.step()
tracker.log(step, loss.item()) # the one added line
results = tracker.get_results() # dict: layer name -> DataFrame
fig = plot_phases(results, metric="stable_rank")results is a dictionary keyed by layer name; each value is a DataFrame with one row per sampled step and one column per metric. The four metrics:
| Metric | What it measures |
|---|---|
participation_ratio |
effective number of active dimensions in the representation |
stable_rank |
Frobenius-to-spectral norm ratio, a noise-robust rank |
cumulative_energy_90 |
dimensions needed to hold 90% of the spectral energy |
nuclear_norm_ratio |
nuclear norm normalized by dimension |
The harness accepts any detector that is a callable from a value sequence to the step indices it flags. It also accepts any fixtures you build, including ones drawn from a real training run whose transition you can see in test accuracy (for example a grokking run). This is how you move from synthetic controls to your own data without changing the interface.
import numpy as np
from ndt.validity import validate_detector, Fixture, planted_transition, pure_noise
# 1. Your own detector: values -> detected step indices.
def slope_detector(values, k=20, threshold=3.0):
v = np.asarray(values)
steps = []
for i in range(k, len(v) - k):
before = v[i - k:i].mean()
after = v[i:i + k].mean()
if abs(after - before) > threshold:
steps.append(i)
return steps
# 2. Your own fixtures: a positive control, a null, and your real run.
my_run = Fixture(
values=load_my_stable_rank_trajectory(), # your metric over steps
ground_truth=(4200,), # the step you trust from test accuracy
name="grokking_run",
tolerance=25,
)
battery = [planted_transition(), pure_noise(), my_run]
report = validate_detector(slope_detector, battery, name="slope_detector")
print(report.render())A detector that passes here recovers your known transition and stays quiet on the null. A detector that fails is measuring its own method, whatever it reports on data you cannot check.
ndt is the tool built out of a result, so the result is worth stating plainly.
Across 55 training runs and 30,147 high-frequency measurements, transition detection turned out to be a property of the detector as much as of the data. Change the detector, or change the metric, and the "phase transitions" move or vanish. A z-score threshold detector and the threshold-free PELT algorithm, run on the same trajectories, correlated at -0.029. Most of what looks like a discrete jump in a training curve is a smooth, continuous change that a detector rounds into a step.
The conclusion is not that transitions never happen. It is that a detection is a claim, and a claim about a transition is only as good as the detector's behavior on ground truth. ndt exists to make that behavior measurable in milliseconds, so the claim can carry weight. This is the same construct-validity idea that the evaluation-science literature applies to any measurement instrument, brought to training dynamics.
Full method and results are in the companion paper, Phase Transitions or Continuous Evolution? Methodological Sensitivity in Neural Network Training Dynamics.
your model (MLP / CNN / Transformer / ViT)
│
HighFrequencyTracker.log(step, loss)
│
┌───────────────────┴───────────────────┐
│ │
activation hooks four dimensionality
(per-layer capture) metrics per layer
│ │
└───────────────────┬───────────────────┘
│
results → export (CSV / JSON / HDF5)
│
plot_phases
│
── validity harness ──
any detector → ground-truth fixtures → PASS / FAIL verdict
Complete, runnable scripts live in examples/:
01_quickstart_mnist.py: basic MLP on MNIST02_cnn_cifar10.py: CNN on CIFAR-1003_reproduce_tds_experiment.py: reproduces the 8000-step, 784-256-128-10 MNIST experiment from the Towards Data Science article, measurements every 5 steps
@software{marin2026ndt,
author = {Marín, Javier},
title = {ndt: Validity-Checked Tracking of Neural Network Training Dynamics},
year = {2026},
publisher = {GitHub},
url = {https://github.com/Javihaus/ndt},
version = {2.0.0}
}Contributions are welcome. See CONTRIBUTING.md for setup, the ruff-based lint and format checks, and the test layout. The validity tests run on pure NumPy and need no GPU.
Built by Javier Marín. ndt is part of a body of work on trustworthy AI: respect the system, do not just fit the data.
Apache 2.0. See LICENSE.