[Mattral]

What changed

Owner decisions applied on top of the S0.3 branch.

• Deleted docs/COMPLETION_SUMMARY.md, docs/PHASE_1_3_SUMMARY.md, docs/RESEARCH_REPORT.md, results/benchmarks.md, docs/PHASE_1_CONFIG.md, docs/PHASE_2_PPO.md, docs/PHASE_3_CLI.md, and docs/PHASE_4_BENCHMARKS.md. docs/index.md now points only at docs that remain. CODE_MAP.md is not written here.
• ROADMAP.md records a judgment for each of the 13 failures. The 11 DPO and PPO failures are unchanged. No xfail and no skip.
• RolloutBuffer calls pin_memory() only when torch.cuda.is_available().
• RolloutGenerator._sample_token returns [batch, 1] instead of a squeezed 1-D token, so the time-axis cat works.
• Added S0.11 after S0.8: the CLI must actually run SFT, the reward model, DPO, and PPO on a tiny CPU model, with an end-to-end test.

How it works

pin_memory() asks the CUDA caching allocator for page-locked host memory. The CPU wheel has no accelerator, so the call raises. The guard keeps the detached CPU tensor and pins it only when CUDA is present.

torch.multinomial(..., num_samples=1) returns [batch, 1]. The old squeeze(-1) made that [batch]. torch.cat on dim 1 then had no dim 1. The generator now concatenates those [batch, 1] steps into [batch, response_length].

The DPO margin test expects the mean of policy chosen minus policy rejected (1.5 on its fixture). The code subtracts the reference gap as well and gets 0.4. The beta test uses a fixture whose policy gap equals the reference gap, so every beta produces -logsigmoid(0). The PPO train_step gathers with gather(1, ...) and the test model mean-pools the sequence to [batch, vocab]. Those stay open for S0.6 against TRL. Details are in the S0.6 notes.

How to verify

git ls-files ROADMAP.md
git check-ignore agent.md
test ! -f docs/PHASE_1_CONFIG.md && test ! -f results/benchmarks.md
PYTHONPATH=src pytest -q -m "not slow and not gpu"
ruff check . && ruff format --check .

Local result: ruff passes. Pytest is 71 passed, 11 failed. tests/test_rollout.py is 3 passed. ROADMAP.md is tracked. agent.md is ignored.

PR 7's README does not say the CLI was validated on a T4. It says run-ppo only writes a config and run-dpo does not train.

What is still not proven

The 11 red tests. No TRL parity. No end-to-end CLI run. No GPU number. S0.4 has not started.