Phase 8: softmax cell, tile dataflow, naive vs flash views

CONTEXT.md

@@ -36,7 +36,18 @@ On a fresh machine delete the symlinks and run `.tools/setup.sh` and `.tools/ems
 - Phase 7 (web visualizer `/app`): DONE. Presets, editable Q/K/V (quantized in the browser), schedule, tile values, K/V
   buffers (read from the RTL), HBM meter against the golden traffic model, heatmap (engine P for NAIVE, float reference for
   FLASH), Playwright smoke test (5 tests, production build).
-- Next: Phase 8 (L1 softmax cell, L2 tile dataflow, L4 naive vs flash with two instances, N sweep, error panel).
+- Phase 8 (L1 softmax cell, L2 tile dataflow, L4 naive vs flash, N sweep, error panel): DONE. Engine readouts `tileDone`
+  and `getDatapath`. 12 Playwright tests, 15 Vitest, Node WASM test, one-seed cocotb. Light-mode screenshots not reviewed.
+- Next: Phase 9 (docs, landing page, launch). Also open: the `astro dev` hang.
+
+## Gotchas found in Phase 8
+- The debug datapath wires are idle between tiles. The L2 view keeps the last busy clock's values, or the panel shows zeros.
+- A datapath or buffer value is only meaningful while the engine is inside a tile; say so in the UI.
+- Verilator's wide ports are word arrays; the `wordsOf` template reads any of them.
+- A full run in the browser is cheap, but the pages overflowed at 375 px only after a run (long decimals in the output grid,
+  then a 1 px overhang in the tile grids). The phone check must run a computation first.
+- `astro preview` hung (process state D) during Phase 8; a static file server over `dist/` works. Restart it after each build,
+  because the build removes its working directory.
 
 ## Gotchas found in Phase 7
 - `wasm/fixtures/` holds engine runs and `quantize_cases.json`; the Node test must skip the quantization file (it did not for

README.md

@@ -8,7 +8,7 @@ Fourth project in the series after [tiny-gpu](https://github.com/deaneeth/tiny-g
 [tiny-tpu](https://github.com/deaneeth/tiny-tpu) and tiny-cpu. Same rule as the others: nothing on screen is
 faked; every value comes from a `dbg_*` port on the compiled RTL.
 
-**Status:** Phases 0-7 done: the engine (FLASH, NAIVE, K/V overlap) is bit-exact against the golden model, compiled to WebAssembly, and shown clock by clock in a browser visualizer at `/app`, with editable inputs, K/V buffers, HBM traffic against the model, and a Playwright smoke test. Next: the L1, L2 and L4 views. See [`docs/PLAN.md`](docs/PLAN.md) for the roadmap.
+**Status:** Phases 0-8 done: the engine (FLASH, NAIVE, K/V overlap) is bit-exact against the golden model and compiled to WebAssembly. The browser visualizer at `/app` has four views (softmax cell, tile dataflow, schedule, naive vs flash with an N sweep) with editable inputs, K/V buffers, HBM traffic against the model, and 12 Playwright tests. Next: the landing page and the documentation pages. See [`docs/PLAN.md`](docs/PLAN.md) for the roadmap.
 
 ## Quick start
 

docs/ARCHITECTURE.md

@@ -391,3 +391,33 @@ browser checks use `astro build` + `astro preview`.
 needs the files in git. Rebuild with `bash wasm/build.sh` after any RTL or harness change and commit them with it. CI does
 not yet check that the committed files match a fresh build; the wasm workflow's `node wasm/test.mjs` checks behavior, not
 the files.
+
+## 13. Phase 8: the other levels (L1, L2, L4)
+
+The `/app` page has four views over one engine: L1 softmax cell, L2 tile dataflow, L3 schedule (Phase 7), L4 naive vs
+flash.
+
+- **Engine readouts added.** `tileDone` (a one-cycle pulse, from the tile core's `tile_done`) and `getDatapath()` (the
+  MM1 and MM2 weights, activations and partial sums, live). Checked in `wasm/test.mjs`: one completion per FLASH tile, and
+  the running row max never decreases within a row group (keyed on the global row group; the in-block index repeats across
+  Q blocks).
+- **L1 (`SoftmaxCellView`).** The visualizer records the engine's row state at every tile completion. For a chosen row,
+  the table shows each K/V tile's four logits (engine and float), m, α and l. The float reference (`softmax-cell.ts`)
+  uses the same INT8 inputs dequantized, so the difference is fixed-point rounding. A FLASH run is required; NAIVE has no
+  running max.
+- **L2 (`TileView`).** Live wires of MM1 and MM2 and the tile's S, Z, P, m, α, l, O. The values are from the last busy
+  clock of the run so far; between tiles the wires are idle. Stepping through a tile shows the activations streaming.
+- **L4 (`CompareView`).** Two engine instances on the same inputs, each freed after its run. Bars for HBM bytes, busy
+  cycles, tiles and K/V loads (measured). The on-chip footprint, the full-S-matrix size and the tile-core occupancy are
+  formulas, labelled as such. The error panel shows |engine − float| per element for both schedules on one colour scale.
+  The N sweep runs N = 4 to 32 on seeded inputs and plots HBM bytes against N for both schedules.
+
+Gaps against the plan's Phase 8 text:
+- The N sweep uses `runToDone` (one call per run), not a separate `runFast`; the cost is the same.
+- The L2 view is a grid of values, not an animated wire diagram. Each block shows its live values.
+- L1's exp2 split is described in text; the view does not draw the integer and LUT parts separately.
+- "Every level works in light and dark": the tests check that both colour schemes render with a background. Nobody has
+  reviewed the light-mode screenshots yet.
+
+Checks: `pnpm test:e2e` (12 tests, including all four views at 375 px and the light/dark background), `pnpm test` (15),
+`node wasm/test.mjs`, `SEEDS=1 bash sim/run_e2e.sh`, `bash sim/lint.sh`, `pnpm lint`, `pnpm typecheck`, `pnpm build`.

docs/PLAN.md

@@ -452,7 +452,7 @@ The estimates assume one person part-time, at tiny-cpu's pace.
   `HbmMeter`, and **L3 Schedule first**.
 - **Exit:** pick a preset, step forward and back, and watch tiles, buffers and bytes update; the Playwright smoke test passes.
 
-### Phase 8: Remaining levels (1.5 weeks)
+### Phase 8: Remaining levels (1.5 weeks) ✅ DONE (see ARCHITECTURE §13)
 - L1 Softmax cell, L2 Tile dataflow, L4 Naive vs Flash (two instances, N sweep, error panel).
 - **Exit:** every level works on desktop and at 375 px, in light and dark themes.
 

docs/STATE_SCHEMA.md

@@ -20,6 +20,7 @@ the ports for it (`dbg_*` in `tiny_attn_top` and `attn_tile`); only the bindings
 | `cycles` | number | busy-cycle counter (state != idle, DONE included) |
 | `tiles` | number | tile computations (FLASH: group tiles; NAIVE: pass-1 tile pairs) |
 | `kvLoads` | number | K/V tile loads (FLASH: K loads incl. prefetch; NAIVE: one per tile pair) |
+| `tileDone` | boolean | one-cycle pulse: a tile completed on the last edge (FLASH: one per tile; NAIVE: one per tile operation) |
 
 **Counters are final one edge after `done`.** `done` is high during the DONE cycle, and the busy counter increments at the
 end of that cycle. `wasm/test.mjs` takes one more `step()` before reading.
@@ -41,6 +42,7 @@ class TinyAttnSim {
   isDone(): boolean;
   getTile(): TileState;        // tile-core values of the most recent tile (S, Z, P, m, l, alpha, O)
   getKVBuffers(): KVBuffers;   // K and V tile buffers (both), and the buffer in use
+  getDatapath(): Datapath;     // MM1 and MM2 weights, activations and partial sums (live wires; 16 values each)
   readProbabilities(): number[];  // NAIVE: P from HBM, n*n values (UQ1.8)
   delete(): void;
 }
@@ -58,6 +60,12 @@ Input validation throws: `n` must be a multiple of 4 in 4..32, `qrows` one of 4,
 
 Multiple instances are independent: each owns its Verilator context. Steps may be interleaved across instances.
 
+## Datapath (`getDatapath`)
+
+Index `r*4 + c` is PE (r, c). `mm1W` (Kᵀ weights) and `mm1A` (Q activations) are int8; `mm1Psum` (S partial sums) is int17.
+`mm2W` (V weights) is int8, `mm2A` (P activations) is uint9, `mm2Psum` (P·V partial sums) is int18. These are the wires at
+the current clock.
+
 ## K/V buffers (`getKVBuffers`)
 
 `k[(buf*4 + row)*4 + col]` is K[row][col] of buffer `buf` (int8, sign-extended); `v` the same for V. `sel` is the buffer

rtl/tiny_attn_top.sv

@@ -58,7 +58,8 @@ module tiny_attn_top
     // K/V buffers: word (buf*4 + row) holds row `row` of buffer `buf` (4 int8 lanes); dbg_kv_sel is the buffer in use
     output logic [2*4*32-1:0]   dbg_kbuf,
     output logic [2*4*32-1:0]   dbg_vbuf,
-    output logic                dbg_kv_sel
+    output logic                dbg_kv_sel,
+    output logic                dbg_tile_done    // one-cycle pulse: a group tile (or NAIVE row group) completed
 );
     // ---- controllers (only the one selected by cfg_naive is started)
     logic                f_busy, f_done, n_busy, n_done;
@@ -117,6 +118,7 @@ module tiny_attn_top
         end
     end
     assign dbg_kv_sel = kv_sel;
+    assign dbg_tile_done = tile_done;
     logic [31:0] tin_words [8];
 
     logic                q_load_valid;

wasm/harness/bindings.cpp

@@ -12,6 +12,17 @@ using namespace emscripten;
 namespace {
 std::vector<int32_t> toI32(const val& arr) { return vecFromJSArray<int32_t>(arr); }
 
+val datapathToJs(const Datapath& d) {
+    val o = val::object();
+    o.set("mm1W", val::array(d.mm1W.begin(), d.mm1W.end()));
+    o.set("mm1A", val::array(d.mm1A.begin(), d.mm1A.end()));
+    o.set("mm1Psum", val::array(d.mm1Psum.begin(), d.mm1Psum.end()));
+    o.set("mm2W", val::array(d.mm2W.begin(), d.mm2W.end()));
+    o.set("mm2A", val::array(d.mm2A.begin(), d.mm2A.end()));
+    o.set("mm2Psum", val::array(d.mm2Psum.begin(), d.mm2Psum.end()));
+    return o;
+}
+
 val kvToJs(const KVBuffers& kv) {
     val o = val::object();
     o.set("sel", kv.sel);
@@ -45,6 +56,7 @@ val snapshotToJs(const Snapshot& s) {
     o.set("cycles", s.cycles);
     o.set("tiles", s.tiles);
     o.set("kvLoads", s.kvLoads);
+    o.set("tileDone", s.tileDone);
     return o;
 }
 }  // namespace
@@ -73,6 +85,7 @@ EMSCRIPTEN_BINDINGS(tiny_attn) {
         }))
         .function("snapshot", optional_override([](TinyAttnSim& self) { return snapshotToJs(self.snapshot()); }))
         .function("isDone", &TinyAttnSim::isDone)
+        .function("getDatapath", optional_override([](TinyAttnSim& self) { return datapathToJs(self.getDatapath()); }))
         .function("getKVBuffers", optional_override([](TinyAttnSim& self) { return kvToJs(self.getKVBuffers()); }))
         .function("getTile", optional_override([](TinyAttnSim& self) { return tileToJs(self.getTile()); }))
         .function("readProbabilities", optional_override([](TinyAttnSim& self) {

wasm/harness/harness.cpp

@@ -149,6 +149,7 @@ Snapshot TinyAttnSim::snapshot() const {
     s.cycles = top_->cycles;
     s.tiles = top_->tiles;
     s.kvLoads = top_->kv_loads;
+    s.tileDone = top_->dbg_tile_done != 0;
     return s;
 }
 
@@ -243,3 +244,30 @@ KVBuffers TinyAttnSim::getKVBuffers() const {
     }
     return kv;
 }
+
+namespace {
+// Signed or unsigned field of `width` bits at bit `lsb` of a word array, as int32.
+int32_t field(const uint32_t* words, int lsb, int width, bool isSigned) {
+    uint64_t v = bitsAt(words, lsb, width);
+    return static_cast<int32_t>(isSigned ? signExtend(v, width) : static_cast<int64_t>(v));
+}
+}  // namespace
+
+Datapath TinyAttnSim::getDatapath() const {
+    Datapath d;
+    const uint32_t* w1 = wordsOf(top_->dbg_mm1_w);
+    const uint32_t* a1 = wordsOf(top_->dbg_mm1_a);
+    const uint32_t* p1 = wordsOf(top_->dbg_mm1_psum);
+    const uint32_t* w2 = wordsOf(top_->dbg_mm2_w);
+    const uint32_t* a2 = wordsOf(top_->dbg_mm2_a);
+    const uint32_t* p2 = wordsOf(top_->dbg_mm2_psum);
+    for (int i = 0; i < 16; ++i) {
+        d.mm1W.push_back(field(w1, i * 8, 8, true));
+        d.mm1A.push_back(field(a1, i * 8, 8, true));
+        d.mm1Psum.push_back(field(p1, i * 17, 17, true));
+        d.mm2W.push_back(field(w2, i * 8, 8, true));
+        d.mm2A.push_back(field(a2, i * 9, 9, false));
+        d.mm2Psum.push_back(field(p2, i * 18, 18, true));
+    }
+    return d;
+}

wasm/harness/harness.h

@@ -21,6 +21,7 @@ struct Snapshot {
     uint32_t cycles;     // busy cycles (perf counter)
     uint32_t tiles;
     uint32_t kvLoads;
+    bool tileDone;       // one-cycle pulse: a tile completed on the last edge
 };
 
 // Tile-core state of the most recent tile (see attn_tile.sv dbg_* ports). S, Z, P, m, l, alpha and O are for the
@@ -39,6 +40,13 @@ struct TileState {
 
 // K and V tile buffers. k[(buf*4 + row)*4 + col] is K[row][col] of buffer buf (int8, sign-extended); same for v.
 // sel is the buffer the engine is using for the current tile.
+// Datapath wires of the two systolic arrays, live. Index r*4 + c is PE (r, c). MM1 computes S = Q . Kt (weights Kt,
+// activations Q rows); MM2 computes P . V (weights V, activations P). Values are signed except mm2A (P, unsigned).
+struct Datapath {
+    std::vector<int32_t> mm1W, mm1A, mm1Psum;  // 16 each: int8, int8, int17
+    std::vector<int32_t> mm2W, mm2A, mm2Psum;  // 16 each: int8, uint9, int18
+};
+
 struct KVBuffers {
     int sel;
     std::vector<int32_t> k;  // 32
@@ -71,6 +79,7 @@ class TinyAttnSim {
     bool isDone() const;
     TileState getTile() const;
     KVBuffers getKVBuffers() const;
+    Datapath getDatapath() const;
     // P tiles written by a NAIVE run (HBM words 768..): n x n probabilities (UQ1.8), row-major. Requires n known.
     std::vector<int32_t> readProbabilities();
 

wasm/test.mjs

@@ -25,6 +25,25 @@ const check = (ok, msg) => {
   }
 };
 
+// FLASH tile order (web/src/lib/schedule.ts): Q blocks outer, K/V tiles, row groups; causal skips masked tiles.
+function flashOrder(n, qrows, causal) {
+  const nb = n / 4;
+  const g = qrows / 4;
+  const order = [];
+  for (let gb = 0; gb < nb; gb += g) {
+    const bg = Math.min(g, nb - gb);
+    const last = causal ? gb + bg - 1 : nb - 1;
+    for (let kj = 0; kj <= last; kj++) {
+      for (let gi = 0; gi < bg; gi++) {
+        const qi = gb + gi;
+        if (causal && kj > qi) continue;
+        order.push({ qi, kj });
+      }
+    }
+  }
+  return order;
+}
+
 function load(fx) {
   const sim = new mod.TinyAttnSim();
   sim.configure(fx.n, fx.qrows, fx.causal, fx.mode === "NAIVE" ? 1 : 0, fx.overlap, fx.latency, fx.cfgScale);
@@ -40,6 +59,23 @@ for (let t = 0; t < maxCycles && sims.some((s) => !s.finished); t++) {
   for (const s of sims) {
     if (s.finished) continue;
     s.last = s.sim.step();
+    if (s.last.tileDone && s.fx.mode === "FLASH") {
+      // one completion per tile; the running row max m never decreases within a row group
+      s.tileDones = (s.tileDones ?? 0) + 1;
+      const t = s.sim.getTile();
+      s.order = s.order ?? flashOrder(s.fx.n, s.fx.qrows, s.fx.causal);
+      const key = s.order[s.tileDones - 1].qi;  // global row group of this completion
+      s.prevM = s.prevM ?? new Map();
+      const prev = s.prevM.get(key);
+      if (prev) {
+        for (let r = 0; r < 4; r++) {
+          if (t.m[r] < prev[r]) {
+            check(false, `${s.fx.name}: m decreased for group ${key} row ${r}: ${prev[r]} -> ${t.m[r]}`);
+          }
+        }
+      }
+      s.prevM.set(key, t.m.slice());
+    }
     if (s.last.done) {
       // the DONE cycle is counted by the busy-cycle counter at its end: one more edge makes the counters final
       s.last = s.sim.step();
@@ -48,7 +84,7 @@ for (let t = 0; t < maxCycles && sims.some((s) => !s.finished); t++) {
   }
 }
 
-for (const { fx, sim, last, finished } of sims) {
+for (const { fx, sim, last, finished, tileDones } of sims) {
   console.log(`${fx.name} (${fx.mode} N=${fx.n} causal=${fx.causal} qrows=${fx.qrows} overlap=${fx.overlap} L=${fx.latency})`);
   check(finished, "engine did not finish");
   const out = Array.from(sim.readOutput());
@@ -57,6 +93,7 @@ for (const { fx, sim, last, finished } of sims) {
   check(last.hbmRdBytes === fx.hbmRdBytes, `read bytes ${last.hbmRdBytes} != ${fx.hbmRdBytes}`);
   check(last.hbmWrBytes === fx.hbmWrBytes, `write bytes ${last.hbmWrBytes} != ${fx.hbmWrBytes}`);
   check(last.tiles === fx.tiles, `tiles ${last.tiles} != ${fx.tiles}`);
+  if (fx.mode === "FLASH") check(tileDones === fx.tiles, `tile completions ${tileDones} != ${fx.tiles}`);
   check(last.kvLoads === fx.kvLoads, `K/V loads ${last.kvLoads} != ${fx.kvLoads}`);
   check(last.cycles === fx.cycles, `busy cycles ${last.cycles} != ${fx.cycles}`);
   if (fx.lastTile) {

web/e2e/levels.spec.ts

@@ -0,0 +1,85 @@
+import { expect, test, type Page } from "@playwright/test";
+
+async function openApp(page: Page) {
+  await page.goto("/app");
+  await expect(page.getByRole("button", { name: "Start" })).toBeVisible({ timeout: 30_000 });
+}
+
+test("L1 softmax cell: running max and sum for the chosen row, against the float reference", async ({ page }) => {
+  await openApp(page);
+  await page.getByRole("button", { name: "Start" }).click();
+  await page.getByRole("button", { name: "Run to end" }).click();
+  await page.getByRole("tab", { name: "L1 Softmax cell" }).click();
+
+  await expect(page.getByText("m engine / float")).toBeVisible();
+  await expect(page.getByRole("cell", { name: "kv0" })).toBeVisible();
+  const text = await page.locator("main").innerText();
+  const m = text.match(/m engine \/ float\n([0-9.-]+) \/ ([0-9.-]+)/);
+  expect(m, "m engine / float line").not.toBeNull();
+  expect(Math.abs(Number(m?.[1]) - Number(m?.[2]))).toBeLessThan(0.05);
+});
+
+test("L1 says so for NAIVE", async ({ page }) => {
+  await openApp(page);
+  await page.getByRole("combobox", { name: /^Schedule/ }).selectOption("1");
+  await page.getByRole("tab", { name: "L1 Softmax cell" }).click();
+  await expect(page.getByText("NAIVE has no running max")).toBeVisible();
+});
+
+test("L2 tile dataflow shows the MM1 and MM2 arrays with live values", async ({ page }) => {
+  await openApp(page);
+  await page.getByRole("button", { name: "Start" }).click();
+  await page.getByRole("button", { name: "Run to end" }).click();
+  await page.getByRole("tab", { name: "L2 Tile dataflow" }).click();
+
+  await expect(page.getByText("MM1: S = Q · Kᵀ (array)")).toBeVisible();
+  await expect(page.getByText("MM2: P · V (array)")).toBeVisible();
+  await expect(page.getByText("O accumulator (rescaled by α)")).toBeVisible();
+});
+
+test("L4 runs both schedules on the same inputs and sweeps N", async ({ page }) => {
+  await openApp(page);
+  await page.getByRole("tab", { name: "L4 Naive vs Flash" }).click();
+  await page.getByRole("button", { name: "Run both on these inputs" }).click();
+  await expect(page.getByText("Same inputs, both schedules")).toBeVisible();
+  await expect(page.getByText(/FLASH, max [0-9.]+, mean [0-9.]+/)).toBeVisible();
+
+  await page.getByRole("button", { name: "Sweep N = 4 to 32" }).click();
+  await expect(page.getByText("HBM bytes against N (seeded inputs, same settings)")).toBeVisible();
+  // header row plus one row per N: 4, 8, ..., 32
+  const sweep = page.locator("table").last();
+  await expect(sweep.locator("tbody tr")).toHaveCount(8);
+});
+
+test("light and dark colour schemes both render the visualizer", async ({ browser }) => {
+  for (const scheme of ["light", "dark"] as const) {
+    const context = await browser.newContext({ colorScheme: scheme });
+    const page = await context.newPage();
+    await openApp(page);
+    const bg = await page.evaluate(() => getComputedStyle(document.body).backgroundColor);
+    expect(bg).not.toBe("rgba(0, 0, 0, 0)");
+    await context.close();
+  }
+});
+
+test("L4 at phone width: no horizontal scroll", async ({ page }) => {
+  await page.setViewportSize({ width: 375, height: 812 });
+  await openApp(page);
+  await page.getByRole("tab", { name: "L4 Naive vs Flash" }).click();
+  await page.getByRole("button", { name: "Run both on these inputs" }).click();
+  await expect(page.getByText("Same inputs, both schedules")).toBeVisible();
+  const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth);
+  expect(overflow).toBeLessThanOrEqual(0);
+});
+
+test("every view has no horizontal scroll at phone width", async ({ page }) => {
+  await page.setViewportSize({ width: 375, height: 812 });
+  await openApp(page);
+  await page.getByRole("button", { name: "Start" }).click();
+  await page.getByRole("button", { name: "Run to end" }).click();
+  for (const name of ["L1 Softmax cell", "L2 Tile dataflow", "L3 Schedule", "L4 Naive vs Flash"]) {
+    await page.getByRole("tab", { name }).click();
+    const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth);
+    expect(overflow, name).toBeLessThanOrEqual(0);
+  }
+});