fix(sessionize): split app_ledger sessions on app switch

README.md

@@ -1,6 +1,7 @@
 # activity-frames powering Nocta
 
 [![Downloads](https://static.pepy.tech/badge/activity-frames)](https://pepy.tech/projects/activity-frames)
+[![GitHub stars](https://img.shields.io/github/stars/nossa-y/activity-frames)](https://github.com/nossa-y/activity-frames/stargazers)
 [![Paper](https://img.shields.io/badge/paper-PDF-b31b1b)](https://github.com/nossa-y/activity-frames/blob/main/paper/activity-frames-paper.pdf)
 [![HackerNoon](https://img.shields.io/badge/HackerNoon-top%20story-00E980?logo=hackernoon&logoColor=white)](https://hackernoon.com/i-compiled-55-days-of-screen-activity-into-episodic-memory-for-my-ai-agent)
 [![Python](https://img.shields.io/pypi/pyversions/activity-frames)](https://pypi.org/project/activity-frames/)
@@ -10,13 +11,11 @@
 [![PyPI](https://img.shields.io/pypi/v/activity-frames)](https://pypi.org/project/activity-frames/)
 
 
-> **[Download the desktop app](https://usenocta.app)** - Nocta uses activity-frames to watch how you work and brief you daily on what needs your attention. 100% local.
+**Turn your workday into structured workflows agents can execute.**
 
-**Episodic memory for AI agents - and the routines they can replay.**
+Computer-use agents work every task out from scratch, even one you've done a hundred times. And between tasks, your agent has no idea what you've been doing all day, so it starts every conversation blind.
 
-Your agent can read your code, search the web, and call APIs - but it has no idea what you've been doing all day, so it starts every conversation blind. And when it runs a task for you, it works it out from scratch every time, even one you've done a hundred times.
-
-activity-frames fixes both. It records your screen locally and compiles what it sees into structured **activity frames**: bounded, deterministic episodes of tasks you actually did. The recurring ones compile into **routines a computer-use agent can use** instead of working them out again. So it does your repetitive computer tasks **cheaper** (enriching a compiled routine costs almost no tokens) and **more reliable** (the same steps, grounded the same way every time, instead of guessing from a screenshot).
+activity-frames fixes both. It records your screen locally and compiles what it sees into structured **activity frames**: bounded, deterministic records of the tasks you actually did. The recurring ones become **workflows an agent can execute** instead of working out again. So your repetitive computer tasks get done **cheaper** (running a compiled workflow costs almost no tokens) and **more reliable** (the same steps, grounded the same way every time, instead of guessing from a screenshot) - and everything else becomes context your agent can use.
 
 ```bash
 pip install activity-frames
@@ -28,7 +27,7 @@ aframes context     # your last 2 hours, agent-ready
 
 Capture stores instants: thousands of snapshot rows a day, each one saying "at 22:53:05, Chrome showed linkedin.com/in/...". Useless to reason over.
 
-activity-frames compiles those instants into episodes:
+activity-frames compiles those instants into activity frames:
 
 ```yaml
 - id: f-0007
@@ -59,28 +58,47 @@ away: 18:47-20:24 (97m)
 
 Drop that into a prompt and your agent knows your day. A full day compiles in under a second and costs zero tokens.
 
-## Episodic memory, done honestly
+## Workflows agents can execute
 
-Agent memory today means conversation memory: what you told the model. Episodic memory is what you actually *did* - and the hard part is representing it without lying.
+Computer-use agents re-derive every task from scratch - screenshot, reason, act, repeat - even for a workflow they've run a hundred times. That re-derivation is where the token cost goes, and it's waste: the workflow hasn't changed.
 
-activity-frames enforces a two-tier contract ([SPEC.md](SPEC.md)):
+Because activity-frames compiles recurring activity deterministically, a task you've demonstrated becomes an executable script:
 
-- **Tier 1, measured (this package):** everything is derivable by deterministic code from capture data - sessions, durations, typed page entities, input volume, coverage gaps. No interpretation, no intent labels. Same input, same output, every time.
-- **Tier 2, inferred (optional extension):** tools that add interpretation must namespace it, tag confidence (`high | medium | speculative`), and link evidence. Facts and guesses can never silently mix.
+```bash
+aframes steps --find "message john doe"
+```
 
-Every frame carries evidence pointers back to raw capture rows. Every document declares its blind spots. What the system did not see, it says it did not see.
+```json
+{
+  "steps": [
+    {"t": "20:24:09", "op": "focus", "target": "Google Chrome · LinkedIn", "n": 1},
+    {"t": "20:24:14", "op": "click", "target": "Search", "role": "TextField", "url": "https://www.linkedin.com/feed/", "n": 2},
+    {"t": "20:24:16", "op": "type", "chars": 8, "text": "john doe", "n": 3},
+    {"t": "20:24:21", "op": "click", "target": "John Doe", "role": "Link", "url": "https://www.linkedin.com/search/results/people/", "n": 4},
+    {"t": "20:24:29", "op": "click", "target": "Message", "role": "Button", "url": "https://www.linkedin.com/in/john-doe/", "n": 5},
+    {"t": "20:24:35", "op": "type", "chars": 71, "text": "hey, loved your post on agent memory - open to a quick chat next week?", "n": 6}
+  ],
+  "step_count": 6,
+  "unresolved_clicks": 0
+}
+```
 
-## Beyond memory: routines agents can replay
+That's the replay view of a demonstrated run - ordered clicks grounded by element name and role, typed runs, focus changes. An agent repeats the task instead of re-deriving it: fill the slots with new values (a different name, the same steps) and execute. On the happy path it replays at zero model calls; anything unexpected halts and asks instead of guessing.
 
-Episodic memory tells an agent what you did. The bigger result is what it lets an agent *do*.
+We measured how much agents overpay to re-derive workflows they've already performed - the **Routine Overhead Ratio** - on weeks of real activity, replicated it on a public web-task dataset, and built a deterministic executor that replays a compiled workflow in a real browser. Instrument, measurements, and executor: [`research/`](research/).
 
-Computer-use agents re-derive every task from scratch - screenshot, reason, act, repeat - even for a routine they've run a hundred times. That re-derivation is where the token cost goes, and it's waste: the routine hasn't changed.
+Passively-captured activity becomes **deterministic action** - and the cheapest computer task is the one an agent never reasons through twice.
 
-Because activity-frames compiles recurring activity deterministically, a routine you've done before becomes a **replayable script** - steps an agent executes directly, grounded by the accessibility tree, with no model in the loop. The agent only picks *which* routine and fills in what's new (message a different person, the same way); the replay itself costs essentially zero tokens.
+## Measured, not guessed
 
-We measured how much agents overpay to re-derive routines they've already performed - the **Routine Overhead Ratio** - on weeks of real activity, replicated it on a public web-task dataset, and built a deterministic executor that replays a compiled routine in a real browser. Instrument, measurements, and executor: [`research/`](research/).
+Agent memory today means conversation memory: what you told the model. What you actually *did* is the missing half - and the hard part is representing it without lying.
 
-Passively-captured activity becomes **deterministic action** - and the cheapest computer task is the one an agent never reasons through twice.
+activity-frames enforces a two-tier contract ([SPEC.md](SPEC.md)):
+
+- **Tier 1, measured (this package):** everything is derivable by deterministic code from capture data - sessions, durations, typed page entities, input volume, coverage gaps. No interpretation, no intent labels. Same input, same output, every time.
+- **Tier 2, inferred (optional extension):** tools that add interpretation must namespace it, tag confidence (`high | medium | speculative`), and link evidence. Facts and guesses can never silently mix.
+
+Every frame carries evidence pointers back to raw capture rows. Every document declares its blind spots. What the system did not see, it says it did not see.
 
 ## Use it from an agent (MCP)
 

pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
 [project]
 name = "activity-frames"
 dynamic = ["version"]
-description = "Episodic memory for AI agents: compile raw screen capture into structured, deterministic activity frames."
+description = "Turn your workday into structured workflows agents can execute. 100% local, served over MCP."
 readme = "README.md"
 license = "MIT"
 requires-python = ">=3.9"

src/activity_frames/sessionize.py

@@ -333,22 +333,27 @@ def app_ledger(db: Database, start_utc: str, end_utc: str,
         by_device.setdefault(f.device, []).append(f)
 
     for frames in by_device.values():
-        cur_session: dict[str, float] = {}
+        cur_app: str | None = None
+        cur_dur: float = 0.0
         for i, f in enumerate(frames[:-1]):
             gap = frames[i + 1].epoch - f.epoch
             if gap > session_gap:
-                cur_session.clear()
+                cur_app = None
+                cur_dur = 0.0
                 continue
             d = min(gap, dwell_cap)
             dwell[f.app] = dwell.get(f.app, 0.0) + d
             if f.window:
                 windows.setdefault(f.app, {})
                 windows[f.app][f.window] = windows[f.app].get(f.window, 0.0) + d
             if d > 0:
-                if cur_session.get(f.app, 0.0) == 0.0:
+                if f.app != cur_app:
+                    cur_app = f.app
+                    cur_dur = d
                     sessions[f.app] = sessions.get(f.app, 0) + 1
-                cur_session[f.app] = cur_session.get(f.app, 0.0) + d
-                longest[f.app] = max(longest.get(f.app, 0.0), cur_session[f.app])
+                else:
+                    cur_dur += d
+                longest[f.app] = max(longest.get(f.app, 0.0), cur_dur)
 
     out = []
     for app, secs in sorted(dwell.items(), key=lambda kv: -kv[1]):

tests/test_sessionize.py

@@ -52,6 +52,64 @@ def test_app_ledger_ordering_and_sessions(fixture_db, day_window):
     assert chrome.sessions >= 2  # linkedin block + github block
 
 
+def test_app_ledger_splits_on_app_switch(tmp_path):
+    """Cursor -> Chrome -> Cursor within 300s: Cursor should be 2 sessions."""
+    import sqlite3
+    from activity_frames.db import Database
+
+    path = tmp_path / "db.sqlite"
+    conn = sqlite3.connect(path)
+    conn.executescript(
+        """
+        CREATE TABLE frames (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            timestamp TIMESTAMP NOT NULL,
+            app_name TEXT, window_name TEXT, focused BOOLEAN,
+            browser_url TEXT, document_path TEXT,
+            device_name TEXT NOT NULL DEFAULT 'monitor_1'
+        );
+        """
+    )
+    # Cursor 10:00:00-10:10:00 (every 20s, 30 frames)
+    # Chrome  10:10:00-10:20:00 (every 20s, 30 frames)
+    # Cursor  10:20:00-10:30:00 (every 20s, 30 frames)
+    base = "2026-08-01T"
+    t = 0
+    for _ in range(30):
+        conn.execute(
+            "INSERT INTO frames (timestamp, app_name, window_name, focused)"
+            " VALUES (?, 'Cursor', 'main.py', 1)",
+            (f"{base}10:{t//60:02d}:{t%60:02d}.000000+00:00",),
+        )
+        t += 20
+    for _ in range(30):
+        conn.execute(
+            "INSERT INTO frames (timestamp, app_name, window_name, focused)"
+            " VALUES (?, 'Google Chrome', 'github.com', 1)",
+            (f"{base}10:{t//60:02d}:{t%60:02d}.000000+00:00",),
+        )
+        t += 20
+    for _ in range(30):
+        conn.execute(
+            "INSERT INTO frames (timestamp, app_name, window_name, focused)"
+            " VALUES (?, 'Cursor', 'server.py', 1)",
+            (f"{base}10:{t//60:02d}:{t%60:02d}.000000+00:00",),
+        )
+        t += 20
+    conn.commit()
+    conn.close()
+
+    db = Database(str(path))
+    ledger = app_ledger(db, f"{base}00:00:00", f"{base}12:00:00")
+    cursor = next(a for a in ledger if a.app == "Cursor")
+    chrome = next(a for a in ledger if a.app == "Google Chrome")
+
+    # Chrome had 1 uninterrupted block, so 1 session.
+    assert chrome.sessions == 1
+    # Cursor was interrupted by Chrome, so 2 distinct sessions.
+    assert cursor.sessions == 2
+
+
 def test_empty_window(fixture_db):
     assert segments(fixture_db, "2020-01-01T00:00:00", "2020-01-02T00:00:00") == []
     cov = coverage(fixture_db, "2020-01-01T00:00:00", "2020-01-02T00:00:00")