Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 0 additions & 26 deletions .changeset/feat-f8-event-loop-lag-observability.md

This file was deleted.

5 changes: 0 additions & 5 deletions .changeset/fix-f3-publish-drainer.md

This file was deleted.

25 changes: 0 additions & 25 deletions .changeset/fix-f7-destroy-tombstone.md

This file was deleted.

15 changes: 0 additions & 15 deletions .changeset/fix-f9d-acquire-jitter.md

This file was deleted.

16 changes: 0 additions & 16 deletions .changeset/fix-f9e-dir-scan-index.md

This file was deleted.

76 changes: 76 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,81 @@
# Changelog

## 0.10.0

### Minor Changes

- [#157](https://github.com/Hazzng/sql-fs/pull/157) Thanks [@Hazzng](https://github.com/Hazzng)! - feat(observability): event-loop lag monitoring for the Redis leases (F8).

The exec-lock writer lease, the RW-lock writer flag, and the RW-lock reader ZSET
scores are all kept alive by `setTimeout` heartbeats that silently assume timers
fire on schedule. A long event-loop stall (a V8 GC pause or a pathological
synchronous bash stretch) can fire a renewal past the lease, voiding it — Lock 3
keeps Postgres consistent, so this was always an observability gap, not a
correctness bug, but nothing measured it.

New `src/api/event-loop-monitor.ts` (purely observational, no behavior change):

- A `perf_hooks.monitorEventLoopDelay` histogram started at boot, sampled every
`EVENT_LOOP_MONITOR_INTERVAL_MS` (default 10s) and logged as
`event:"event_loop_lag"` (`p50Ms`/`p99Ms`/`maxMs`/`meanMs`), then reset.
- Per-heartbeat gap measurement wired into all three lease sites: each heartbeat
reports actual-minus-expected fire time as `event:"heartbeat_gap"` at
`severity:"warn"` (gap > renewMs) or `"critical"` (gap > leaseMs), tagged with
the lock kind (`exec`/`rw-writer`/`rw-reader`) and key.

Alert thresholds are documented in DEVELOPER.md ("Lock observability"). End-to-end
smoke tests reproduce a >lease stall on each lease and assert the critical
heartbeat_gap fires (with a no-stall control proving no false positives).

- [#156](https://github.com/Hazzng/sql-fs/pull/156) Thanks [@Hazzng](https://github.com/Hazzng)! - Heal stranded cross-replica version publishes after a Redis INCR failure (F3): a background drainer and reap-time best-effort publish flush the bump even if no further client traffic arrives or the session is idle-evicted.

- [#155](https://github.com/Hazzng/sql-fs/pull/155) Thanks [@Hazzng](https://github.com/Hazzng)! - fix(session): destroy now reaches warm sessions on other replicas (F7).

Destroying a sandbox on one replica previously left warm sessions on other
replicas serving ghost state: a written session would reload a deleted tree
into an empty pathCache (surfacing as a non-zero exit + garbage stderr inside an
HTTP 200 exec), and a never-written session would never reload at all because
the deleted version key read as 0 and matched its `lastSeenVersion === 0`.

Two layered fixes:

- Primary (Redis-independent): `SqlFs.reload()` now detects a zero-row
`loadAllPaths` — which for a live sandbox always returns at least its root dir
— and throws a typed `ESANDBOXGONE` instead of installing an empty pathCache.
The session manager catches it, tears the stale warm session down (drops it
from the pool and disconnects the per-session Postgres pool), and surfaces a
clean `ENOENT` → 404.
- Secondary (tombstone): `destroy` now writes a distinct `DESTROYED` sentinel to
the version key (with the version-key TTL) instead of deleting it.
`ensureFreshCache` recognises the sentinel before the numeric parse and tears
the session down — covering the never-written variant. Re-creating a
tombstoned sandbox clears the sentinel and starts cleanly at version 0.

- [#153](https://github.com/Hazzng/sql-fs/pull/153) Thanks [@Hazzng](https://github.com/Hazzng)! - fix(lock): add bounded jitter + tunable retry to the distributed acquire loops (F9d, [#141](https://github.com/Hazzng/sql-fs/issues/141))

The distributed exec lock and RW lock polled Redis on a flat `acquireRetryMs`
(default 50 ms) interval, leaving competing replicas phase-aligned so a
cross-replica writer could be repeatedly passed over (bounded by
`acquireTimeoutMs`, then 503). Every acquire/drain poll now sleeps a jittered
`retryMs/2 + random()*retryMs/2` (range `[retryMs/2, retryMs]`) to
de-synchronize pollers. The retry interval is now configurable via
`REDIS_EXEC_LOCK_ACQUIRE_RETRY_MS` (previously hardcoded — `server.ts` omitted
it). Circuit-breaker / error-budget behavior is unchanged. The FIFO ZSET ticket
queue is deferred as a follow-up.

- [#154](https://github.com/Hazzng/sql-fs/pull/154) Thanks [@Hazzng](https://github.com/Hazzng)! - perf(cache): O(1) pathCache byte accounting to avoid full-map scans (F9e, [#142](https://github.com/Hazzng/sql-fs/issues/142))

SqlFs now maintains an incremental `#pathCacheBytes` counter, adjusted on
every pathCache set/delete and reset on `reload()`/`ready()`, and exposes
`getPathCacheBytes()`. SessionManager's path-cache memory budget calls it
instead of re-walking the entire pathCache (`Σ path.length + 100`) on every
dirty exec. The value equals the previous full-walk exactly. Falls back to
the full walk for backends that do not expose the counter.

The `#childrenByParent` children index (part B of [#142](https://github.com/Hazzng/sql-fs/issues/142)) is deferred to a
follow-up; it is benchmark-gated and (A) delivers the higher-value, lower-risk
win without touching readdir correctness.

## 0.9.0

### Minor Changes
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
</p>

<p align="center">
<img src="https://img.shields.io/badge/version-0.9.0-blue" alt="Version" />
<img src="https://img.shields.io/badge/version-0.10.0-blue" alt="Version" />
<img src="https://img.shields.io/badge/node-%3E%3D22-brightgreen?logo=node.js&logoColor=white" alt="Node.js" />
<img src="https://img.shields.io/badge/TypeScript-strict-3178c6?logo=typescript&logoColor=white" alt="TypeScript" />
<img src="https://img.shields.io/badge/just--bash-3.0.1-f59e0b" alt="just-bash" />
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sql-fs-api",
"version": "0.9.0",
"version": "0.10.0",
"description": "Persistent filesystem backend (Postgres) + HTTP/MCP API for just-bash sandboxes",
"type": "module",
"license": "MIT",
Expand Down
2 changes: 1 addition & 1 deletion src/api/openapi-spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ export const openapiSpec = {
openapi: "3.0.0",
info: {
title: "SQL-FS API",
version: "0.9.0",
version: "0.10.0",
description: "Persistent filesystem backend + HTTP/MCP API for just-bash sandboxes. Backed by Postgres.",
},
servers: [{ url: "/v1", description: "API v1" }],
Expand Down
Loading