Architecture: Deepen Persist Engine seam and unify persistence lifecycle in Queue Manager

[jonbaldie]

Candidate Summary

• Recommendation Strength: Strong
• Modules Involved: src/persist.ts, src/manager.ts, main.ts
• Domain Context: Queue Manager, Persist Engine, Queue, Payload
• Architecture Vocabulary: module, interface, depth, seam, adapter, leverage, locality

Problem & Evidence

The persistence seam between Queue Manager, Persist Engine (QueueStore), and main.ts exhibits significant architectural friction across three areas:

1. Filesystem details leaking into the QueueStore interface:
   QueueStore declares dir(dir: string): void.
   MemoryStore (an in-memory adapter) is forced to implement a dummy method dir(): void {}.
   In FileStore, this creates temporal coupling: FileStore is instantiated with an empty directory, and callers must remember to invoke .dir(CONFIG.persistDir) before doing storage operations.

2. Adapter bypass and boolean flag in QueueManager:
   QueueManager constructor accepts both store: QueueStore<T> AND persistEnabled?: boolean:

constructor(store: QueueStore<T>, queueDepthLimit?: number, queueCountLimit?: number, persistEnabled?: boolean)

Inside QueueManager.enqueue and dequeue:

if (this.persistEnabled) {
    this.store.saveEvent(name, payload, true);
}

When persistence is disabled, main.ts passes new Persistency.MemoryStore, yet QueueManager bypasses the store via if (this.persistEnabled). MemoryStore is rendered redundant because QueueManager ignores it when persistEnabled is false.

3. Type-sniffing / instanceof inspection in main.ts:
   In main.ts:

if (PERSIST_ENGINE instanceof Persistency.FileStore) {
    writeLog("Loading in data from persist.dat...\n");
    MANAGER.load();
}
...
if (PERSIST_ENGINE instanceof Persistency.FileStore) {
    writeLog("Flushing data to persist.dat...\n");
    MANAGER.save();
}

main.ts violates polymorphism at the QueueStore seam by checking whether PERSIST_ENGINE is an instanceof FileStore before deciding whether to call MANAGER.load() and MANAGER.save().

Test Evidence (/deintrovert-tests)

In tests/persist_test.ts and tests/manager_test.ts, tests must manually orchestrate dir() calls and pass both new MemoryStore() and persistEnabled: false or true. Tests observe that passing a store does not mean persistence is active unless the separate boolean flag is also set.

The Deletion Test

• Delete dir() from QueueStore: Directory configuration moves to FileStore's constructor (new FileStore(persistDir)). The QueueStore interface shrinks, temporal coupling disappears, and MemoryStore no longer needs a dummy method.
• Delete persistEnabled boolean from QueueManager: QueueManager always writes to its store. If persistence is off, a MemoryStore or NoopStore is passed, satisfying the seam without conditional checks inside QueueManager.
• Delete instanceof FileStore checks from main.ts: QueueManager coordinates its persistence lifecycle uniformly across all adapters. For in-memory or empty stores, loadState() simply returns [] without error.

Proposed Change

1. Remove dir() from the QueueStore interface. Pass directory: string directly to the FileStore constructor.
2. Remove persistEnabled from QueueManager constructor. QueueManager always calls this.store.saveEvent(). Provide a MemoryStore (or NoopStore) when persistence is disabled.
3. Remove instanceof FileStore checks in main.ts. QueueManager.load() runs unconditionally on startup; QueueManager.save() (or a unified flush()/close()) runs on graceful shutdown.

Benefits

• Leverage: Callers configure storage once at construction. The QueueStore interface is strictly focused on event persistence (saveEvent, replace, loadState, close).
• Locality: Persistence lifecycle management concentrates inside Queue Manager and Persist Engine, removing procedural orchestration from main.ts.
• Adapters: Both FileStore and MemoryStore become true polymorphic adapters satisfying a clean seam.

[jonbaldie]

This was generated by AI during triage.

Verification: Confirmed in src/persist.ts, src/manager.ts, and main.ts. Currently, QueueStore declares dir(dir: string): void, forcing MemoryStore to implement a no-op method and introducing temporal coupling in FileStore. In src/manager.ts, the persistEnabled flag causes QueueManager to bypass QueueStore when false rather than relying on polymorphic store implementations. In main.ts, PERSIST_ENGINE instanceof Persistency.FileStore checks gate MANAGER.load() on startup and MANAGER.save() on shutdown, breaking polymorphism.

Redundancy check: Searched src/persist.ts, src/manager.ts, main.ts, and tests/persist_test.ts. The dir() method, persistEnabled flag, and instanceof guards remain in place. No matching .out-of-scope/ record exists.

Agent Brief

Category: enhancement
Summary: Deepen the Persist Engine seam by removing dir() from QueueStore, eliminating persistEnabled branching in QueueManager, and removing instanceof guards in main.ts.

Current behavior:
QueueStore exposes dir(), requiring MemoryStore to provide a dummy implementation and forcing callers to invoke dir() after instantiation. QueueManager accepts a persistEnabled boolean and guards every saveEvent call with if (this.persistEnabled), ignoring the provided store when false. main.ts inspects PERSIST_ENGINE instanceof Persistency.FileStore before calling MANAGER.load() and MANAGER.save().

Desired behavior:
The QueueStore interface is strictly focused on storage events without filesystem path mechanics (saveEvent, saveBatch, replace, loadState, clear, close). FileStore receives its directory path via its constructor (new FileStore(directory)). QueueManager delegates event persistence unconditionally to its QueueStore adapter; when persistence is disabled, a MemoryStore is supplied. main.ts invokes MANAGER.load() on startup and MANAGER.save() (and store close()) on shutdown uniformly across store adapters.

Key interfaces:

• QueueStore<T> in src/persist.ts: remove dir(dir: string): void.
• FileStore<T> in src/persist.ts: constructor accepts directory?: string (normalizing trailing slash), eliminating the need for post-construction dir() calls.
• QueueManager<T> in src/manager.ts: remove persistEnabled property and constructor parameter; unconditionally call this.store.saveEvent().
• main.ts: instantiate FileStore with CONFIG.persistDir; invoke MANAGER.load() and MANAGER.save() without instanceof type checks.

Acceptance criteria:
[ ] dir() is removed from QueueStore and MemoryStore.
[ ] FileStore accepts its target directory at construction time.
[ ] persistEnabled is removed from QueueManager constructor and methods; QueueManager delegates persistence directly to store.
[ ] instanceof Persistency.FileStore checks in main.ts are removed; startup load and shutdown save execute polymorphically.
[ ] All existing unit tests and integration tests in tests/persist_test.ts, tests/manager_test.ts, and tests/e2e_test.ts pass with updated constructors.
[ ] In-memory queue operations with MemoryStore record events correctly without filesystem access.

Out of scope:

• Changing the disk serialization format or snapshot mechanics in persist.dat.
• Changing config parsing or CLI flags for persistence.
• Modifying HTTP handler routing or queue limits.