#611: add context window to memory extractors + environment field + confidence honesty - #620
Open
NOVA-Openclaw wants to merge 8 commits into
Open
#611: add context window to memory extractors + environment field + confidence honesty#620NOVA-Openclaw wants to merge 8 commits into
NOVA-Openclaw wants to merge 8 commits into
Conversation
- Add max_prior_messages (default 10) and context_window_enabled config keys - Create shared context_window_helper for config, truncation, formatting - extract_memories.py reads EXTRACTION_CONTEXT_JSON, builds prompt guardrail - Add environment field to events schema and INSERT - Add apply_confidence_honesty cap for short no-context messages - Add events.environment and events_archive.environment columns to schema.sql
- Hot-reload max_prior_messages and context_window_enabled from config - loadPriorContext fetches last N prior messages from same channel_session - Pass EXTRACTION_CONTEXT_JSON to extract_memories.py - Graceful degradation: context fetch failure logs warning and proceeds no-context
- extraction-replay.sh reconstructs same-session prior context from channel_transcripts - memory-catchup.sh calls extract_memories.py directly with EXTRACTION_CONTEXT_JSON - Both scripts read shared max_prior_messages / context_window_enabled config
- Add 25 pytest cases covering config, truncation, prompt, confidence, environment - Add blockhenge-regression and cross-channel-privacy fixtures - Add REPLAY-PROCEDURE.md for LLM-behavioral TCs - Update memory-extraction-pipeline.md with context window docs
Keep event_date in vals_list unconditionally and use COALESCE(%s::timestamptz, NOW()) so placeholders stay symmetric when event_date is None and src_timestamp is empty. Adds mock-free placeholder count regression guards.
Gates context assembly in add_to_cache_and_get_context() with is_context_window_enabled(), matching extraction-replay.sh and handler.ts. Cache/duplicate detection still runs; context fetch is skipped when disabled.
Uses a scratch-schema copy of public.events and a fresh subprocess to exercise store_extracted with (a) date omitted + empty src_timestamp and (b) same with environment populated. Skipped when nova_memory is unavailable.
…e honesty, fix replay log-string reference - memory/docs/memory-extraction-pipeline.md: intro note, path diagram, §3 Components, troubleshooting table, and Next Steps all still described memory-catchup.sh's pre-#611 broken process-input.sh call path as current behavior despite the doc's own opening note claiming it was fixed. Corrected all five locations to describe the AS-BUILT #611 behavior (memory-catchup.sh calls extract_memories.py directly, passing EXTRACTION_CONTEXT_JSON). - Added a new "Confidence Honesty (#611)" subsection documenting both the prompt-level CONFIDENCE GUIDANCE change and the apply_confidence_honesty() post-hoc capping safety net (short-message + no-context gate, trust-level-based cap via get_initial_confidence(), owner-entity short-circuit) — previously undocumented despite being a #611 deliverable (C7/E1-E6). - memory/ARCHITECTURE.md, memory/INSTALLATION.md, memory/README.md: same process-input.sh drift existed in three more places; corrected to match. memory/README.md's older 20-message rolling-context-window section (2026-02-07) also cross-referenced against the #611 config-driven window to clarify they are related-but-distinct mechanisms, and its stale extract-memories.sh/ store-memories.sh script-update bullets were annotated as historical (both scripts were removed in #174 and do not exist). - tests/issue-611/REPLAY-PROCEDURE.md: fixed the per-turn-hook verification step's log-string reference (handler.ts logs '[memory-extract] Loaded prior context messages' with a count field, not 'Loaded N prior message(s) as context'); added an explicit coverage caveat cross-referencing step8-qa-validation.md so this manual procedure isn't mistaken for automated coverage. Per Step 8 QA (Gem, PASS-WITH-NOTES): corrects doc-source drift found during the #611 documentation pass. No test-coverage overclaims (e.g. '125/125') existed in-repo docs at 8edc8c5; none required correction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #611.
Summary
Adds a ~10-message context window to both memory extractors (per-turn hook + daily/REM batch) with extract-current-only guardrails, environment field extraction, and confidence honesty. Fixes a pre-existing event-INSERT drop bug surfaced during QA.
Changes
feat(memory): context window, environment field, confidence honesty (57b33b2)feat(memory-extract): per-turn context assembly in hook (d255ef8)feat(memory-catchup,replay): batch/replay context assembly (1f2ee32)test(docs): automated tests, fixtures, pipeline docs (e462810)fix(memory): restore event_date COALESCE guard in events INSERT (88d76ed)fix(memory-catchup): honor context_window_enabled gate (103e137)test(memory): real-DB integration test for events INSERT (8edc8c5)docs(memory): correct process-input.sh drift, document confidence honesty (f3c901a)Test results
Coverage honesty
Actual #611 coverage = 29 automated tests (27 unit + 2 integration) + manual LLM-behavioral replay. The 94 turn-context tests are a separate unrelated suite.
Follow-ups (non-blocking)
SE Run #718.