Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ci/integration-paths.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ src/tests/eval/test_s58_consumption.py
src/tests/profiles/test_selfhost_profile.py
src/tests/spikes/test_operational_scale.py
src/tests/spine/test_backfill.py
src/tests/spine/test_canonical_bounds_sql.py
src/tests/spine/test_clustering.py
src/tests/spine/test_component_versions.py
src/tests/spine/test_deployment_bootstrap.py
Expand Down
1 change: 1 addition & 0 deletions .github/ci/unit-paths.txt
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ src/tests/core/test_knowledge_planner.py
src/tests/core/test_knowledge_writer.py
src/tests/core/test_section_snap.py
src/tests/core/test_structure_skeleton.py
src/tests/core/test_temporal.py
src/tests/core/test_text_metering.py
src/tests/eval/test_s55_hard_forget.py
src/tests/eval/test_s55_selfhost_restore.py
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@ jobs:
trap - EXIT
test "$(docker compose --env-file .env.example exec -T postgres \
psql -U rememberstack -d rememberstack -Atc \
'SELECT version_num FROM alembic_version')" = 'p9_25_0046'
'SELECT version_num FROM alembic_version')" = 'p9_26_0047'
test "$(docker compose --env-file .env.example exec -T postgres \
psql -U rememberstack -d rememberstack -Atc \
'SELECT count(*) FROM deployments')" = '1'
Expand Down
11 changes: 8 additions & 3 deletions benchmarks/locomo/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# RS-LoCoMo-Full-v21 setup
# RS-LoCoMo-Full-v22 setup

This directory contains the unshipped full-system LoCoMo adapter. It does not vendor or
auto-download LoCoMo. Supply the exact pinned `locomo10.json` only after confirming its
Expand All @@ -16,12 +16,12 @@ The safe first command is local and makes no API or model call:
uv run --extra benchmark python -m benchmarks.locomo prepare \
--dataset /absolute/path/locomo10.json \
--tier smoke \
--protocol full-v21 \
--protocol full-v22 \
--output .benchmark-runs/locomo-smoke
```

The harness validates the pinned bytes, renders session documents, and fingerprints the
eight-question smoke plan. `--protocol` is prepare-only; `full-v21` is the one
eight-question smoke plan. `--protocol` is prepare-only; `full-v22` is the one
current-system protocol, and every later stage reads that immutable choice from
`run.json`. Do not run remote stages until reviewing
[`locomo_benchmark_design.md`](../../plan/designs/locomo_benchmark_design.md).
Expand Down Expand Up @@ -64,6 +64,11 @@ call sees the complete bounded candidate snapshot and returns a supplied
candidate id or `new`. There is no insufficient-evidence result or
confidence-routed frontier call.

V22 fingerprints D107 WP-T.0a: the observation adjudicator and `claims_as_of`
compare canonical half-open bounds (a day is the whole calendar day, an
instant a non-empty point, adjacent units do not overlap); everything else is
v21's.

V21 fingerprints the D106 observation adjudicator: dated events with disjoint
resolved windows never collapse onto or supersede each other (they may only
contradict or stay distinct), a dated event is never `evidence` for an undated
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/locomo/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
"""The pinned RS-LoCoMo-Full-v21 benchmark adapter."""
"""The pinned RS-LoCoMo-Full-v22 benchmark adapter."""
2 changes: 1 addition & 1 deletion benchmarks/locomo/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="python -m benchmarks.locomo",
description=(
"RS-LoCoMo-Full-v21: prepare is local; ingest/answer/judge require "
"RS-LoCoMo-Full-v22: prepare is local; ingest/answer/judge require "
"explicit execution acknowledgements"
),
)
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/locomo/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,10 +140,10 @@ def load_manifest(tier: str) -> QuestionManifest:
)
if manifest.dataset_commit != DATASET_COMMIT:
raise DatasetValidationError(
"manifest dataset commit is not RS-LoCoMo-Full-v21"
"manifest dataset commit is not RS-LoCoMo-Full-v22"
)
if manifest.dataset_sha256 != DATASET_SHA256:
raise DatasetValidationError("manifest dataset hash is not RS-LoCoMo-Full-v21")
raise DatasetValidationError("manifest dataset hash is not RS-LoCoMo-Full-v22")
actual = item_ids_hash(item_ids=manifest.item_ids)
if actual != manifest.item_ids_sha256:
raise DatasetValidationError(
Expand Down
10 changes: 5 additions & 5 deletions benchmarks/locomo/model.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Typed values for the full-system RS-LoCoMo-Full-v21 protocol."""
"""Typed values for the full-system RS-LoCoMo-Full-v22 protocol."""

from __future__ import annotations

Expand Down Expand Up @@ -28,8 +28,8 @@
Category = Literal[1, 2, 3, 4, 5]
RetainedCategory = Literal[1, 2, 3, 4]
Tier = Literal["smoke", "development", "publication"]
ProtocolKey = Literal["full-v21"]
ProtocolName = Literal["RS-LoCoMo-Full-v21"]
ProtocolKey = Literal["full-v22"]
ProtocolName = Literal["RS-LoCoMo-Full-v22"]
SourceTimezoneBasis = Literal["assumed_utc"]
AnswerAgentModel = Literal["openai/gpt-5.6-luna"]
JudgeModel = Literal["openai/gpt-5.6-luna"]
Expand Down Expand Up @@ -120,7 +120,7 @@ class QuestionManifest(FrozenModel):
class RunConfiguration(FrozenModel):
"""Immutable identity of one prepared benchmark run."""

protocol_name: ProtocolName = "RS-LoCoMo-Full-v21"
protocol_name: ProtocolName = "RS-LoCoMo-Full-v22"
adapter_version: NonEmpty
prepared_at: datetime
repository_revision: NonEmpty
Expand Down Expand Up @@ -393,7 +393,7 @@ class SessionDiagnosticSummary(FrozenModel):
class RunSummary(FrozenModel):
"""Publication-ready local aggregate with no hidden denominator."""

protocol_name: ProtocolName = "RS-LoCoMo-Full-v21"
protocol_name: ProtocolName = "RS-LoCoMo-Full-v22"
protocol_fingerprint: NonEmpty
tier: Tier
questions: int = Field(ge=1)
Expand Down
14 changes: 7 additions & 7 deletions benchmarks/locomo/protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@
from rememberstack.model import Envelope
from rememberstack.model import ToolDescriptor

PROTOCOL_NAME: Final = "RS-LoCoMo-Full-v21"
DEFAULT_PROTOCOL_KEY: Final = "full-v21"
ADAPTER_VERSION: Final = "locomo-full-adapter-2026.09-temporal-gate-v21"
PROTOCOL_NAME: Final = "RS-LoCoMo-Full-v22"
DEFAULT_PROTOCOL_KEY: Final = "full-v22"
ADAPTER_VERSION: Final = "locomo-full-adapter-2026.09-canonical-bounds-v22"
MAX_TOOL_CALLS: Final = 8
MAX_AGENT_CALLS: Final = 9
ANSWER_READER_RETRY_BUDGET: Final = 2
Expand Down Expand Up @@ -74,7 +74,7 @@
"no-types-1:binary-t4-1:document-t0-1"
),
"adjudicate_observations": (
"e3-obs-flush-2026.09a:temporal-gate-1:claim-fanout-1:entity-fanout-1"
"e3-obs-flush-2026.09b:canonical-bounds-1:temporal-gate-1:claim-fanout-1:entity-fanout-1"
),
"adjudicate_supersession": "adjudicator-2026.07b:temp0-1",
"embed_claim": "p1-embed-claims-2026.07",
Expand Down Expand Up @@ -211,8 +211,8 @@ class LoCoMoProtocol:
answer_word_cap: int | None = None


_FULL_V21 = LoCoMoProtocol(
key="full-v21",
_FULL_V22 = LoCoMoProtocol(
key="full-v22",
name=PROTOCOL_NAME,
answer_agent_model=ANSWER_AGENT_MODEL,
judge_model=JUDGE_MODEL,
Expand All @@ -234,7 +234,7 @@ class LoCoMoProtocol:
)

PROTOCOL_REGISTRY: Final[Mapping[ProtocolKey, LoCoMoProtocol]] = MappingProxyType(
{_FULL_V21.key: _FULL_V21}
{_FULL_V22.key: _FULL_V22}
)


Expand Down
10 changes: 5 additions & 5 deletions benchmarks/locomo/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ def ingest_sample(
_require_current_ingest_bindings(model_bindings=build.model_bindings)
if build.document_binding_generation != EXPECTED_DOCUMENT_BINDING_GENERATION:
raise ExecutionGuardError(
"deployment document binding generation differs from RS-LoCoMo-Full-v21"
"deployment document binding generation differs from RS-LoCoMo-Full-v22"
)
_require_current_query_surface(context=context, client=client)
_require_exact_live_ingests(
Expand Down Expand Up @@ -570,7 +570,7 @@ def answer_sample(
):
raise ExecutionGuardError(
"the deployment did not report the exact completed"
" RS-LoCoMo-Full-v21 pipeline, live graph, and fresh P3 projection"
" RS-LoCoMo-Full-v22 pipeline, live graph, and fresh P3 projection"
)
_require_serving_revision(context=context, readiness=readiness)
prior_readiness = context.state.readiness.get(sample_id)
Expand Down Expand Up @@ -1177,7 +1177,7 @@ def _validate_run(
"""Recompute immutable run identity before any local or remote stage."""
selected_protocol = protocol_for_name(configuration.protocol_name)
if configuration.dataset_sha256 != DATASET_SHA256:
raise BenchmarkRunError("run dataset hash is not RS-LoCoMo-Full-v21")
raise BenchmarkRunError("run dataset hash is not RS-LoCoMo-Full-v22")
if item_ids_hash(item_ids=manifest.item_ids) != manifest.item_ids_sha256:
raise BenchmarkRunError("run manifest item hash changed")
if manifest_bytes_hash(manifest=manifest) != configuration.manifest_sha256:
Expand All @@ -1187,7 +1187,7 @@ def _validate_run(
if manifest.tier != configuration.tier:
raise BenchmarkRunError("run manifest tier changed")
if configuration.dataset_commit != DATASET_COMMIT:
raise BenchmarkRunError("run dataset commit is not RS-LoCoMo-Full-v21")
raise BenchmarkRunError("run dataset commit is not RS-LoCoMo-Full-v22")
if configuration.adapter_version != ADAPTER_VERSION:
raise BenchmarkRunError("run adapter version differs from current code")
if _models_hash(values=documents) != configuration.documents_sha256:
Expand Down Expand Up @@ -1445,7 +1445,7 @@ def _require_current_ingest_bindings(*, model_bindings: dict[str, str]) -> None:
if model_bindings.get(name) != expected.get(name)
)
raise ExecutionGuardError(
"deployment ingest model bindings differ from RS-LoCoMo-Full-v21: "
"deployment ingest model bindings differ from RS-LoCoMo-Full-v22: "
+ ", ".join(mismatches)
)

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/locomo/sharding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ as source-derived data.
| --- | ---: | --- |
| `LOCOMO_PYTHON` | `.venv/bin/python` | repository virtual-environment Python |
| `LOCOMO_TIER` | `publication` | prepared manifest tier |
| `LOCOMO_PROTOCOL` | `full-v21` | prepare-time protocol key |
| `LOCOMO_PROTOCOL` | `full-v22` | prepare-time protocol key |
| `LOCOMO_MOUNT_ROOT` | `$RUN_DIR/.mounts` | host/container-identical P3 mount root |
| `LOCOMO_MAX_DOCUMENTS` | `100` | per-sample ingest authorization |
| `LOCOMO_MAX_QUESTIONS` | `1540` | run-absolute answer item authorization |
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/locomo/sharding/run_shard.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ run_dir=$2
dataset_path=$3
python_bin=${LOCOMO_PYTHON:-.venv/bin/python}
tier=${LOCOMO_TIER:-publication}
protocol=${LOCOMO_PROTOCOL:-full-v21}
protocol=${LOCOMO_PROTOCOL:-full-v22}
mount_root=${LOCOMO_MOUNT_ROOT:-$run_dir/.mounts}
max_documents=${LOCOMO_MAX_DOCUMENTS:-100}
max_questions=${LOCOMO_MAX_QUESTIONS:-1540}
Expand All @@ -50,7 +50,7 @@ export GOOGLE_APPLICATION_CREDENTIALS=${LOCOMO_GCP_CREDENTIALS_FILE:-/etc/rememb
export GOOGLE_API_CERTIFICATE_CONFIG=${LOCOMO_GCP_CERTIFICATE_CONFIG_FILE:-/etc/rememberstack/locomo-gcs/certificate-config.json}
export GOOGLE_API_USE_CLIENT_CERTIFICATE=true

# RS-LoCoMo-Full-v21's non-secret ingest identity. Override ambient self-host
# RS-LoCoMo-Full-v22's non-secret ingest identity. Override ambient self-host
# defaults so every shard runs the exact Luna/Qwen pipeline the protocol checks.
export REMEMBERSTACK_STRUCTURER_MODEL=openai/gpt-5.6-luna
export REMEMBERSTACK_SKELETON_CHECK_MODEL=openai/gpt-5.6-luna
Expand Down
23 changes: 17 additions & 6 deletions plan/designs/locomo_benchmark_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,19 @@
> assured surface rolls this protocol; the sequencing is
> `plan/plans/temporal_clocks.md`. Contract: `temporal_clocks_design.md` §6–§8.

> **Binding D106 amendment (2026-09-03).** The current protocol is
> `RS-LoCoMo-Full-v21`. It retains v20's dataset, rendered documents, models,
> **Binding D107 amendment, WP-T.0a (2026-09-03).** The current protocol is
> `RS-LoCoMo-Full-v22`. It retains v21's dataset, rendered documents, models,
> tools, budgets, prompts, and scoring. Its pinned `adjudicate_observations`
> component version now carries canonical half-open bounds (D107 §5): the
> observation adjudicator compares a day as the whole calendar day and an
> instant as a non-empty point, and `claims_as_of` finds a day-precision
> claim from an intraday window. Ingestion provenance, the library's
> `claims_as_of` result set, protocol identity, and fingerprint roll; the
> benchmark's exposed read-tool set, retry, model-effort, and call-budget
> behavior are unchanged.

> **Historical D106 amendment (2026-09-03; superseded by D107 WP-T.0a).** The
> D106 protocol was `RS-LoCoMo-Full-v21`. It retained v20's dataset, rendered documents, models,
> tools, budgets, answer and judge prompts, counterfactual and complete-answer
> instructions, content-before-`Unknown` harness guard, and no-review scoring
> rule. Its pinned `adjudicate_observations` component version now carries the
Expand Down Expand Up @@ -98,7 +109,7 @@ and spend ceiling.
## 2. Fixed protocol

```text
protocol RS-LoCoMo-Full-v21
protocol RS-LoCoMo-Full-v22
dataset commit 3eb6f2c585f5e1699204e3c3bdf7adc5c28cb376
dataset SHA-256 79fa87e90f04081343b8c8debecb80a9a6842b76a7aa537dc9fdf651ea698ff4
categories 1, 2, 3, 4
Expand Down Expand Up @@ -670,7 +681,7 @@ compatibility form. The response contains:
same-snapshot proven-absent-anchor execution checks when live graph is required;
- an overall `ready` that is the conjunction of the requested capabilities;
- every non-secret ingestion/query model binding; and
- the non-secret `document_binding_generation`, which Full-v21 requires to be
- the non-secret `document_binding_generation`, which Full-v22 requires to be
exactly `document-t0-v1` and stores in `run.json` plus the protocol
fingerprint.

Expand Down Expand Up @@ -816,11 +827,11 @@ Local preparation:
uv run --extra benchmark python -m benchmarks.locomo prepare \
--dataset /absolute/path/locomo10.json \
--tier smoke \
--protocol full-v21 \
--protocol full-v22 \
--output .benchmark-runs/locomo-smoke
```

`--protocol` exists only on `prepare`. The sole choice is `full-v21`; ingest,
`--protocol` exists only on `prepare`. The sole choice is `full-v22`; ingest,
answer, judge, and summarize read it from the prepared run and expose no
protocol override.

Expand Down
11 changes: 7 additions & 4 deletions plan/designs/temporal_clocks_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,12 +348,15 @@ ends as +∞. A caller's inclusive request `claims_as_of(from, to)` converts
to `[from, to + 1 µs)`, so `from == to` is a point-in-time query, not an
empty one.

**Where it runs.** The function ships as one immutable SQL function in the
`memory_v1` query space (`memory_v1.canonical_bounds`) with a companion
**Where it runs.** The engine's own SQL calls two IMMUTABLE public-schema
functions, `claim_canonical_start(from, precision)` and
`claim_canonical_end(from, until, precision)`, backed by an expression index
so the as-of scan stays indexed; the query space exposes the same
canonicalisation as `memory_v1.canonical_bounds` with a companion
`claims_canonical` view exposing `canon_start`/`canon_end` beside the raw
columns, so saved examples, open SQL, the catalog metadata and the
open-query prose use the same canonicalisation as the engine; the Python
side calls the same definition. **Fact windows are stored canonical** —
open-query prose use it too; the Python side (`core/temporal.py`) mirrors
the same table and a test pins the twins equal. **Fact windows are stored canonical** —
`valid_from`/`valid_until` and `occurs_from`/`occurs_until` are written from
`canonical_bounds` at seeding, so every fact predicate the engine already
has (`valid_until > :as_of`, `tstzrange(valid_from, valid_until)` in the
Expand Down
Loading
Loading