design: agent-visible source access (D107), originals on the navigation path (D108) - #361
Open
fazpu wants to merge 1 commit into
Open
design: agent-visible source access (D107), originals on the navigation path (D108)#361fazpu wants to merge 1 commit into
fazpu wants to merge 1 commit into
Conversation
…ation path (D108) D65 bound how a media source is derived and pointed at, and left the last step -- an unmounted agent actually perceiving it -- unbuilt. D51 mounted originals but kept them off the navigation path. Both gaps are about the same thing: the system derives text from media well and makes the media itself awkward to reach. D107 binds `source_open(version_id, representation_id?, locator?, accept?)` as a direct retrieval primitive sharing hydrate's resolution path, authorization, and audit. It returns the D49 envelope at evidence grain with perceptual content in the client's native channels and a content_manifest[] pairing each block to its role, hash, original/agent_rendition origin, transforms, locator, and untrusted label. There is no caller-selected view mode: the server decides original-versus-rendition and the result declares which. Formats are negotiated by caller declaration, because no protocol declares them -- MCP has no client capability announcing accepted tool-result MIME types. The guarantee is scoped honestly: it covers server-side delivery only. MCP leaves host-to-model forwarding undefined, so the audit records that content of a stated kind, size, and hash reached a named principal, never that a model perceived it. End-to-end perception is proved by evaluation, bound as media_design section 10 spike 12b. D107 also replaces D65's exclusive picture-versus-document discriminator with lane emphasis, and adds four independent readiness dimensions. D108 withdraws D51's off-navigation-path clause. Every original is reachable at `<doc_id>/<content_hash>/original.<ext>`, one level above the representation directory, so all representations of the same bytes share one copy and a re-conversion never duplicates it. D51 already reached this conclusion for media and stopped short: its own Context says the original IS the artifact and a transcript is the lossy rendering a multimodal agent needs to bypass, and storage_routing.py encodes that in code. The barrier it kept was navigational, and models that read video, audio, and images directly are now ordinary. The read record follows D51's own basis and is unchanged where the object store provides it -- a gcsfuse read is a GCS read under Cloud Audit Logs. AuditedRawReader remains the audited path for API/CLI byte fetches. A deployment backing raw with a plain local filesystem does not record browse-path reads; that is stated rather than left to surprise an operator. For a mounted agent D108 removes the need for source_open entirely. It remains the only path for an unmounted agent, and the locator-scoped path for large media. Also: e0_files_design's router is corrected to match lane emphasis, and the media_converter_routes_bound acceptance check is amended so a correct lane-emphasis implementation passes rather than fails it. Validation: `eval-banana validate` -- 83 checks valid. `git diff --check` clean. Reviewed by codex over three rounds against this repository's CLAUDE.md rules. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019NXUunrARvBWBteHSUDCYR
fazpu
force-pushed
the
design/source-open-agent-visual
branch
from
September 3, 2026 13:13
31c5608 to
45c0973
Compare
source_open serves perception (D106)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two gaps, both about the same thing: the system derives text from media well and makes the media itself awkward to reach.
D65 bound how a media source is derived and pointed at, and left the last step — an unmounted agent actually perceiving it — unbuilt.
media_design.md§4 named "asource_openprimitive" once as an alternative; §8 andretrieval_design.md§3/§7 all selectedhydrate depth=bytes, whose bound result is a seekable byte segment. Nothing ever required the result to be something a model can consume.D51 mounted originals but kept them off the navigation path, reachable only by resolving an explicit pointer. That was right when a harness could do nothing with a video but read its transcript.
D107 —
source_openserves perceptionsource_open(version_id, representation_id?, locator?, accept?), a direct retrieval primitive (D87's four assured operations unchanged) sharinghydrate depth=bytes's resolution path, authorization, and audit. Returns the D49 envelope atevidencegrain with perceptual content in the client's native channels, plus acontent_manifest[]pairing each block to its role, hash,original/agent_renditionorigin, transforms, locator, and untrusted label.acceptstates it; an empty intersection with the served set is a typedboundary.content_manifest[], notparts[]— D87 removedEnvelope.partsas envelope-of-envelopes composition. This names one response's own content blocks and composes nothing.D108 — originals sit on the navigation path
Every original is reachable at
<doc_id>/<content_hash>/original.<ext>, one level above the representation directory, so all representations of the same bytes share one copy and a re-conversion never duplicates it. D51's off-navigation-path clause is withdrawn; everything else in D51 stands.D51 already reached this conclusion and stopped one step short. Its own Context says that for whole-file media "the original is the artifact" and "a transcript is precisely the lossy rendering a multimodal agent needs to bypass";
storage_routing.pyencodes the same judgement in code. What it kept off-path was navigation, which it treats as distinct from reachability ("promotion ≠ reachability"). Models that read video, audio, and images directly are now ordinary, so the barrier costs more than it protects — it asks an agent to resolve a pointer to reach the one artifact it handles best, on a filesystem the same design tells it to prefer.Scope is every original, not only media: one rule is easier to teach a cold agent, and the cost that motivated the split is a storage-class cost, already solved by mime-routed classes.
The read record follows D51's own basis and is unchanged where the object store provides it — a gcsfuse read is a GCS read under Cloud Audit Logs.
AuditedRawReaderremains the audited path for API/CLI byte fetches. A deployment backing raw with a plain local filesystem does not record browse-path reads; stated here rather than left to surprise an operator.For a mounted agent, D108 removes the need for
source_openentirely — it opens the file.source_openremains the only path for an unmounted agent (most agents against a managed deployment) and the locator-scoped path for large media.Scope
decisions.md(D107, D108; amendment notes on D51, D65, D38),media_design.md(§1, §2, §4, §4a, §4b, §8, §9, §10),retrieval_design.md(§3, §5, §7, §8),e0_files_design.md(router),media_converter_routes_bound.yaml. No code.Validation
eval-banana validate— 83 checks valid.git diff --checkclean.Limitations — reviewer decisions
agent_view_readyreads "safe-decode admission at ingest", which has no persisted authority or schema yet.content_manifest[]sits close to what D87 removed. The rationale is in §4a and D107; a reviewer may want D87 amended explicitly.locomo_benchmark_design.md§6 deliberately untouched. Adding a public operation to that answer catalog forces a new protocol version and breaks score comparability — the benchmark owner's call. The catalog stays true untilsource_openis implemented and exposed.LocalMountPublisheris currently called withoutraw_root/artifacts_root(selfhost.py:1124-1129), so those views are empty placeholders andraw_uriresolves to nothing on the mount today. Themounts.pymodule docstring's guardrail 1 also becomes stale.Related in-flight work
SourceHandlePort, bounded reads of an immutable hashed source) is the seam D107's ephemeral rendition transform needs — should land first.Decision numbers have collided twice while this sat open (D103→D106→D107). Please assign final numbers at merge.