Hi maintainers — I’m updating this RFC after narrowing the integration boundary I’m testing, and I’d appreciate guidance before proposing a small example/documentation PR.
The current idea is not to add grounding/RAG or a new inference path to ExecuTorch. It is a small application-owned evidence/answer-policy boundary around the existing extension/llm runner:
- the application/external component supplies bounded approved evidence/messages plus answer requirements after its own offline qualification;
- the host keeps model-specific prompt rendering and runner loading;
- generation-required requests render once and call the existing
IRunner::generate(prompt, GenerationConfig, ...) once;
- raw generation is checked afterward by an external deterministic finalizer;
- no vector DB, second model, agent loop, retry/repair loop, or ExecuTorch runtime change is required.
A proof-based no-model-call path is optional and out of scope for the first example unless the application has a sound request-specific verifier. A saved profile/configuration by itself is not treated as proof of an answer.
Current local evidence is intentionally limited:
- the adapter compiles against the current upstream
extension/llm/runner/irunner.h declaration;
- an executable fake-
IRunner smoke verifies the generation path performs one renderer call plus one IRunner::generate() call and returns raw output for external validation;
- model-specific rendering and execution stay host-owned;
- a real
.pte + tokenizer TextLLMRunner smoke is still pending, so I am not claiming real-model compatibility yet.
ExactScope is here: https://github.com/ot4562-glitch/ExactScope . This is unreleased v1.1 research. I am not claiming cross-runtime answer-quality transfer or asking ExecuTorch to adopt ExactScope semantics; the proposed upstream contribution would only show a clean application-owned policy boundary around the existing runner API.
Would this fit better as an examples/ sample or as documentation near extension/llm? I’d like the eventual diff to stay small and use the existing public runner surface rather than add framework logic to ExecuTorch.
Hi maintainers — I’m updating this RFC after narrowing the integration boundary I’m testing, and I’d appreciate guidance before proposing a small example/documentation PR.
The current idea is not to add grounding/RAG or a new inference path to ExecuTorch. It is a small application-owned evidence/answer-policy boundary around the existing
extension/llmrunner:IRunner::generate(prompt, GenerationConfig, ...)once;A proof-based no-model-call path is optional and out of scope for the first example unless the application has a sound request-specific verifier. A saved profile/configuration by itself is not treated as proof of an answer.
Current local evidence is intentionally limited:
extension/llm/runner/irunner.hdeclaration;IRunnersmoke verifies the generation path performs one renderer call plus oneIRunner::generate()call and returns raw output for external validation;.pte+ tokenizerTextLLMRunnersmoke is still pending, so I am not claiming real-model compatibility yet.ExactScope is here: https://github.com/ot4562-glitch/ExactScope . This is unreleased v1.1 research. I am not claiming cross-runtime answer-quality transfer or asking ExecuTorch to adopt ExactScope semantics; the proposed upstream contribution would only show a clean application-owned policy boundary around the existing runner API.
Would this fit better as an
examples/sample or as documentation nearextension/llm? I’d like the eventual diff to stay small and use the existing public runner surface rather than add framework logic to ExecuTorch.