feat(tracing): trace scheduled Runtime processing (#1243) - #1245
Draft
guozhihao-224 wants to merge 1 commit into
Draft
feat(tracing): trace scheduled Runtime processing (#1243)#1245guozhihao-224 wants to merge 1 commit into
guozhihao-224 wants to merge 1 commit into
Conversation
- Updated documentation to reflect the addition of a new `memory.flush` span in the tracing output. - Implemented minimal OpenTelemetry-free tracing hooks in `tracing.py` for domain code. - Added tracing for scheduled source window processing and experience incubation, capturing success, failure, and cancellation outcomes. - Introduced a `DomainTracer` to adapt server tracing for domain-specific spans. - Enhanced tests to verify the correct tracing behavior for scheduled operations and memory flush events.
|
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds scheduled-processing trace roots to the built-in Runtime (per RFC 0046 / issue #1243) by introducing a minimal tracing protocol for domain code and adapting the server’s OpenTelemetry tracing to that protocol. The goal is to make scheduled Source-window processing and Experience incubation observable in traces without leaking scope/content data and without coupling Runtime code to OTel.
Changes:
- Introduce an OpenTelemetry-free
Tracer/Spanprotocol for domain code and inject it into the built-in Runtime. - Add scheduled trace roots (
scheduled.process_source_window,scheduled.incubate_experience_candidates) and boundary spans (memory.flush,experience.incubation) with outcome attributes. - Update tests and Phoenix tracing docs to reflect the new span tree (including
memory.flushunder HTTP operations and scheduled activations).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/test_observability.py | Extends e2e assertions for the new memory.flush boundary span and adds an e2e scheduled-activation trace-root test. |
| tests/builtin/runtime/test_scheduler.py | Adds unit tests asserting scheduled trace roots and outcome attributes (noop/success/failure/cancelled) plus data-policy checks. |
| src/powercontext/tracing.py | Introduces the minimal domain tracing protocol (Tracer/Span) to decouple Runtime from OTel. |
| src/powercontext/server/tracing.py | Adds DomainTracer adapter bridging the protocol to ServerTracing and implements root-span creation for scheduled work. |
| src/powercontext/server/factory.py | Wires DomainTracer into Runtime construction in the server app lifespan. |
| src/powercontext/builtin/runtime/composition.py | Extends open_builtin_runtime to accept and pass through the injected domain tracer. |
| src/powercontext/builtin/runtime/application.py | Emits scheduled root spans and adds memory.flush / experience.incubation boundary spans with outcomes and bounded attributes. |
| docs/zh/docs/how-to/trace-with-phoenix.md | Updates the Phoenix trace walkthrough to include memory.flush and scheduled trace roots. |
| docs/en/docs/how-to/trace-with-phoenix.md | Updates the Phoenix trace walkthrough to include memory.flush and scheduled trace roots. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+147
to
+167
| def start_span(self, name: str, *, attributes: dict[str, object]) -> _ActiveSpan: | ||
| span_attributes = dict(attributes) | ||
| request_id = current_request_id() | ||
| # note(guozhihao-224): only child spans join a request; scheduled roots are fresh traces with no request id. | ||
| if request_id is not None: | ||
| span_attributes["powercontext.request.id"] = request_id | ||
| return self._tracing.start_span( | ||
| name, | ||
| kind=SpanKind.INTERNAL, | ||
| attributes=span_attributes, | ||
| context=None, | ||
| ) | ||
|
|
||
| def start_root_span(self, name: str, *, attributes: dict[str, object]) -> _ActiveSpan: | ||
| # note(guozhihao-224): fresh empty context keeps scheduled activations as independent trace roots. | ||
| return self._tracing.start_span( | ||
| name, | ||
| kind=SpanKind.INTERNAL, | ||
| attributes=dict(attributes), | ||
| context=Context(), | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue or RFC does this PR close?
Closes #1243.
Rationale for this change
Scheduled Source-window processing and Experience incubation previously reported outcomes only through logs (
background.operation.completed), with no trace root. RFC 0046 requires background work to start its own trace when tracing is enabled. This PR adds one trace root per scoped scheduled activation so operators can see when a scheduled run ran, which outcome it produced, and where latency or failure occurred.What changes are included in this PR?
Tracer/Spaninsrc/powercontext/tracing.py) injected into the built-in Runtime; Runtime code stays independent of any OpenTelemetry implementation.DomainTraceradapter insrc/powercontext/server/tracing.pythat maps the protocol ontoServerTracing(start_root_spanopens a fresh trace;start_spanjoins the current context).scheduled.process_source_window/scheduled.incubate_experience_candidatesroot spans from the scheduled processors, withmemory.flush/experience.incubationboundary spans underneath. Outcomes are success/noop/failure/cancelled, matching the existing logs. Scheduled roots never inherit an HTTP or MCP context and never record scope IDs or content.open_builtin_runtimeandcreate_server_app.flush_memoryspan tree (thememory.flushboundary span also appears underpowercontext flush_memory), the e2e parent-chain assertion, and the Phoenix docs (EN/ZH).Are there any user-facing changes?
Observability-only: when tracing is enabled, scheduled activations now emit spans. No breaking API or persisted-format changes; tracing remains optional, and scheduler/Runtime behavior is unchanged when tracing is off.
How was this change tested?
pytest tests/builtin/runtime/test_scheduler.py→ 12 passedpytest tests/e2e/test_observability.py→ 3 passedruff check/ruff format --check/ty check→ all pass