Skip to content

feat: preserve opaque_body provider state on prompt messages and LLM results - #261

Open
hpiclaranet wants to merge 1 commit into
langgenius:mainfrom
hpiclaranet:feat/opaque-body-prompt-messages
Open

feat: preserve opaque_body provider state on prompt messages and LLM results#261
hpiclaranet wants to merge 1 commit into
langgenius:mainfrom
hpiclaranet:feat/opaque-body-prompt-messages

Conversation

@hpiclaranet

Copy link
Copy Markdown

Important

  1. Make sure you have read our contribution guidelines
  2. Search existing issues and pull requests to confirm this change is not a duplicate
  3. Open or identify the issue this pull request resolves or advances
  4. Use a Conventional Commits title for this pull request, and mark breaking changes with !
  5. Remember that the pull request title will become the squash merge commit message
  6. If CLA Assistant prompts you, sign CLA.md in the pull request conversation

Related Issue

Closes #260

Summary

Plugin LLM results can carry provider round-trip state in opaque_body (e.g. signed/encrypted Claude thinking blocks, Gemini thought_signature). The plugin SDK (dify_plugin >= 0.10.0) defines the field on both AssistantPromptMessage and PromptMessageContent, and the plugin daemon passes it through to Dify core — but graphon dropped it in two places:

  • Entities: AssistantPromptMessage / PromptMessageContent had no opaque_body field, so the daemon → core Pydantic boundary silently discarded it during LLMResultChunk / LLMResult validation.
  • Result accumulators: _LLMChunkAccumulator and _StreamingInvokeAccumulator rebuilt the assistant message from content + tool calls only, dropping the field even when chunks carried it (affects non-stream normalization of stream-only models and streaming callback bookkeeping).

This change:

  • Adds opaque_body: JsonValue | None = None to AssistantPromptMessage and PromptMessageContent, mirroring the plugin SDK field name, type, and default exactly (message-level for reasoning state; content-block-level for per-part state such as Gemini thought_signature).
  • Carries the field through both accumulators with complete-snapshot semantics: the last non-None chunk wins, and later None chunks do not clobber a captured snapshot.
  • Adds tests: entity JSON round-trips (including a simulated daemon → core boundary via LLMResultChunk.model_validate_json), backward compatibility for payloads without the field, and accumulator preservation for both stream and non-stream paths.

Compatibility notes:

  • Backward compatible: payloads without the field still validate (None default); no behavior change for providers that do not use the field.
  • Serialized content blocks now include opaque_body: null when unset, matching the plugin SDK's existing serialization behavior at the daemon boundary.
  • Known remaining drop sites intentionally left out of scope: the slim runtime result collector (dsl/slim/llm.py), workflow LLM node prompt reconstruction (nodes/llm/llm_utils.py), and parameter extractor message building — none of them are on the agent-loop path this issue covers.

Context: langgenius/dify#41092 (Claude extended thinking dropped between Agent tool-use rounds), plugin-side counterpart langgenius/dify-official-plugins#3715, prior core attempt langgenius/dify#31566.

Validation: uv run pytest (742 passed), uv lock --check, uv run ruff format --check, uv run ruff check, uv run ty check all green. I will sign the CLA if CLA Assistant prompts.

Checklist

  • This pull request links the issue it resolves or advances
  • This pull request title follows Conventional Commits, and any breaking change is marked with !
  • If CLA Assistant prompted me, I signed CLA.md in the pull request conversation

…results

Plugin LLM results carry provider round-trip state in opaque_body
(e.g. signed/encrypted thinking blocks, Gemini thought_signature),
matching the dify_plugin SDK entities. Graphon dropped that state in
two places:

- AssistantPromptMessage / PromptMessageContent had no opaque_body
  field, so daemon -> core Pydantic validation silently discarded it.
- _LLMChunkAccumulator and _StreamingInvokeAccumulator rebuilt the
  assistant message from content and tool calls only.

Add the field (same name, type and default as the plugin SDK) and
carry the last non-None snapshot through both accumulators.

Refs: langgenius#260
@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. enhancement New feature or request labels Aug 22, 2026
@github-actions

github-actions Bot commented Aug 22, 2026

Copy link
Copy Markdown

All contributors on this pull request have signed the CLA.
Posted by the CLA Assistant Lite bot.

@hpiclaranet

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provider round-trip state (opaque_body) is dropped on assistant prompt messages and LLM results

1 participant