Harness: StopRuntimeSession returns 200 but does not cancel the in-flight model call of InvokeHarness (billed up to maxTokens)

[jvjfranca]

Description

For a managed harness, the only way to stop a session is StopRuntimeSession called with the harness ARN (the harness has no stop/cancel operation of its own; botocore 1.43.72 exposes only InvokeHarness on the data plane). The call returns 200, but the model call the harness already made keeps generating to the end, is billed in full (up to maxTokens), and the InvokeHarness stream keeps delivering text for minutes after the stop.

The Runtime docs (runtime-stop-session) say the operation "instantly terminates the specified session and stops any ongoing streaming responses". Nothing in the harness pages says a stop behaves differently for InvokeHarness, so either the behavior or the docs need a fix. We use the stop to implement a user-facing "Stop" button, so today a stopped answer costs the same as a finished one and nothing reports that cost back to the caller.

Steps to Reproduce

1. Harness on the DEFAULT endpoint, us-east-1, model global.anthropic.claude-sonnet-5, no tools needed. An after_invocation lifecycle hook (Lambda) is configured.
2. invoke_harness(harnessArn=..., qualifier="DEFAULT", runtimeSessionId=<new id>, messages=[{"role":"user","content":[{"text":"Write a ~3,000-word essay ... no tools"}]}], model={"bedrockModelConfig":{"modelId":"global.anthropic.claude-sonnet-5","maxTokens":8000}}, maxIterations=2, timeoutSeconds=600) and read the event stream.
3. At 10 s, from a second client: stop_runtime_session(runtimeSessionId=<same id>, agentRuntimeArn=<harness ARN>, qualifier="DEFAULT").
4. Keep reading the stream; then check Bedrock model invocation logging and the hook's logs.

Expected Behavior

As documented for StopRuntimeSession: the session ends and the ongoing streaming response stops right away, the in-flight model call is cancelled (so it is not billed to maxTokens), and ideally the after_invocation hook or a final metadata event reports the usage consumed up to the stop.

Actual Behavior

Measured on 2026-09-28, one run alone:

t	event
0 s	InvokeHarness; messageStart at 5.7 s
10.7 s	StopRuntimeSession → 200 (0.61 s call)
10.7 → 141.6 s	stream keeps delivering: 822 chars before the stop, 14,288 chars after it; it then breaks off with no contentBlockStop / messageStop / metadata

• Bedrock model invocation logging shows one ConverseStream from the harness execution role for that session: input 6,588 tokens, output 8,000 tokens (the whole maxTokens). Bedrock request id: 8e41387b-1902-444e-a812-d71003f5d480.
• The after_invocation hook never fired for that session (checked 10 min later), so the caller has no way to learn the usage.
• Closing the stream client-side without a stop does not cancel the generation either: in a separate run it went on to end_turn (hook reported 6,592 in / 7,356 out, 94.5 s after the close).
• In an earlier run a stop cut the stream only after ~38 s, and that one's hook reported 0/0 usage with stopReason: "interrupted".

CLI Version

Not used for this. The calls are boto3 / botocore 1.43.72 (bedrock-agentcore). Filing here because this repo is where harness issues are tracked; please move it if another repo fits better.

Operating System

macOS

Additional Context

What would help, in order:

1. StopRuntimeSession on a harness session cancels the in-flight model call (as the Runtime docs describe).
2. If cancellation isn't possible: the harness reports the usage of a stopped invocation (final metadata event or after_invocation with real token counts), so callers can account for it.
3. Otherwise, the harness docs state that a stop does not cancel the model call and that the call is billed to completion.

Workaround we use: the caller polls its own "stopped" state, closes the stream within ~2 s and discards the rest, and reconciles cost from Bedrock invocation logs.