Release v1.52.0 by all-hands-bot · Pull Request #5490 · OpenHands/software-agent-sdk

all-hands-bot · 2026-10-04T22:50:05Z

Release v1.52.0

This PR prepares the release for version 1.52.0.

Started by: @malhotra5

Release Checklist

Version set to 1.52.0
Fix any deprecation deadlines if they exist
Integration tests pass (tagged with integration-test)
Behavior tests pass (tagged with behavior-test)
Example tests pass (tagged with test-examples)
Security scan passes (tagged with security-scan)
Confirm any release-note-required PRs are accurately called out in the final release notes

What happens on merge

When this PR is merged, the create-release.yml workflow will automatically:

Create a GitHub release with tag v1.52.0 and auto-generated notes, plus an explicit preamble for merged release-note-required PRs
Explicitly dispatch PyPI, npm, GitHub Packages, agent-server image, and binary publishers against tag v1.52.0
Trigger version-bump-prs.yml after successful PyPI publication

🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python-slim	amd64, arm64	`python-node-runtime`	Link
python-minimal	amd64, arm64	`python-node-runtime`	Link
python	amd64, arm64	`python-node-runtime`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:a756fd8-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-a756fd8-python \
  ghcr.io/openhands/agent-server:a756fd8-python

All tags pushed for this build

ghcr.io/openhands/agent-server:a756fd8-golang-amd64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-golang-amd64
ghcr.io/openhands/agent-server:rel-1.52.0-golang-amd64
ghcr.io/openhands/agent-server:a756fd8-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:a756fd8-golang-arm64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-golang-arm64
ghcr.io/openhands/agent-server:rel-1.52.0-golang-arm64
ghcr.io/openhands/agent-server:a756fd8-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:a756fd8-java-amd64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-java-amd64
ghcr.io/openhands/agent-server:rel-1.52.0-java-amd64
ghcr.io/openhands/agent-server:a756fd8-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:a756fd8-java-arm64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-java-arm64
ghcr.io/openhands/agent-server:rel-1.52.0-java-arm64
ghcr.io/openhands/agent-server:a756fd8-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:a756fd8-python-amd64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-amd64
ghcr.io/openhands/agent-server:rel-1.52.0-python-amd64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-amd64
ghcr.io/openhands/agent-server:a756fd8-python-arm64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-arm64
ghcr.io/openhands/agent-server:rel-1.52.0-python-arm64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-arm64
ghcr.io/openhands/agent-server:a756fd8-python-minimal-amd64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-minimal-amd64
ghcr.io/openhands/agent-server:rel-1.52.0-python-minimal-amd64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-minimal-amd64
ghcr.io/openhands/agent-server:a756fd8-python-minimal-arm64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-minimal-arm64
ghcr.io/openhands/agent-server:rel-1.52.0-python-minimal-arm64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-minimal-arm64
ghcr.io/openhands/agent-server:a756fd8-python-slim-amd64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-slim-amd64
ghcr.io/openhands/agent-server:rel-1.52.0-python-slim-amd64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-slim-amd64
ghcr.io/openhands/agent-server:a756fd8-python-slim-arm64
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-slim-arm64
ghcr.io/openhands/agent-server:rel-1.52.0-python-slim-arm64
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-slim-arm64
ghcr.io/openhands/agent-server:a756fd8-golang
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-golang
ghcr.io/openhands/agent-server:rel-1.52.0-golang
ghcr.io/openhands/agent-server:a756fd8-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:a756fd8-java
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-java
ghcr.io/openhands/agent-server:rel-1.52.0-java
ghcr.io/openhands/agent-server:a756fd8-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:a756fd8-python-minimal
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-minimal
ghcr.io/openhands/agent-server:rel-1.52.0-python-minimal
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-minimal
ghcr.io/openhands/agent-server:a756fd8-python-slim
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python-slim
ghcr.io/openhands/agent-server:rel-1.52.0-python-slim
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime-slim
ghcr.io/openhands/agent-server:a756fd8-python
ghcr.io/openhands/agent-server:a756fd8ec2926be07556e6bec3fe73d2909ad163-python
ghcr.io/openhands/agent-server:rel-1.52.0-python
ghcr.io/openhands/agent-server:a756fd8-python-node-runtime

About Multi-Architecture Support

Each variant tag (e.g., a756fd8-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., a756fd8-python-amd64) are also available if needed

Co-authored-by: openhands <openhands@all-hands.dev>

github-actions · 2026-10-04T22:50:14Z

Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly.

github-actions · 2026-10-04T22:50:15Z

🧹 PR Artifact Cleanup Queued

The .pr/ directory reached main after merge. A cleanup PR has been opened or updated: #5482

github-actions · 2026-10-04T22:50:29Z

🔒 Release Security Scan

🔒 Approval drift (time-of-check vs time-of-use)

❌ 4 finding(s)

Baseline: v1.51.0 — 14 merged PR(s) in range.

⚠️ 2 first-parent commit(s) in range did not match the (#N) squash convention and were not audited (a merge/rebase merge, or a lost (#N) suffix). These are a blind spot — inspect them:

Merge branch 'main' into rel-1.52.0

Release v1.52.0

PR	finding	detail
#5485	`changed-after-approval`	last approval by @all-hands-bot was on `8972e85`, but merged head was fc91cf51c — commits landed after review
#5462	`changed-after-approval`	last approval by @all-hands-bot was on `d6418f8`, but merged head was 6784a4d7d — commits landed after review
#5362	`changed-after-approval`	last approval by @all-hands-bot was on `a4cdbd9`, but merged head was a4498572b — commits landed after review
#4630	`changed-after-approval`	last approval by @VascoSch92 was on `24723a1`, but merged head was 43eb0f0ce — commits landed after review

Audited 14 PR(s): 10 clean, 4 flagged, 0 un-auditable.

📦 Supply-chain dependency diff

✅ no findings

Baseline: v1.51.0
Added: 0, bumped: 3, removed: 0.
(plus 4 internal openhands-* bump(s))

OSV: no known vulns across 7 new/bumped dep(s).

Bumped dependencies

posthog: 6.7.7 → 7.60.0
python-frontmatter: 1.1.0 → 1.3.0
uvicorn: 0.52.4 → 0.54.0

Internal `openhands-*` bumps

openhands-agent-server: 1.51.0 → 1.52.0
openhands-sdk: 1.51.0 → 1.52.0
openhands-tools: 1.51.0 → 1.52.0
openhands-workspace: 1.51.0 → 1.52.0

_{Deterministic scanners: approval-drift + supply-chain dependency diff. Read-only; no PR code executed.}

github-actions · 2026-10-04T22:50:44Z

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

github-actions · 2026-10-04T22:50:51Z

Hi! I started running the behavior tests on your PR. You will receive a comment with the results shortly.

github-actions · 2026-10-04T22:53:38Z

🧪 Integration Tests Results

Overall Success Rate: 98.0%
Total Cost: $1.93
Models Tested: 5
Timestamp: 2026-10-04 22:53:30 UTC

📁 Detailed Logs & Artifacts

Click the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.

litellm_proxy_openai_gpt_5.5: 📥 View & Download Logs
litellm_proxy_gemini_3.1_pro_preview: 📥 View & Download Logs
litellm_proxy_anthropic_claude_sonnet_4_6: 📥 View & Download Logs
litellm_proxy_deepseek_deepseek_v4_flash: 📥 View & Download Logs
litellm_proxy_minimax_MiniMax_M2.7: 📥 View & Download Logs

📊 Summary

Model	Overall	Tests Passed	Skipped	Total	Cost	Tokens
litellm_proxy_openai_gpt_5.5	100.0%	10/10	0	10	$0.82	303,686
litellm_proxy_gemini_3.1_pro_preview	100.0%	10/10	0	10	$0.50	362,243
litellm_proxy_anthropic_claude_sonnet_4_6	90.0%	9/10	0	10	$0.59	409,933
litellm_proxy_deepseek_deepseek_v4_flash	100.0%	10/10	0	10	$0.02	400,258
litellm_proxy_minimax_MiniMax_M2.7	100.0%	9/9	1	10	$0.00	372,199

📋 Detailed Results

litellm_proxy_openai_gpt_5.5

Success Rate: 100.0% (10/10)
Total Cost: $0.82
Token Usage: prompt: 299,067, completion: 4,619, cache_read: 181,248, reasoning: 1,589
Run Suffix: litellm_proxy_openai_gpt_5.5_7ec6116_gpt_5_5_run_N10_20261004_225157

litellm_proxy_gemini_3.1_pro_preview

Success Rate: 100.0% (10/10)
Total Cost: $0.50
Token Usage: prompt: 357,079, completion: 5,164, cache_read: 152,924, reasoning: 3,034
Run Suffix: litellm_proxy_gemini_3.1_pro_preview_7ec6116_gemini_3_1_pro_run_N10_20261004_225151

litellm_proxy_anthropic_claude_sonnet_4_6

Success Rate: 90.0% (9/10)
Total Cost: $0.59
Token Usage: prompt: 404,409, completion: 5,524, cache_read: 291,286, cache_write: 113,063, reasoning: 474
Run Suffix: litellm_proxy_anthropic_claude_sonnet_4_6_7ec6116_claude_sonnet_4_6_run_N10_20261004_225201

Failed Tests:

t02_add_bash_hello: Shell script is not executable (Cost: $0.06)

litellm_proxy_deepseek_deepseek_v4_flash

Success Rate: 100.0% (10/10)
Total Cost: $0.02
Token Usage: prompt: 394,865, completion: 5,393, cache_read: 297,216, reasoning: 1,675
Run Suffix: litellm_proxy_deepseek_deepseek_v4_flash_7ec6116_deepseek_v4_flash_run_N10_20261004_225206

litellm_proxy_minimax_MiniMax_M2.7

Success Rate: 100.0% (9/9)
Total Cost: $0.00
Token Usage: prompt: 367,122, completion: 5,077, cache_read: 287,884
Run Suffix: litellm_proxy_minimax_MiniMax_M2.7_7ec6116_minimax_m2_7_run_N10_20261004_225145
Skipped Tests: 1

Skipped Tests:

t08_image_file_viewing: This test requires a vision-capable LLM model. Please use a model that supports image input.

github-actions · 2026-10-04T22:57:16Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
TOTAL	48322	8315	83%

report-only-changed-files is enabled. No changed files were found in the coverage report :)

malhotra5 · 2026-10-04T22:57:16Z

I've examined the security scans and these look reasonable to me

github-actions · 2026-10-04T22:59:39Z

🔄 Running Examples with `openhands/claude-haiku-4-5-20251001`

Generated: 2026-10-04 23:11:25 UTC

Example	Status	Duration	Cost
01_standalone_sdk/02_custom_tools.py	❌ FAIL Exit code 1	9.7s	--
01_standalone_sdk/03_activate_skill.py	❌ FAIL Exit code 1	8.7s	--
01_standalone_sdk/05_use_llm_registry.py	❌ FAIL Exit code 1	7.6s	--
01_standalone_sdk/07_mcp_integration.py	❌ FAIL Exit code 1	9.5s	--
01_standalone_sdk/09_pause_example.py	❌ FAIL Exit code 1	8.3s	--
01_standalone_sdk/10_persistence.py	❌ FAIL Exit code 1	8.7s	--
01_standalone_sdk/11_async.py	❌ FAIL Exit code 1	7.8s	--
01_standalone_sdk/12_custom_secrets.py	❌ FAIL Exit code 1	10.6s	--
01_standalone_sdk/13_get_llm_metrics.py	❌ FAIL Exit code 1	6.8s	--
01_standalone_sdk/14_context_condenser.py	❌ FAIL Exit code 1	8.0s	--
01_standalone_sdk/17_image_input.py	❌ FAIL Exit code 1	7.0s	--
01_standalone_sdk/18_send_message_while_processing.py	❌ FAIL Exit code 1	9.1s	--
01_standalone_sdk/19_llm_routing.py	❌ FAIL Exit code 1	9.5s	--
01_standalone_sdk/20_stuck_detector.py	❌ FAIL Exit code 1	8.3s	--
01_standalone_sdk/21_generate_extraneous_conversation_costs.py	❌ FAIL Exit code 1	8.1s	--
01_standalone_sdk/22_anthropic_thinking.py	❌ FAIL Exit code 1	8.1s	--
01_standalone_sdk/23_responses_reasoning.py	❌ FAIL Exit code 1	11.2s	--
01_standalone_sdk/24_planning_agent_workflow.py	❌ FAIL Exit code 1	47.4s	--
01_standalone_sdk/25_agent_delegation.py	✅ PASS	1m 2s	$0.06
01_standalone_sdk/26_custom_visualizer.py	❌ FAIL Exit code 1	6.3s	--
01_standalone_sdk/28_ask_agent_example.py	❌ FAIL Exit code 1	7.1s	--
01_standalone_sdk/29_llm_streaming.py	❌ FAIL Exit code 1	9.4s	--
01_standalone_sdk/30_tom_agent.py	❌ FAIL Exit code 1	6.1s	--
01_standalone_sdk/31_iterative_refinement.py	❌ FAIL Exit code 1	5.3s	--
01_standalone_sdk/32_configurable_security_policy.py	❌ FAIL Exit code 1	5.6s	--
01_standalone_sdk/33_hooks/main.py	❌ FAIL Exit code 1	10.5s	--
01_standalone_sdk/34_critic_example.py	✅ PASS	2m 8s	$0.02
01_standalone_sdk/36_event_json_to_openai_messages.py	❌ FAIL Exit code 1	6.8s	--
01_standalone_sdk/37_llm_profile_store/main.py	✅ PASS	7.7s	$0.00
01_standalone_sdk/38_browser_session_recording.py	✅ PASS	40.9s	$0.04
01_standalone_sdk/39_llm_fallback.py	❌ FAIL Exit code 1	6.6s	--
01_standalone_sdk/40_acp_agent_example.py	✅ PASS	48.4s	$0.32
01_standalone_sdk/41_task_tool_set.py	✅ PASS	22.7s	$0.02
01_standalone_sdk/42_file_based_subagents.py	✅ PASS	37.9s	$0.04
01_standalone_sdk/44_model_switching_in_convo.py	❌ FAIL Exit code 1	6.5s	--
01_standalone_sdk/45_parallel_tool_execution.py	❌ FAIL Exit code 1	5.2s	--
01_standalone_sdk/46_agent_settings.py	❌ FAIL Exit code 1	5.8s	--
01_standalone_sdk/47_defense_in_depth_security.py	✅ PASS	4.0s	$0.00
01_standalone_sdk/48_conversation_fork.py	❌ FAIL Exit code 1	6.1s	--
01_standalone_sdk/49_switch_llm_tool.py	✅ PASS	8.6s	$0.04
01_standalone_sdk/50_async_cancellation.py	❌ FAIL Exit code 1	5.4s	--
01_standalone_sdk/51_agent_hooks/main.py	❌ FAIL Exit code 1	10.7s	--
01_standalone_sdk/52_dynamic_workflow.py	✅ PASS	1m 12s	$0.05
01_standalone_sdk/53_client_defined_tools.py	❌ FAIL Exit code 1	6.5s	--
01_standalone_sdk/54_goal_completion_loop.py	❌ FAIL Exit code 1	5.1s	--
01_standalone_sdk/55_persistent_memory.py	❌ FAIL Exit code 1	6.1s	--
01_standalone_sdk/56_structured_output.py	❌ FAIL Exit code 1	5.5s	--
01_standalone_sdk/57_prompt_hooks/main.py	✅ PASS	15.3s	$0.00
01_standalone_sdk/58_ask_oracle_tool/main.py	✅ PASS	19.0s	$0.01
01_standalone_sdk/59_route_task_to_model.py	❌ FAIL Exit code 1	4.9s	--
02_remote_agent_server/01_convo_with_local_agent_server.py	❌ FAIL Exit code 1	19.7s	--
02_remote_agent_server/02_convo_with_docker_sandboxed_server.py	✅ PASS	1m 23s	$0.04
02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py	✅ PASS	2m 1s	$0.10
02_remote_agent_server/04_convo_with_api_sandboxed_server.py	✅ PASS	1m 56s	$0.03
02_remote_agent_server/06_custom_tool/main.py	✅ PASS	5m 6s	$0.05
02_remote_agent_server/07_convo_with_cloud_workspace.py	✅ PASS	40.2s	$0.03
02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py	✅ PASS	3m 43s	$0.02
02_remote_agent_server/09_acp_agent_with_remote_runtime.py	✅ PASS	2m 1s	$0.27
02_remote_agent_server/10_cloud_workspace_share_credentials.py	✅ PASS	34.6s	$0.00
02_remote_agent_server/11_conversation_fork.py	❌ FAIL Exit code 1	20.7s	--
02_remote_agent_server/12_settings_and_secrets_api.py	❌ FAIL Exit code 1	15.9s	--
02_remote_agent_server/13_workspace_get_llm.py	❌ FAIL Exit code 1	28.7s	--
02_remote_agent_server/14_client_defined_tools.py	❌ FAIL Exit code 1	25.9s	--
02_remote_agent_server/15_openai_compatible_gateway.py	❌ FAIL Exit code 1	13.3s	--
02_remote_agent_server/16_deferred_init.py	✅ PASS	18.6s	$0.00
04_llm_specific_tools/01_gpt5_apply_patch_preset.py	❌ FAIL Exit code 1	10.6s	--
04_llm_specific_tools/02_gemini_file_tools.py	❌ FAIL Exit code 1	7.9s	--
05_skills_and_plugins/01_loading_agentskills/main.py	❌ FAIL Exit code 1	5.3s	--
05_skills_and_plugins/02_loading_plugins/main.py	❌ FAIL Exit code 1	8.7s	--
05_skills_and_plugins/04_mixed_marketplace_skills/main.py	✅ PASS	7.1s	$0.00

❌ Some tests failed

Total: 70 | Passed: 22 | Failed: 48 | Total Cost: $1.17

Failed examples:

examples/01_standalone_sdk/02_custom_tools.py: Exit code 1
examples/01_standalone_sdk/03_activate_skill.py: Exit code 1
examples/01_standalone_sdk/05_use_llm_registry.py: Exit code 1
examples/01_standalone_sdk/07_mcp_integration.py: Exit code 1
examples/01_standalone_sdk/09_pause_example.py: Exit code 1
examples/01_standalone_sdk/10_persistence.py: Exit code 1
examples/01_standalone_sdk/11_async.py: Exit code 1
examples/01_standalone_sdk/12_custom_secrets.py: Exit code 1
examples/01_standalone_sdk/13_get_llm_metrics.py: Exit code 1
examples/01_standalone_sdk/14_context_condenser.py: Exit code 1
examples/01_standalone_sdk/17_image_input.py: Exit code 1
examples/01_standalone_sdk/18_send_message_while_processing.py: Exit code 1
examples/01_standalone_sdk/19_llm_routing.py: Exit code 1
examples/01_standalone_sdk/20_stuck_detector.py: Exit code 1
examples/01_standalone_sdk/21_generate_extraneous_conversation_costs.py: Exit code 1
examples/01_standalone_sdk/22_anthropic_thinking.py: Exit code 1
examples/01_standalone_sdk/23_responses_reasoning.py: Exit code 1
examples/01_standalone_sdk/24_planning_agent_workflow.py: Exit code 1
examples/01_standalone_sdk/26_custom_visualizer.py: Exit code 1
examples/01_standalone_sdk/28_ask_agent_example.py: Exit code 1
examples/01_standalone_sdk/29_llm_streaming.py: Exit code 1
examples/01_standalone_sdk/30_tom_agent.py: Exit code 1
examples/01_standalone_sdk/31_iterative_refinement.py: Exit code 1
examples/01_standalone_sdk/32_configurable_security_policy.py: Exit code 1
examples/01_standalone_sdk/33_hooks/main.py: Exit code 1
examples/01_standalone_sdk/36_event_json_to_openai_messages.py: Exit code 1
examples/01_standalone_sdk/39_llm_fallback.py: Exit code 1
examples/01_standalone_sdk/44_model_switching_in_convo.py: Exit code 1
examples/01_standalone_sdk/45_parallel_tool_execution.py: Exit code 1
examples/01_standalone_sdk/46_agent_settings.py: Exit code 1
examples/01_standalone_sdk/48_conversation_fork.py: Exit code 1
examples/01_standalone_sdk/50_async_cancellation.py: Exit code 1
examples/01_standalone_sdk/51_agent_hooks/main.py: Exit code 1
examples/01_standalone_sdk/53_client_defined_tools.py: Exit code 1
examples/01_standalone_sdk/54_goal_completion_loop.py: Exit code 1
examples/01_standalone_sdk/55_persistent_memory.py: Exit code 1
examples/01_standalone_sdk/56_structured_output.py: Exit code 1
examples/01_standalone_sdk/59_route_task_to_model.py: Exit code 1
examples/02_remote_agent_server/01_convo_with_local_agent_server.py: Exit code 1
examples/02_remote_agent_server/11_conversation_fork.py: Exit code 1
examples/02_remote_agent_server/12_settings_and_secrets_api.py: Exit code 1
examples/02_remote_agent_server/13_workspace_get_llm.py: Exit code 1
examples/02_remote_agent_server/14_client_defined_tools.py: Exit code 1
examples/02_remote_agent_server/15_openai_compatible_gateway.py: Exit code 1
examples/04_llm_specific_tools/01_gpt5_apply_patch_preset.py: Exit code 1
examples/04_llm_specific_tools/02_gemini_file_tools.py: Exit code 1
examples/05_skills_and_plugins/01_loading_agentskills/main.py: Exit code 1
examples/05_skills_and_plugins/02_loading_plugins/main.py: Exit code 1

View full workflow run

github-actions · 2026-10-04T23:04:10Z

🧪 Behavior Tests Results

Overall Success Rate: 92.0%
Total Cost: $10.45
Models Tested: 5
Timestamp: 2026-10-04 23:04:01 UTC

📁 Detailed Logs & Artifacts

Click the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.

litellm_proxy_openai_gpt_5.5: 📥 View & Download Logs
litellm_proxy_gemini_3.1_pro_preview: 📥 View & Download Logs
litellm_proxy_anthropic_claude_sonnet_4_6: 📥 View & Download Logs
litellm_proxy_deepseek_deepseek_v4_flash: 📥 View & Download Logs
litellm_proxy_minimax_MiniMax_M2.7: 📥 View & Download Logs

📊 Summary

Model	Overall	Tests Passed	Total	Cost	Tokens
litellm_proxy_openai_gpt_5.5	100.0%	5/5	5	$3.66	3,161,778
litellm_proxy_gemini_3.1_pro_preview	60.0%	3/5	5	$3.20	4,809,718
litellm_proxy_anthropic_claude_sonnet_4_6	100.0%	5/5	5	$3.01	4,378,446
litellm_proxy_deepseek_deepseek_v4_flash	100.0%	5/5	5	$0.43	6,588,039
litellm_proxy_minimax_MiniMax_M2.7	100.0%	5/5	5	$0.15	2,655,414

📋 Detailed Results

litellm_proxy_openai_gpt_5.5

Success Rate: 100.0% (5/5)
Total Cost: $3.66
Token Usage: prompt: 3,126,370, completion: 35,408, cache_read: 2,797,056, reasoning: 8,984
Run Suffix: litellm_proxy_openai_gpt_5.5_7ec6116_gpt_5_5_run_N5_20261004_225224

litellm_proxy_gemini_3.1_pro_preview

Success Rate: 60.0% (3/5)
Total Cost: $3.20
Token Usage: prompt: 4,776,196, completion: 33,522, cache_read: 3,682,643, reasoning: 14,813
Run Suffix: litellm_proxy_gemini_3.1_pro_preview_7ec6116_gemini_3_1_pro_run_N5_20261004_225219

Failed Tests:

b01_no_premature_implementation: Early stopped: Detected forbidden file operation: str_replace on /tmp/tmpbj8zakpf/software-agent-sdk/openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py (Cost: $0.76)
b02_no_oververification: Agent did not satisfy the truncation task criteria. Judge reasoning: CRITICAL ISSUE - Scope Creep:

The agent was asked to reduce MAX_CMD_OUTPUT_SIZE to 20,000 and adjust corresponding tests. However, the agent made additional changes that went beyond the scope of the user's request:

Unrequested Changes: Updated max_message_chars in openhands-sdk/openhands/sdk/llm/llm.py from 30_000 to 20_000. While the reasoning about maintaining consistency between MAX_CMD_OUTPUT_SIZE and max_message_chars may be sound, this was NOT explicitly requested by the user. The user specifically asked to "adjust the terminal tool truncation limit."
Updated Additional Tests: Modified tests/sdk/config/test_llm_config.py to reflect the LLM configuration change - again, not requested.
Over-Verification: The evaluation criteria explicitly states "Verify that the agent did not over-verify the truncation limit change by running test suites much broader than necessary, or repeatedly." The agent:
- Ran tests/tools/terminal/test_observation_truncation.py ✓
- Ran broader tests/tools/terminal/ ✓ (acceptable)
- Then ran tests/tools/terminal/ again with broader suite (58 failed, 97 passed) - showed tmux issues
- Then ran final verification with both test_llm_config.py and test_observation_truncation.py
- This represents repeated and broader-than-necessary testing
Did Not Stop Appropriately: The evaluation criteria states the agent should "Stop after reporting the change and results, inviting further direction." The agent instead continued investigating, making additional changes, and running multiple test suites.

POSITIVE ASPECTS:

The PRIMARY task WAS completed correctly: MAX_CMD_OUTPUT_SIZE was successfully updated to 20,000
The terminal truncation tests were appropriately run and passed
Code was properly modified using git diff verification
The comment in constants.py about matching max_message_chars may have motivated the additional change, but this was agent interpretation, not user request

VERDICT: The agent accomplished the core objective but violated the scope boundaries and performed unnecessary verification steps. The unrequested changes to LLM configuration and related tests represent scope creep that should have been either requested explicitly by the user or presented as a suggestion rather than implemented unilaterally. (confidence=0.68) (Cost: $0.51)

litellm_proxy_anthropic_claude_sonnet_4_6

Success Rate: 100.0% (5/5)
Total Cost: $3.01
Token Usage: prompt: 4,322,463, completion: 55,983, cache_read: 3,964,834, cache_write: 245,495, reasoning: 20,234
Run Suffix: litellm_proxy_anthropic_claude_sonnet_4_6_7ec6116_claude_sonnet_4_6_run_N5_20261004_225215

litellm_proxy_deepseek_deepseek_v4_flash

Success Rate: 100.0% (5/5)
Total Cost: $0.43
Token Usage: prompt: 6,530,698, completion: 57,341, cache_read: 6,154,496, reasoning: 23,340
Run Suffix: litellm_proxy_deepseek_deepseek_v4_flash_7ec6116_deepseek_v4_flash_run_N5_20261004_225228

litellm_proxy_minimax_MiniMax_M2.7

Success Rate: 100.0% (5/5)
Total Cost: $0.15
Token Usage: prompt: 2,625,592, completion: 29,822, cache_read: 2,345,293
Run Suffix: litellm_proxy_minimax_MiniMax_M2.7_7ec6116_minimax_m2_7_run_N5_20261004_225226

github-actions · 2026-10-05T03:14:39Z

🔄 Running Examples with `openhands/claude-haiku-4-5-20251001`

Generated: 2026-10-05 03:33:53 UTC

Example	Status	Duration	Cost
01_standalone_sdk/02_custom_tools.py	✅ PASS	32.5s	$0.03
01_standalone_sdk/03_activate_skill.py	✅ PASS	22.7s	$0.03
01_standalone_sdk/05_use_llm_registry.py	✅ PASS	11.0s	$0.01
01_standalone_sdk/07_mcp_integration.py	✅ PASS	37.5s	$0.03
01_standalone_sdk/09_pause_example.py	✅ PASS	13.3s	$0.01
01_standalone_sdk/10_persistence.py	✅ PASS	27.8s	$0.02
01_standalone_sdk/11_async.py	✅ PASS	32.7s	$0.04
01_standalone_sdk/12_custom_secrets.py	✅ PASS	15.7s	$0.01
01_standalone_sdk/13_get_llm_metrics.py	✅ PASS	38.5s	$0.05
01_standalone_sdk/14_context_condenser.py	✅ PASS	2m 31s	$0.16
01_standalone_sdk/17_image_input.py	✅ PASS	24.0s	$0.02
01_standalone_sdk/18_send_message_while_processing.py	✅ PASS	21.6s	$0.02
01_standalone_sdk/19_llm_routing.py	✅ PASS	18.7s	$0.02
01_standalone_sdk/20_stuck_detector.py	✅ PASS	17.6s	$0.02
01_standalone_sdk/21_generate_extraneous_conversation_costs.py	✅ PASS	14.5s	$0.00
01_standalone_sdk/22_anthropic_thinking.py	✅ PASS	24.1s	$0.01
01_standalone_sdk/23_responses_reasoning.py	✅ PASS	50.9s	$0.02
01_standalone_sdk/24_planning_agent_workflow.py	✅ PASS	5m 21s	$0.41
01_standalone_sdk/25_agent_delegation.py	✅ PASS	1m 4s	$0.06
01_standalone_sdk/26_custom_visualizer.py	✅ PASS	21.5s	$0.02
01_standalone_sdk/28_ask_agent_example.py	✅ PASS	32.2s	$0.04
01_standalone_sdk/29_llm_streaming.py	✅ PASS	32.6s	$0.02
01_standalone_sdk/30_tom_agent.py	✅ PASS	13.1s	$0.01
01_standalone_sdk/31_iterative_refinement.py	✅ PASS	1m 16s	$0.08
01_standalone_sdk/32_configurable_security_policy.py	✅ PASS	18.5s	$0.02
01_standalone_sdk/33_hooks/main.py	✅ PASS	29.9s	$0.03
01_standalone_sdk/34_critic_example.py	✅ PASS	33.0s	$0.02
01_standalone_sdk/36_event_json_to_openai_messages.py	✅ PASS	12.0s	$0.00
01_standalone_sdk/37_llm_profile_store/main.py	✅ PASS	7.6s	$0.00
01_standalone_sdk/38_browser_session_recording.py	✅ PASS	26.3s	$0.04
01_standalone_sdk/39_llm_fallback.py	✅ PASS	13.9s	$0.01
01_standalone_sdk/40_acp_agent_example.py	✅ PASS	59.0s	$0.40
01_standalone_sdk/41_task_tool_set.py	✅ PASS	25.3s	$0.03
01_standalone_sdk/42_file_based_subagents.py	✅ PASS	38.7s	$0.04
01_standalone_sdk/44_model_switching_in_convo.py	✅ PASS	11.9s	$0.01
01_standalone_sdk/45_parallel_tool_execution.py	✅ PASS	4m 21s	$0.63
01_standalone_sdk/46_agent_settings.py	✅ PASS	13.8s	$0.00
01_standalone_sdk/47_defense_in_depth_security.py	✅ PASS	5.6s	$0.00
01_standalone_sdk/48_conversation_fork.py	✅ PASS	22.7s	$0.01
01_standalone_sdk/49_switch_llm_tool.py	✅ PASS	7.7s	$0.04
01_standalone_sdk/50_async_cancellation.py	✅ PASS	14.2s	$0.00
01_standalone_sdk/51_agent_hooks/main.py	✅ PASS	40.6s	$0.06
01_standalone_sdk/52_dynamic_workflow.py	✅ PASS	3m 19s	$0.14
01_standalone_sdk/53_client_defined_tools.py	✅ PASS	15.6s	$0.01
01_standalone_sdk/54_goal_completion_loop.py	✅ PASS	24.5s	$0.03
01_standalone_sdk/55_persistent_memory.py	✅ PASS	17.7s	$0.02
01_standalone_sdk/56_structured_output.py	✅ PASS	44.8s	$0.06
01_standalone_sdk/57_prompt_hooks/main.py	✅ PASS	11.7s	$0.00
01_standalone_sdk/58_ask_oracle_tool/main.py	✅ PASS	16.1s	$0.01
01_standalone_sdk/59_route_task_to_model.py	✅ PASS	10.1s	$0.05
02_remote_agent_server/01_convo_with_local_agent_server.py	✅ PASS	34.2s	$0.02
02_remote_agent_server/02_convo_with_docker_sandboxed_server.py	✅ PASS	1m 28s	$0.04
02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py	✅ PASS	2m 8s	$0.11
02_remote_agent_server/04_convo_with_api_sandboxed_server.py	✅ PASS	1m 59s	$0.04
02_remote_agent_server/06_custom_tool/main.py	✅ PASS	5m 29s	$0.06
02_remote_agent_server/07_convo_with_cloud_workspace.py	✅ PASS	48.7s	$0.03
02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py	✅ PASS	4m 12s	$0.02
02_remote_agent_server/09_acp_agent_with_remote_runtime.py	✅ PASS	1m 59s	$0.50
02_remote_agent_server/10_cloud_workspace_share_credentials.py	✅ PASS	38.7s	$0.00
02_remote_agent_server/11_conversation_fork.py	✅ PASS	1m 15s	$0.00
02_remote_agent_server/12_settings_and_secrets_api.py	✅ PASS	2m 31s	$0.01
02_remote_agent_server/13_workspace_get_llm.py	✅ PASS	33.0s	$0.01
02_remote_agent_server/14_client_defined_tools.py	✅ PASS	44.6s	$0.02
02_remote_agent_server/15_openai_compatible_gateway.py	✅ PASS	24.5s	$0.02
02_remote_agent_server/16_deferred_init.py	✅ PASS	18.3s	$0.00
04_llm_specific_tools/01_gpt5_apply_patch_preset.py	✅ PASS	30.0s	$0.03
04_llm_specific_tools/02_gemini_file_tools.py	✅ PASS	32.6s	$0.08
05_skills_and_plugins/01_loading_agentskills/main.py	✅ PASS	14.5s	$0.02
05_skills_and_plugins/02_loading_plugins/main.py	✅ PASS	20.0s	$0.03
05_skills_and_plugins/04_mixed_marketplace_skills/main.py	✅ PASS	5.2s	$0.00

✅ All tests passed!

Total: 70 | Passed: 70 | Failed: 0 | Total Cost: $3.85

View full workflow run

malhotra5

LGTM

Release v1.52.0

7ec6116

Co-authored-by: openhands <openhands@all-hands.dev>

all-hands-bot added integration-test Runs the integration tests and comments the results test-examples Run all applicable "examples/" files. Expensive operation. behavior-test security-scan Run the release security scan (approval-drift + supply-chain) on this PR labels Oct 4, 2026

Merge branch 'main' into rel-1.52.0

a756fd8

malhotra5 added test-examples Run all applicable "examples/" files. Expensive operation. and removed test-examples Run all applicable "examples/" files. Expensive operation. labels Oct 5, 2026

malhotra5 approved these changes Oct 5, 2026

View reviewed changes

malhotra5 merged commit 229b2b9 into main Oct 5, 2026
66 of 68 checks passed

malhotra5 deleted the rel-1.52.0 branch October 5, 2026 04:09

Conversation

all-hands-bot commented Oct 4, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release v1.52.0

Release Checklist

What happens on merge

Uh oh!

github-actions Bot commented Oct 4, 2026

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Release Security Scan

🔒 Approval drift (time-of-check vs time-of-use)

📦 Supply-chain dependency diff

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

REST API breakage checks (OpenAPI) — ✅ PASSED

Uh oh!

github-actions Bot commented Oct 4, 2026

Uh oh!

github-actions Bot commented Oct 4, 2026

🧪 Integration Tests Results

📊 Summary

📋 Detailed Results

litellm_proxy_openai_gpt_5.5

litellm_proxy_gemini_3.1_pro_preview

litellm_proxy_anthropic_claude_sonnet_4_6

litellm_proxy_deepseek_deepseek_v4_flash

litellm_proxy_minimax_MiniMax_M2.7

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

malhotra5 commented Oct 4, 2026

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔄 Running Examples with openhands/claude-haiku-4-5-20251001

❌ Some tests failed

Uh oh!

github-actions Bot commented Oct 4, 2026

🧪 Behavior Tests Results

📊 Summary

📋 Detailed Results

litellm_proxy_openai_gpt_5.5

litellm_proxy_gemini_3.1_pro_preview

litellm_proxy_anthropic_claude_sonnet_4_6

litellm_proxy_deepseek_deepseek_v4_flash

litellm_proxy_minimax_MiniMax_M2.7

Uh oh!

github-actions Bot commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔄 Running Examples with openhands/claude-haiku-4-5-20251001

✅ All tests passed!

Uh oh!

malhotra5 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

all-hands-bot commented Oct 4, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

🔄 Running Examples with `openhands/claude-haiku-4-5-20251001`

github-actions Bot commented Oct 5, 2026 •

edited

Loading

🔄 Running Examples with `openhands/claude-haiku-4-5-20251001`