Skip to content

fix(e2e): stabilize runtime template tests - #2619

Merged
jariy17 merged 1 commit into
aws:refactorfrom
notgitika:fix/e2e-python-315
Oct 19, 2026
Merged

jariy17 merged 1 commit into
aws:refactorfrom
notgitika:fix/e2e-python-315

Conversation

@notgitika

Copy link
Copy Markdown
Contributor

Description

Stabilizes the runtime-template E2E suite across the failures observed in:

  • Windows job 118219256686
  • Linux job 118219256715
  • Earlier Windows and macOS failures from run 39286608485

The changes:

  • Select Python 3.14 in the E2E workflow so uv does not choose unsupported
    CPython 3.15 and attempt native dependency builds on Windows.
  • Give each concurrent runtime-template scenario a distinct local development
    port instead of allowing separate agentcore dev processes to race for the
    same protocol defaults.
  • Preserve the scenario's primary failure when best-effort cleanup of a project
    that never deployed also fails.

Related Issue

Follow-up to #2605 and the linked failing Actions runs.

Documentation PR

Not applicable; this changes CI and E2E test orchestration only.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • I ran bun test (4,685 passed)
  • I ran the relevant end-to-end tests with bun run test:e2e, or explained why they are not applicable
    • Not run locally because the suite provisions AWS resources; the PR workflow will exercise it on all supported runners.
  • I ran bun run typecheck
  • I ran bun run lint:check
  • I ran bun run format:check
  • I ran bun run build
  • If I modified src/assets/, I updated affected snapshots with bun test <test-file> --update-snapshots and committed them
    • Not applicable; no assets changed.

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

@github-actions github-actions Bot added the size/s PR size: S label Oct 19, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Oct 19, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 19, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Narrowly scoped CI/E2E stabilization PR. I checked the three behavior changes:

  • Pinning uv to Python 3.14 matches the comment ("newest Python supported by AgentCore CodeZip") — PYTHON_3_14 is the newest value referenced in src/core/project/runtimeBuild.ts, so this is accurate, and 3.14 is stable upstream.
  • Per-scenario --port offset uses LOCAL_DEV_PORT_BASE + RUNTIME_TEMPLATES.indexOf(runtime). Since RUNTIME_TEMPLATES is the single post-filter array shared by every helper, indices are stable per run, and the resulting range (18080..18088) is well above protocol defaults. --port is a real flag on agentcore dev (src/handlers/project/dev/index.ts).
  • scenarioCompleted guard correctly preserves the primary scenario failure: on scenario failure the parseResult on the cleanup deploy is skipped, so the original error propagates out of the try; on success, cleanup is still validated. The trade-off of silently swallowing a cleanup-only failure when the scenario already failed is the stated intent and is reasonable for a finally block.

No test/telemetry concerns — this touches only E2E orchestration.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Oct 19, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.61%. Comparing base (2561439) to head (f58f031).

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2619   +/-   ##
=========================================
  Coverage     96.61%   96.61%           
=========================================
  Files           702      702           
  Lines         58344    58344           
=========================================
  Hits          56369    56369           
  Misses         1975     1975           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jariy17
jariy17 merged commit 9e9de05 into aws:refactor Oct 19, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants