Skip to content

fix(smoketests): give the snapshot devbox tests the shared provisioning timeout - #841

Open
jrvb-rl wants to merge 1 commit into
mainfrom
rob/smoketest-devbox-timeouts
Open

jrvb-rl wants to merge 1 commit into
mainfrom
rob/smoketest-devbox-timeouts

Conversation

@jrvb-rl

@jrvb-rl jrvb-rl commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

User description

Description

Both tests in tests/smoketests/snapshots.test.ts override jest's 5 minute smoketest default with a 30 second per-test timeout even though each one provisions a devbox. They now use the shared MEDIUM_TIMEOUT (5 minutes), matching the other smoketests that create devboxes.

Motivation

In the dev smoketest run for runloopai/runloop deploy 35385327925, snapshot devbox failed on both the http1 and http2 jobs with Exceeded timeout of 30000 ms. Devbox provisioning in the same run took 35 to 70 seconds because node agents were still restarting after the deploy. That is inside the platform's provisioning budget, so the test was failing on latency rather than on a regression. The timeout should express the budget, not the typical fast path.

Testing

Timeout-only change to two existing smoketests; no new assertions. The smoketest workflow runs against a live environment and is exercised by the runloop deploy pipeline.

🤖 Generated with Claude Code


CodeAnt-AI Description

Allow snapshot devbox smoke tests enough time for provisioning

What Changed

  • Snapshot creation and snapshot launch tests now allow up to five minutes instead of 30 seconds
  • Tests no longer fail when devbox provisioning takes longer during deployments or node-agent restarts

Impact

✅ Fewer false smoketest failures
✅ Reliable snapshot validation during slower provisioning

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

…ng timeout

Both tests in snapshots.test.ts override jest's 5 minute smoketest default
with a 30 second per-test timeout even though each one provisions a devbox.
Devbox provisioning routinely takes 35 to 70 seconds right after a dev
deploy while node agents restart, so the tests failed on latency that is
within the platform's provisioning budget rather than on a real regression.

Use the shared MEDIUM_TIMEOUT (5 minutes) like the other devbox-creating
smoketests do.
@reflex-loop

reflex-loop Bot commented Sep 18, 2026

Copy link
Copy Markdown

🔄 Reflex agent status: Working

Agent Status
agt_2uXI0PQI6y6txpGSr0ppMm 🔄 Working

This PR was created by Reflex.

This comment updates in place as agents work.

@jrvb-rl
jrvb-rl marked this pull request as ready for review September 20, 2026 21:43
@codeant-ai

codeant-ai Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR 7ea025c Sep 20, 2026 · 21:43 21:45

@codeant-ai

codeant-ai Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@jrvb-rl
jrvb-rl requested review from james-rl and wall-rl September 20, 2026 21:44
@codeant-ai codeant-ai Bot added the size:XS This PR changes 0-9 lines, ignoring generated files label Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant