Skip to content

Add smoke tests for model configs#323205

Draft
pwang347 wants to merge 13 commits into
mainfrom
pawang/addModelConfigSmokeTests
Draft

Add smoke tests for model configs#323205
pwang347 wants to merge 13 commits into
mainfrom
pawang/addModelConfigSmokeTests

Conversation

@pwang347

@pwang347 pwang347 commented Jun 26, 2026

Copy link
Copy Markdown
Member

Flake busting #323242

Copilot AI review requested due to automatic review settings June 26, 2026 20:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds a new end-to-end smoke test suite that validates chat model configuration selections (reasoning effort + context size) are forwarded to the LLM request payload, using the existing mock LLM server infrastructure and new automation helpers for interacting with the chat model picker UI.

Changes:

  • Register and run a new “Chat Model Configuration” smoke suite in the Electron (non-remote) smoke test run.
  • Add chat automation helpers to select a model and its configuration options via the model picker UI.
  • Extend the mock LLM server to expose captured request bodies for assertions, and add a dedicated mock model that supports both reasoning effort and a long-context tier.
Show a summary per file
File Description
test/smoke/src/main.ts Wires the new chat model configuration smoke suite into the Electron smoke run.
test/smoke/src/areas/chat/chatModelConfig.test.ts New smoke suite that drives the model picker UI and asserts forwarded reasoning.effort and context-management threshold in /responses.
test/automation/src/chat.ts Adds automation methods for selecting a chat model and interacting with the model configuration dropdown.
scripts/chat-simulation/common/mock-llm-server.ts Adds a mock model with config pickers and exposes captured request bodies via getRequests().

Review details

  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Low

Comment thread test/smoke/src/areas/chat/chatModelConfig.test.ts
Comment thread scripts/chat-simulation/common/mock-llm-server.ts
@pwang347 pwang347 marked this pull request as ready for review June 27, 2026 05:15
@pwang347 pwang347 marked this pull request as draft June 27, 2026 05:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants