Skip to content

[feat][evaluation] add RunModeConfig.SkillsMode for sandbox agent skills - #645

Merged
bitholic merged 2 commits into
mainfrom
feat/evaluation-skills-mode-field
Sep 2, 2026
Merged

[feat][evaluation] add RunModeConfig.SkillsMode for sandbox agent skills#645
bitholic merged 2 commits into
mainfrom
feat/evaluation-skills-mode-field

Conversation

@bitholic

Copy link
Copy Markdown
Collaborator

Background

Add skills_mode to RunModeConfig, declaring how a sandbox-agent experiment combines test-case-level and experiment-level Agent Skills (merge / disable_test_case). Forward-compatible (optional field).

This is the OSS half of a two-repo change. The commercial fork (cozeloop-commercial) emits this field to the case-file experiment_info.skills_mode, and carries the test-case-level skills list to dataset_item.skills (aligned with runtime testcase.TestCase.Skills). See companion commercial MR.

Changes

  • idl/thrift/.../domain/expt.thrift + domain_openapi/experiment.thrift: add skills_mode field to RunModeConfig
  • backend/.../domain/entity/expt.go: RunModeConfig.SkillsMode
  • backend/.../application/convertor/experiment/{expt,openapi}.go: wire the field through convertors
  • backend/kitex_gen/...: regenerated via code_gen.sh

Verification

  • cd backend && go build ./...

🤖 Generated with Claude Code

Add skills_mode field to RunModeConfig (domain/openapi IDL + entity +
convertor), declaring how sandbox-agent experiment combines test-case-level
and experiment-level Agent Skills (merge / disable_test_case).

The field rides on the experiment-level RunModeConfig, emitted to case-file
experiment_info.skills_mode by the commercial fork; the test-case-level
skills list is carried separately. Forward-compatible (optional field).

Co-Authored-By: Claude <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Aug 29, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Impacted file tree graph

@@           Coverage Diff            @@
##             main     #645    +/-   ##
========================================
  Coverage   78.47%   78.48%            
========================================
  Files         696      696            
  Lines       85755    85978   +223     
========================================
+ Hits        67299    67478   +179     
- Misses      14520    14567    +47     
+ Partials     3936     3933     -3     
Flag Coverage Δ
unittests 78.48% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...valuation/application/convertor/experiment/expt.go 82.65% <100.00%> (+0.05%) ⬆️
...uation/application/convertor/experiment/openapi.go 92.41% <100.00%> (-0.58%) ⬇️
backend/modules/evaluation/domain/entity/expt.go 82.24% <ø> (ø)

... and 10 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7469d17...a3ba7a2. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…overage

Co-Authored-By: Claude <noreply@anthropic.com>
@bitholic
bitholic merged commit 2d02f40 into main Sep 2, 2026
17 checks passed
@bitholic
bitholic deleted the feat/evaluation-skills-mode-field branch September 2, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants