Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 28 additions & 19 deletions frontend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,18 @@ server that `veadk frontend` launches — no separate backend.
action is required.
- **Existing Agent migration**: upload a local project ZIP for read-only
analysis, confirm the detected framework and entry point, then migrate and
validate it in a temporary Sandbox. Successful migration source is saved as
an immutable version in the same private Studio TOS project store. The
separate “已迁移项目” page can view, download, deploy, delete, and compare
versions; any version can be restored into the intelligent-development flow
for another intent-driven iteration after the temporary migration environment
has ended.
validate it in a temporary Sandbox. An optional migration-effect evaluation
is off by default; when enabled, users can enter 1–100 real user questions by
hand or bulk paste, while expected outcomes and criteria remain optional.
Standard evaluation uses three dimensions; users can instead select custom
dimensions before upload. The locked dataset and final HTML report are stored
as immutable owner-only TOS assets. The report is fetched and rendered in a
side drawer only after the user selects “View report,” and remains available
for download. Successful migration source is saved as an
immutable version in the same private Studio TOS project store. The separate
“已迁移项目” page can view, download, deploy, delete, and compare versions;
any version can be restored into the intelligent-development flow for another
intent-driven iteration after the temporary migration environment has ended.
- **Reasoning & tool calls** shown inline (collapsible "thinking", tool blocks).
- **Agent context rail** keeps the selected Agent's description, model, tools,
skills, and optional live multi-Agent topology together in the conversation's
Expand Down Expand Up @@ -261,19 +267,22 @@ server that `veadk frontend` launches — no separate backend.
creation, and service publishing as separate deployment stages.
- **Existing-project migration**: upload one local ZIP of at most 20 MiB from
the add-Agent menu. Studio creates one user-owned Dev Sandbox Session with a
one-hour TTL, then asks the preinstalled Codex to perform read-only framework,
entry-point, and migration-boundary analysis. Migration starts only after the
user confirms the framework, entry point, and open questions. Structured
frameworks run the preinstalled `ak migrate`; Dify and Any projects run
`ak migrate --execution in-place` with Codex in the same Session. State,
logs, and artifacts remain only under
`/home/gem/.studio/migration/v1/` in that Session. Preview, download, and
Runtime deployment stop when the Session expires. Runtime deployment resolves
and verifies the owned Session artifact on the server instead of trusting
browser-provided files or entry points. AgentKit CLI `0.51.1` is only the
current baseline; these CLI changes must be released as a new version. The
Dev Sandbox image must pin that migration-capable release and its SHA256 at
image build time.
one-hour TTL, extended to two hours when effect evaluation is enabled, then
asks the preinstalled Codex to perform read-only framework, entry-point, and
migration-boundary analysis. Migration starts only after the user confirms
the framework, entry point, and open questions. Structured frameworks run the
preinstalled `ak migrate`; Dify and Any projects run
`ak migrate --execution in-place` with Codex in the same Session. Evaluation
deploys a temporary Runtime, checkpoints per-case execution as JSONL, judges
batches in one fresh resumable Codex thread, and always reconciles Runtime
cleanup before completing or cancelling. Reports show 0–100 display scores,
execution success, evidence coverage, N/A counts, low-scoring and failed
cases, versions, evidence severity, and cleanup status without a pass/fail
verdict. Evaluation failure never hides or rolls back the migration artifact.
Runtime deployment resolves and verifies the owned Session artifact on the
server instead of trusting browser-provided files or entry points. The Dev
Sandbox image must pin AgentKit CLI `0.52.16` and its SHA256 at image build
time.
- **Built-in code execution**: selecting `代码执行` adds VeADK's `run_code`
tool to generated Python and reveals the required `AGENTKIT_TOOL_ID` sandbox
field and optional `AGENTKIT_TOOL_REGION` field below the built-in tool list.
Expand Down
24 changes: 23 additions & 1 deletion frontend/server/migration/contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
is_valid_model_id,
is_valid_structured_entry,
)
from .evaluation.dimensions import EVALUATION_DIMENSION_IDS, STANDARD_DIMENSION_IDS

_MAX_PATH_BYTES = 4 * 1024
_MAX_PATH_DEPTH = 64
Expand Down Expand Up @@ -184,7 +185,7 @@ def validate_migration_request(
"session_ttl_seconds",
"created_at",
},
optional={"model_id"},
optional={"model_id", "evaluation"},
)
if (
value.get("schema_version") != 1
Expand All @@ -201,6 +202,27 @@ def validate_migration_request(
_text(value.get("instruction"), maximum=_MAX_TEXT_LENGTH)
if "model_id" in value and not is_valid_model_id(value.get("model_id")):
raise MigrationContractError("invalid model id")
evaluation = value.get("evaluation")
if evaluation is not None:
if not isinstance(evaluation, dict):
raise MigrationContractError("invalid evaluation config")
_exact_keys(
evaluation,
required={"enabled", "preset", "dimensions"},
)
enabled = evaluation.get("enabled")
preset = evaluation.get("preset")
dimensions = evaluation.get("dimensions")
if (
enabled is not True
or preset not in {"standard", "custom"}
or not isinstance(dimensions, list)
or not dimensions
or len(set(str(item) for item in dimensions)) != len(dimensions)
or any(item not in EVALUATION_DIMENSION_IDS for item in dimensions)
or (preset == "standard" and tuple(dimensions) != STANDARD_DIMENSION_IDS)
):
raise MigrationContractError("invalid evaluation config")
created_at = value.get("created_at")
if isinstance(created_at, str):
_timestamp_text(created_at)
Expand Down
35 changes: 35 additions & 0 deletions frontend/server/migration/evaluation/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Migration-effect evaluation for AgentKit Studio."""

from .dimensions import (
EVALUATION_DIMENSIONS,
STANDARD_DIMENSION_IDS,
EvaluationDimensionId,
)
from .models import (
EvaluationCaseBody,
EvaluationDatasetBody,
MigrationEvaluationConfig,
)

__all__ = [
"EVALUATION_DIMENSIONS",
"STANDARD_DIMENSION_IDS",
"EvaluationCaseBody",
"EvaluationDatasetBody",
"EvaluationDimensionId",
"MigrationEvaluationConfig",
]
Loading
Loading