automations: refactor: make provider session templates canonical - #334521
automations: refactor: make provider session templates canonical#334521Ulugbek Abdullaev (ulugbekna) wants to merge 15 commits into
Conversation
Complete the legacy Automation mapping by preserving Autopilot on the Agent Host mode axis while applying Assisted approvals. Ignore generic chat modes at the AHP boundary and revalidate elevated Copilot approvals against current policy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Preserve provider-owned configuration across compatibility edits, repair existing and provider-less Copilot Automation definitions, and reset incompatible state on retargeting. Enforce managed auto-approval policy at both SDK and host decision points while keeping saved preferences intact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Add a versioned provider-neutral session template projection for model, agent, and opaque configuration. Keep legacy flat writers functional during migration, preserve unknown AHP state on edits and transfers, and clear incompatible configuration on retarget. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Let Automation drafts restore and capture provider-owned model, agent, and resolved configuration through the Sessions provider contract. Keep normal New Session defaults isolated, reject replaced draft snapshots, and exclude transient or target-owned values. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Drive Automation configuration through a scoped session draft and the same provider-owned pickers as New Session. Capture complete provider state for Agent Host and legacy Copilot paths while preserving unavailable, opaque, removed, and policy-clamped preferences. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Pass the complete provider-owned Automation configuration during older-host draft creation so browser fallback and AHP execution use the same template semantics. Keep workspace isolation and branch configuration target-owned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Round-trip complete provider session templates through Automation tools and stop new dialog and AHP projections from writing flattened aliases. Keep legacy rows and inputs compatible while enforcing template-first execution, duplication, telemetry, and rollback semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Specify canonical Automation template and draft ownership across Sessions and Agent Host. Add recorded AHP coverage proving independent Mode and Approvals survive host restart into the created run session, and document the remaining Claude/Codex coverage gap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Keep canonical provider templates opaque and authoritative while limiting legacy Autopilot repair to load/import boundaries. Bound dialog capture and preserve per-target configuration across failures and retargeting, retain legacy duplicate/worktree settings, and improve loading accessibility. Add regression coverage for #333723 compatibility, canonical reloads, capture races, and tool configuration limits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Keep the Automation dialog open when provider configuration cannot be captured, expose cancellable saving progress, and serialize draft retargeting. Unify canonical-template authority across stores, require providers to advertise restoration support, preserve definition-owned state, retain scoped picker models across toolbar rebuilds, and keep legacy fallback configuration available. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Send providers one canonical Automation configuration object instead of overlapping template channels. Keep keyboard focus on the cancellable action while form content is inert, and make saving and error live regions visible before their announcements change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Refresh the created run session inside the completion retry so the restart E2E validates Mode and Approvals on the settled session rather than an earlier catalog snapshot. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Exclude aria-disabled controls from the Automation dialog's custom focus ring so Saving keeps keyboard focus on the cancellable action instead of moving onto the disabled primary button. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
Avoid extending the widely shared Dialog widget for the Automation editor's provider capture flow. Keep Save failure handling, cancellation, and completion in the Automation dialog while reusing the standard button styles and platform order. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
990b621 to
96e0625
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Reserved configuration can become unattended permission grants, and reset, fallback-agent, provider-unavailable, and keyboard paths remain incorrect.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 1
New issues introduced by this change (4)
| Severity | Finding |
|---|---|
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAutomationStore.ts — The canonical template is merged without removing platform-owned keys. A configureAutomation… |
|
src/vs/sessions/contrib/automations/browser/automationDialog.ts — Unavailable saved targets are treated as synchronization failures here, so Save reports a capture… |
|
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAutomationStore.ts — An explicit sessionTemplate: null reaches this branch with no template or aliases, but the legacy… |
|
src/vs/platform/actions/common/actions.ts — This menu is owned and consumed only by the Sessions Automation UI. Sessions menu IDs are… |
What changed in this PR
Makes provider-owned session templates canonical for Automations while retaining legacy compatibility.
Changes:
- Adds template persistence, migration, validation, and AHP projection.
- Reuses provider-backed New Session controls and aligns fallback execution.
- Adds policy enforcement and broad unit/E2E coverage.
| File | Description |
|---|---|
src/vs/workbench/contrib/chat/common/automations/automationTelemetry.ts |
Updates telemetry descriptions. |
src/vs/workbench/contrib/chat/common/automations/automationService.ts |
Adds template authority and serialization. |
src/vs/workbench/contrib/chat/common/automations/automation.ts |
Defines canonical templates. |
src/vs/workbench/contrib/chat/browser/widget/input/chatInputPart.ts |
Supports custom input toolbar menus. |
src/vs/workbench/contrib/chat/browser/actions/chatExecuteActions.ts |
Hides the generic permission picker. |
src/vs/sessions/SESSIONS.md |
Documents Automation draft contracts. |
src/vs/sessions/services/sessions/test/browser/sessionsManagementService.test.ts |
Tests template restoration and isolation. |
src/vs/sessions/services/sessions/test/browser/sessionNavigation.test.ts |
Updates service mock capabilities. |
src/vs/sessions/services/sessions/common/sessionsProvider.ts |
Extends provider configuration contracts. |
src/vs/sessions/services/sessions/common/sessionsManagement.ts |
Extends management APIs. |
src/vs/sessions/services/sessions/browser/sessionsManagementService.ts |
Routes templates into provider drafts. |
src/vs/sessions/contrib/sessions/test/browser/automationsView.test.ts |
Tests canonical duplication. |
src/vs/sessions/contrib/sessions/browser/views/automationsView.ts |
Preserves canonical templates when duplicating. |
src/vs/sessions/contrib/providers/copilotChatSessions/test/browser/sandboxPicker.test.ts |
Updates constructor arguments. |
src/vs/sessions/contrib/providers/copilotChatSessions/test/browser/copilotChatSessionsProvider.test.ts |
Tests fallback restore and capture. |
src/vs/sessions/contrib/providers/copilotChatSessions/browser/copilotChatSessionsProvider.ts |
Implements fallback template support. |
src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts |
Tests Agent Host template behavior. |
src/vs/sessions/contrib/providers/agentHost/browser/mobile/mobileChatInputConfigPicker.ts |
Uses provider capabilities for mobile UI. |
src/vs/sessions/contrib/providers/agentHost/browser/baseAgentHostSessionsProvider.ts |
Restores and captures Agent Host templates. |
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAutomationStore.ts |
Projects templates to AHP definitions. |
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAgentPicker.ts |
Scopes cached picker models. |
src/vs/sessions/contrib/providers/agentHost/AGENT_HOST_SESSIONS_PROVIDER.md |
Documents provider behavior. |
src/vs/sessions/contrib/chat/browser/newSessionConfigToolbars.ts |
Extracts shared toolbar helpers. |
src/vs/sessions/contrib/chat/browser/newChatInput.ts |
Reuses toolbar helpers. |
src/vs/sessions/contrib/automations/test/browser/providerAutomationService.test.ts |
Updates migration expectations. |
src/vs/sessions/contrib/automations/test/browser/automationTools.test.ts |
Tests template tool validation. |
src/vs/sessions/contrib/automations/test/browser/automationService.test.ts |
Tests schema and template persistence. |
src/vs/sessions/contrib/automations/test/browser/automationRunner.test.ts |
Tests fallback template forwarding. |
src/vs/sessions/contrib/automations/test/browser/automationDialog.test.ts |
Tests draft synchronization and capture. |
src/vs/sessions/contrib/automations/browser/providerAutomationService.ts |
Migrates provider-backed definitions canonically. |
src/vs/sessions/contrib/automations/browser/media/automationDialog.css |
Styles provider controls and save state. |
src/vs/sessions/contrib/automations/browser/automationTools.ts |
Exposes and validates canonical templates. |
src/vs/sessions/contrib/automations/browser/automationService.ts |
Persists schema-v4 templates. |
src/vs/sessions/contrib/automations/browser/automationRunner.ts |
Aligns fallback session creation. |
src/vs/sessions/contrib/automations/browser/automationDialog.ts |
Manages provider drafts and controls. |
src/vs/sessions/contrib/automations/browser/automationDialogService.ts |
Adds asynchronous save lifecycle. |
src/vs/sessions/AUTOMATIONS.md |
Documents canonical template ownership. |
src/vs/platform/agentHost/test/node/sessionPermissions.test.ts |
Tests policy clamping. |
src/vs/platform/agentHost/test/node/e2e/suites/automationsSuite.ts |
Adds restart execution coverage. |
src/vs/platform/agentHost/test/node/e2e/suites/agentHostE2ESuites.ts |
Registers Automation parity tests. |
src/vs/platform/agentHost/test/node/e2e/KNOWN_ISSUES.md |
Tracks missing provider coverage. |
src/vs/platform/agentHost/test/node/e2e/captures/copilotcli-an-automation-run-restores-mode-and-approvals-after-host-restart.yaml |
Records Copilot replay traffic. |
src/vs/platform/agentHost/test/node/copilotAgentSession.test.ts |
Tests live policy revocation. |
src/vs/platform/agentHost/test/node/agentHostAutomationService.test.ts |
Tests stored migration. |
src/vs/platform/agentHost/test/common/automationMigration.test.ts |
Tests legacy Autopilot migration. |
src/vs/platform/agentHost/node/sessionPermissions.ts |
Enforces approval policy at runtime. |
src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts |
Revokes elevated Copilot permissions. |
src/vs/platform/agentHost/node/agentHostAutomationService.ts |
Migrates persisted definitions. |
src/vs/platform/agentHost/common/sessionConfigKeys.ts |
Defines reusable-template exclusions. |
src/vs/platform/agentHost/common/automationMigration.ts |
Implements compatibility migration. |
src/vs/platform/actions/common/actions.ts |
Adds an Automation toolbar menu ID. |
Suppressed comments (3)
src/vs/sessions/contrib/providers/agentHost/browser/baseAgentHostSessionsProvider.ts:3412
- Fallback draft creation also forwards reserved template keys unchanged. This bypasses the AHP projection path, so a canonical template from the legacy store/tool can seed per-tool
permissionsor transientshellInitScriptsinto an unattended run. Strip non-reusable values before normalizing the template config.
src/vs/sessions/contrib/providers/copilotChatSessions/browser/copilotChatSessionsProvider.ts:1735 - Marking the Copilot provider as supporting canonical Automation templates requires restoring every template axis, but this method never applies
template.agent. A canonical row containing onlyagenttherefore opens and runs with the provider default mode, while capture merely copies the stale agent field back, so the custom-agent selection is not actually editable or executable in this fallback provider. Restore the agent into the provider's custom-mode model and derive/clear it from the live selection during capture.
src/vs/sessions/contrib/automations/browser/automationDialogService.ts:319 - The custom focus trap receives only elements found inside the form, while Save and Cancel are rendered in the separate footer. It intercepts and stops Tab navigation, and the base dialog only discovers links—not custom footer buttons—so keyboard users cannot tab to either action; while the form is inert during save, Tab can also fall into the base handler with no focusable items. Include the footer buttons in this focus order, keeping Cancel reachable during saving.
const keyboardNavigation = disposables.add(registerAutomationDialogKeyboardNavigation(
DOM.getWindow(container),
() => [
...getFocusableElements(),
...(saveButton ? [saveButton.element] : []),
...(cancelButton ? [cancelButton.element] : []),
],
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Filter reserved session state at both Agent Host projection and fallback restoration, preserve configuration for unavailable targets, honor explicit template reset semantics, and keep the Automation prompt menu owned by Sessions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 09e55c0c-bf80-42ea-b8e2-fad38a12d986
|
Copilot review |
I re-checked the branch state and there are no new file changes to submit since |
📬 CODENOTIFYThe following users are being notified based on files changed in this PR: Sandeep Somavarapu (@sandy081)Matched files:
Ladislau Szomoru (@lszomoru)Matched files:
|
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
| private async _resolveAutomationSessionMode(session: CopilotCLISession, modeId: string): Promise<void> { | ||
| const modes = this.chatModeService.createModes(session.resource); | ||
| try { | ||
| await modes.waitForPendingUpdates(); |
There was a problem hiding this comment.
[Experimental performance review bot]
Severity: medium
Initial Automation configuration now launches _resolveAutomationSessionMode with void; that method owns a ChatModes instance but disposes it only after waitForPendingUpdates() settles, with no timeout or cancellation tied to deletion, replacement, send completion, or provider disposal.
If custom-agent discovery is hung or delayed by an unavailable/restarting extension host, each older-host/browser Automation run using an unresolved provider/custom mode leaves another ChatModes object, subscriptions, refresh token, and session closure alive. Recurring Automations then cause renderer memory and listener counts to grow for the lifetime of the stalled discovery.
Suggested fix: Register the resolver/ChatModes instance with the draft or provider lifecycle and race pending discovery with that cancellation (and a bounded timeout). Dispose it immediately when the draft is replaced, deleted, committed, or the provider shuts down; also coalesce discovery where multiple runs resolve the same session scope.
(Written by Copilot)



Summary
This is the architectural follow-up to #333723 and #333949.
Automations are now stored and executed through Agent Host Protocol (AHP), but the workbench still edited their session configuration through the legacy flattened
modelId, generic chatmode, andpermissionLevelfields. That representation cannot faithfully describe provider-native AHP configuration and caused Mode and Approvals to be conflated or dropped.This change makes a provider-owned session template canonical for new Automations:
It then uses a real provider draft and the existing New Session controls to edit that template, and passes the same initial configuration through both AHP-owned execution and older-host/browser fallback.
Why
The old projection had several structural problems:
agent,ask, andedit, while Agent Host providers use their own values such asinteractive,plan, andautopilot;permissionLevel: 'autopilot'represented two now-independent axes:Architecture
Target and template have separate authority
The Automation target owns:
The provider session template owns:
Shared Sessions code remains provider-neutral and does not interpret Copilot-, Claude-, or Codex-specific keys.
Canonical session template
IAutomationSessionTemplatecarries:The existing flat fields remain only as deprecated decode/input aliases for old ledgers and callers. Once an Automation has a canonical template, a shared authority guard prevents stale legacy aliases from mutating it.
Provider-owned Automation drafts
The Automation dialog creates a dedicated provider draft, separate from the normal New Session composer draft.
Providers opt into:
automationConfigurationduring draft creation;The draft synchronizer:
A capture failure therefore cannot be mistaken for a successful request to reset configuration.
Reuse New Session controls
The dialog now renders the same
NewSessionConfigandNewSessionControlcontributions used by New Session, scoped to the Automation draft.This gives each selected provider its actual Model, Agent, Mode, and Approvals controls. Repository configuration is intentionally not reused because the Automation target already owns workspace, isolation, and branch.
ChatInputPartremains the prompt editor but uses Automation-specific toolbar menus so generic configuration controls are not duplicated.Explicit Automation picker changes update the same remembered provider selections used by New Session. Merely opening or cancelling an edit does not.
Lossless AHP projection
The Agent Host Automation store now projects:
The projection preserves model, custom agent, and provider config.
Reusable templates exclude values owned by a concrete run or target, including:
When writing an edited template back, existing definition-owned values are preserved separately.
Same-provider edits preserve authoritative values the editor has no opinion about. Retargeting to another provider does not carry incompatible model/agent/config state.
AHP and fallback execution converge
AHP execution already creates the run session from
AutomationDefinition.session.The older-host/browser runner now passes the same canonical template during draft creation, before the first request, together with target-owned isolation and branch. It no longer depends on optional post-create Mode/Approvals setters.
The older Copilot Chat sessions provider also implements Automation restore/capture so fallback configuration can round-trip instead of being flattened.
Compatibility and policy
Legacy Autopilot migration
Historical Copilot Automations migrate:
The migration also repairs the transitional invalid shape produced around the narrow #333949 fix:
This heuristic is restricted to the historical Copilot Automation boundary. Non-Copilot providers do not acquire Copilot configuration keys.
Preferences are not grants
The stored template is a preference snapshot.
Current provider schema, availability, and managed policy are reapplied when a run starts. If policy requires confirmations, effective approval is clamped at the runtime decision points without rewriting the stored preference.
This also covers policy changes during an active session and avoids elevated fallback if provider resolution fails.
Tool behavior
listAutomationsnow returns the complete JSON-safesessionTemplatefor canonical rows.configureAutomation:nullto reset it;Opaque provider configuration is not emitted through telemetry.
Save and cancellation behavior
Provider capture is asynchronous, so Save must not close the dialog until capture succeeds.
The lifecycle is kept local to the Automation dialog rather than extending the widely used base
Dialogwidget:aria-busy;Saving…and stays disabled;The local footer reuses standard button styles and platform-specific ordering. The final branch has no changes to the shared base
Dialogimplementation or tests relative to its merge base.User-visible behavior
Validation
Static and unit coverage
mainat2e12155dfa05: client type-check, targeted ESLint, source-layer validation, fresh client transpile, 468 provider/Automation tests passed with 13 platform-gated pending, and the strict Copilot restart replay passed.Agent Host E2E
mode: autopilotandautoApprove: assistedsurvive host restart and reach the created run session;Live Code OSS
Validated:
Reviewer attention / remaining merge gates
The branch has been rebased onto
mainat2e12155dfa05. The two provider conflicts were resolved by retaining upstream's configuration-operation serialization and repository-containment behavior while adapting the Automation restore/capture work to those APIs.unknown mode + assistedrepresentation. The repair is narrowly scoped to Copilot, where affected schemas do not admit legitimate unknown modes.chat.agentHost.codexAgent.enabledat its defaultfalse. A live pass should enable Codex and verify selection, Codex-native Approvals, save/reload, and Codex -> Copilot -> Codex retargeting.The branch is broad (52 files), so the commit stack is organized into reviewable layers: migration/policy, canonical domain/storage, provider drafts, New Session control reuse, execution parity, tools/E2E, and robustness fixes.
Known unrelated test/environment findings
The broad test pass also reproduced pre-existing or unrelated failures:
/usr/bin/gitblocked by the local Xcode license state.These were isolated from the Automation behavior and are not hidden by success-shaped fallbacks in this change.
UI
Final 420px Automation edit dialog using the provider-backed configuration controls and local asynchronous Save lifecycle. This isolated profile selected Copilot; live Codex coverage remains an explicit draft gate above.