[aw]: Allow engine.model to be driven by an ADO pipeline variable / variable group #2035

Submission requirements

• I generated this issue with an agent that used .github/agents/ado-aw.agent.md.
• I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.

Problem summary

Description

Context

We run three compiled copilot-engine agents (implementer, pr-reviewer, and pr-comment-fixer) using the Azure AI Foundry BYOM provider and the standalone target. Their model deployment is an operational fleet-wide setting that belongs naturally in a shared Azure DevOps variable group.

Problem

In ado-aw 0.50.0, engine.model must be a compile-time literal. It cannot be driven by an Azure DevOps pipeline variable or variable group and therefore cannot be switched at queue time without editing each *.agent.md, recompiling, and committing each generated *.agent.lock.yml.

Source-verified behavior:

• engine.model is validated against a character allow-list ([A-Za-z0-9._:-]) in src/engine.rs, which rejects ADO macro syntax such as $(VAR) and ${{ }}.
• The COPILOT_MODEL environment variable and engine.args back doors are also blocked as substitution paths.
• Only engine.provider.base-url currently accepts a $(VAR) macro, and its host must then be added to network.allowed manually.

Real-world impact

Rolling our fleet from gpt-5.5 to gpt-5.6-terra required editing every .agent.md, recompiling, and committing every .agent.lock.yml. We would rather change one entry in a shared ADO variable group and re-run the pipelines. This would also make A/B testing, rollback, and cost-based retargeting operational changes rather than source-code changes.

Requested feature

Allow engine.model—and ideally the whole engine block—to be sourced from an Azure DevOps pipeline variable or variable group, with resolution at pipeline runtime.

Preferred resolutions, in order:

1. Permit ADO macro syntax ($(VAR) / variable-group reference) in engine.model, resolved at pipeline runtime rather than compile time. Validate literal values only and pass macros through untouched.
2. Add a dedicated explicitly-variable field, for example engine.model-variable: MY_MODEL_VAR, which compiles to --model $(MY_MODEL_VAR).
3. Re-enable COPILOT_MODEL, or provide an equivalent documented environment override, for the compiled default.

Example desired configuration:

engine:
  id: copilot
  model: $(COPILOT_MODEL_DEPLOYMENT)

Benefits

• One shared variable-group change can retarget all agents.
• Queue-time model selection needs no source edit, recompilation, or lock-file churn.
• Model rollout and rollback become faster and less error-prone.
• The compiled pipeline remains integrity-checkable and reproducible.
• The approach aligns model configuration with the existing runtime-variable support for engine.provider.base-url.

Related issues

• [aw]: Support Azure Copilot Foundry BYOM Configuration via engine.env #1261 added related Azure Copilot Foundry BYOM configuration support through engine.env, but does not make the selected model runtime-configurable.
• Allow engine.model to be driven by an ADO pipeline variable / variable group #2030 is a manually-authored equivalent of this request. It was closed as a duplicate in anticipation of this agent-filed issue and can remain closed in favor of this issue.

Labels

• feature
• copilot
• azure-integration
• engine-configuration

Reproduction details

Steps to reproduce:

1. Create an ado-aw 0.50.0 standalone agent using the copilot engine and Azure AI Foundry BYOM provider.
2. Set engine.model to an ADO variable-group macro such as $(COPILOT_MODEL_DEPLOYMENT).
3. Run ado-aw compile.

Expected behavior:

• Compilation succeeds.
• The generated pipeline passes the macro through to the model argument.
• Azure Pipelines resolves the model from the linked variable group at runtime.
• Changing the variable-group value and re-queuing switches models without source or lock-file changes.

Actual behavior:

• Compilation rejects the model because $, (, and ) are outside the engine.model allow-list.
• COPILOT_MODEL and engine.args cannot provide a supported override.
• Each model change requires editing, recompiling, and committing every agent pipeline.

Environment:

• ado-aw: 0.50.0
• Engine: copilot
• Provider: Azure AI Foundry BYOM
• Target: standalone
• Pipeline configuration: shared Azure DevOps variable group

Minimal reproduction:

---
name: Runtime model test
engine:
  id: copilot
  model: $(COPILOT_MODEL_DEPLOYMENT)
---

Run with the model deployment selected by the shared variable group.

Proposed next step

1. Decide whether to support direct ADO macros in engine.model or introduce an explicit model-variable field.
2. Preserve strict validation for literal model identifiers while safely passing recognized runtime references through unchanged.
3. Compile the selected form to the copilot CLI --model argument at pipeline runtime.
4. Add compiler tests for literal models, valid macro references, malformed expressions, and variable-group-backed execution.
5. Document queue-time model switching, variable-group usage, and interaction with provider/network configuration.
6. Consider extending the same explicit runtime-variable mechanism to other safe fields in the engine block.