Add AIR submission and YAML parity

[ben-hansen-db]

Changes

• Restore file-backed environment dependencies and submit resolved packages.
• Submit custom Docker image URLs and MLflow artifact locations.
• Add provisioned capacity support for native AIR workloads.

Why

Bring AIR YAML validation and native Jobs submission behavior in line with the Python CLI.

Tests

• go test ./experimental/air/cmd
• Focused run-submit-deps and config-help acceptance coverage on the aggregate branch

This PR was written with Codex.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 723e063

Run: 33127286864

	Env	💚​RECOVERED	🙈​SKIP	✅​pass	🙈​skip	Time
💚​	aws linux	1	1	274	1194	4:06
💚​	aws windows	1	1	276	1192	3:24
💚​	azure linux	1	1	273	1194	4:14
💚​	azure windows	1	1	275	1192	3:26
💚​	gcp linux	1	1	274	1194	4:45
💚​	gcp windows	1	1	276	1192	3:26

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚​	TestAccept	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R
🙈​	TestAccept/ssh/connection	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
3:22	azure windows	TestAccept
3:20	gcp windows	TestAccept
3:20	aws windows	TestAccept

[github-actions]

Approval status: pending

/acceptance/experimental/air/ - needs approval

6 files changed
Suggested: @riddhibhagwat-db
Also eligible: @vinchenzo-db, @apeforest, @bfontain, @lu-wang-dl, @panchalhp-db, @maggiewang-db, @pardis-beikzadeh-db

/experimental/air/ - needs approval

12 files changed
Suggested: @riddhibhagwat-db
Also eligible: @vinchenzo-db, @apeforest, @bfontain, @lu-wang-dl, @panchalhp-db, @maggiewang-db, @pardis-beikzadeh-db

General files (require maintainer)

Files: .nextchanges/cli/air-submission-parity.md
Based on git history:

• @pietern -- recent work in experimental/air/cmd/, .nextchanges/cli/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.}

[vinchenzo-db]

Why are we bringing back requirements.yaml? I thought we wanted to deprecate that in favor of inline listing the dependencies like: dependencies: [numpy, torch==2.3.0]