Azure batch 15x migration

[baha-bouali]

closes: #66466
related: #67600

azure-batch 15.x is a ground-up rewrite of Azure's Python SDK (track 1 → track 2); BatchServiceClient and the old model classes are gone. #66452 capped azure-batch<15.0.0 as a stopgap; #66466 tracks the real migration to lift it.

The migration

This PR carries forward @arieleli01212's work from #67600 as-is: BatchServiceClient → BatchClient, shared-key auth moved to AzureNamedKeyCredential, the renamed model classes (PoolAddParameter → BatchPoolCreateOptions, CloudTask → BatchTask, etc.), the move to the new flat client API, dropping CloudServiceConfiguration/os_family/os_version, and the matching test rewrites.

What this PR adds on top

Two commits on top of his 16, kept separate:

1. Rebase onto current main. One real conflict, in docs/changelog.rst (an unrelated 14.0.0 entry had landed since). Resolved by keeping it and placing this migration's pending note above it.

2. Fix for the identity-auth path, addressing @aaron-y-chen's open review comment(#67600 (review)) on the # type: ignore[arg-type] next to the BatchClient credential. The identity-based branch of get_conn() built an AzureIdentityCredentialAdapter, which subclasses msrest's track-1 BasicTokenAuthentication and doesn't implement get_token() — what azure-core's TokenCredential protocol requires, and what BatchClient needs to authenticate. AzureBaseHook.get_token() in this provider already documents this and raises AttributeError for it. So this path passed every mocked test but would fail against a real account. Switched to get_sync_default_azure_credential(), already used by AzureBaseHook and asb.py for this exact purpose, which returns a real DefaultAzureCredential that does implement get_token(). The type: ignore is removed; updated the matching test; added a changelog line.

Testing

pytest providers/microsoft/azure/tests/unit/microsoft/azure/hooks/test_batch.py passes.

@eladkal asked whether this had been tested against a live account. Since the bug above only shows up once the SDK actually tries to authenticate, I validated it two ways:

1. Credential check (no Azure account needed):

hasattr(AzureIdentityCredentialAdapter(None, resource_id="https://batch.core.windows.net/.default"), "get_token")  # False
hasattr(get_sync_default_azure_credential(), "get_token")  # True

2. Live pool lifecycle against a real Azure Batch account (Azure for Students, Sweden Central), identity-based auth via Azure CLI login, no shared key:

from airflow.providers.microsoft.azure.hooks.batch import AzureBatchHook

hook = AzureBatchHook(azure_batch_conn_id="test_live_azure_batch")
client = hook.get_conn()

pool = hook.configure_pool(
    pool_id="claude-smoketest-pool",
    vm_publisher="canonical",
    vm_offer="0001-com-ubuntu-server-jammy",
    vm_sku="22_04-lts",
    vm_node_agent_sku_id="batch.node.ubuntu 22.04",
    vm_size="Standard_A1_v2",
    target_dedicated_nodes=0,
)
client.create_pool(pool)
pools = list(client.list_pools())
client.begin_delete_pool("claude-smoketest-pool").result()

Real HTTP round trips, no mocking:

DefaultAzureCredential acquired a token from AzureCliCredential
POST /pools → 201, pool created
GET /pools → 200, pool visible
DELETE /pools/claude-smoketest-pool → 202, then 404 on follow-up :  confirmed deleted

No AttributeError: the exact failure mode of the pre-fix code. Test resources were deleted after validation.

---

Was generative AI tooling used to co-author this PR?

• Yes (Claude Sonnet 5), reviewed and tested by the submitter.