refactor(kernel): reject azure-oauth on the kernel path; point to databricks-oauth

The kernel dropped its dedicated Azure U2M flow, so `auth_type="azure-oauth"`
on `use_kernel=True` is now rejected with a clear pointer to
`databricks-oauth` — whose in-house OAuth browser flow works against Azure
workspaces (the workspace federates login to Entra). This replaces the thin
forward-the-selector branch. `azure-sp-m2m` is unchanged; `azure-oauth` on the
Thrift path is unaffected. Rejecting (rather than silently remapping) keeps the
user's selected flow explicit.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>

Author: eric-wang-1990

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 # Unreleased
 - Kernel backend (`use_kernel=True`): OAuth U2M with `auth_type="databricks-oauth"` now forwards the connector's `databricks-sql-python` OAuth-app bundle (`client_id` + `sql offline_access` scopes + redirect port) into the kernel, so a bare U2M connection authenticates as `databricks-sql-python` — parity with the Thrift path — instead of inheriting the kernel's own `databricks-sql-connector` default. A caller-supplied `oauth_client_id` (with its coupled `oauth_redirect_port`) is honored, as is a caller-supplied `oauth_scopes`; absent one, the connector default (`sql offline_access`) is forwarded. Note: the kernel binds a single U2M redirect port, so unlike the Thrift path (which tries the full `8020..8024` range) the kernel path uses only one port and does not fall back to the next port if it is already bound — pass `oauth_redirect_port` (with `oauth_client_id`) to pick a free one on a port collision (PECOBLR-4040)
-- Kernel backend (`use_kernel=True`): **Azure Entra (Azure AD) OAuth is now supported.** The kernel is the Azure-aware auth core (it owns the endpoints, scopes, app ids, and tenant discovery); the connector forwards the selector and Azure credentials unchanged, so `connect()` is byte-identical between the Thrift and `use_kernel=True` paths. `auth_type="azure-oauth"` (Azure AD U2M) runs the kernel's browser flow against the workspace v2.0 authorize/token endpoints with the Azure app client id (`96eecda7-…`), redirect port `8030`, and the `{app_id}/user_impersonation offline_access` scope. `auth_type="azure-sp-m2m"` (Azure service principal) forwards `azure_client_id` / `azure_client_secret`; the kernel builds the Entra v2.0 token endpoint and the `{app_id}/.default` scope, and **auto-discovers the tenant** from the workspace's `/aad/auth` redirect when `azure_tenant_id` is omitted (matching Thrift). The service principal authenticates with the Databricks-audience data token, so it must be a workspace member; the kernel path does not yet support the Azure management-token flow (for an RBAC-only SP), so `azure_workspace_resource_id` is ignored with a warning on `use_kernel=True` (it still applies on the Thrift path) (PECOBLR-4141; PECOBLR-4120)
+- Kernel backend (`use_kernel=True`): **Azure Entra (Azure AD) service-principal M2M is now supported.** `auth_type="azure-sp-m2m"` forwards `azure_client_id` / `azure_client_secret`; the kernel is the Azure-aware auth core — it builds the Entra v2.0 token endpoint and the `{app_id}/.default` scope, and **auto-discovers the tenant** from the workspace's `/aad/auth` redirect when `azure_tenant_id` is omitted (matching Thrift). The service principal authenticates with the Databricks-audience data token, so it must be a workspace member; the kernel path does not yet support the Azure management-token flow (for an RBAC-only SP), so `azure_workspace_resource_id` is ignored with a warning on `use_kernel=True` (it still applies on the Thrift path). Azure AD **U2M** (`auth_type="azure-oauth"`) is **not** supported on the kernel path — use `auth_type="databricks-oauth"`, whose in-house OAuth browser flow works against Azure workspaces (the workspace federates login to Entra); `azure-oauth` on `use_kernel=True` is rejected with that pointer (PECOBLR-4141; PECOBLR-4120)
 
 # 4.4.0 (2026-07-22)
 - Raised the minimum supported Python version to 3.10, dropping the end-of-life 3.8/3.9, to update the lockfile and clear CVE-flagged dependencies in the repo (databricks/databricks-sql-python#798)

CONNECTION_PARAMETERS.md

@@ -69,7 +69,7 @@ to change without notice.
 | Option                                              | Type                 | Thrift | Kernel | Default Value             | Note                                                                                                                                                            |
 | --------------------------------------------------- | -------------------- | :----: | :----: | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `access_token` (PAT)                                | `str`                |   ✅   |   ✅   | `None`                    | Personal Access Token / bearer token. The default auth mode when set; otherwise auth falls back to OAuth.                                                       |
-| `auth_type`                                         | `str`                |   ✅   |   ✅   | `None` ⇒ Databricks OAuth | `databricks-oauth` (U2M), `azure-oauth` (Azure AD U2M), or `azure-sp-m2m` (Azure service-principal M2M). All three work on the kernel path; `azure-oauth` / `azure-sp-m2m` kernel support added in #919 (the connector forwards the selector + Azure credentials and the kernel owns Azure resolution). |
+| `auth_type`                                         | `str`                |   ✅   |   ✅   | `None` ⇒ Databricks OAuth | `databricks-oauth` (U2M), `azure-oauth` (Azure AD U2M), or `azure-sp-m2m` (Azure service-principal M2M). On the kernel path: `databricks-oauth` and `azure-sp-m2m` are supported (#919); `azure-oauth` is **not** — use `databricks-oauth` instead (its in-house browser flow works against Azure workspaces), and `azure-oauth` on `use_kernel=True` is rejected with that pointer. All three work on the Thrift path. |
 | `oauth_client_id` (U2M)                             | `str`                |   ✅   |   ✅   | built-in client id        | Custom U2M client id. Forwarded on both; when absent, each path applies its own built-in default.                                                              |
 | `oauth_redirect_port` (U2M)                         | `int`                |   ✅   |   ✅   | `None`                    | Localhost redirect port for the browser flow. On **both** backends it is only honored when a custom `oauth_client_id` is also supplied — then that single port becomes the redirect URI. With the built-in client id (or when omitted) the connector uses the full registered range 8020–8024 and binds the first free port, so a bare `oauth_redirect_port` has no effect. (Thrift: `auth.py` `oauth_redirect_port_range`; Kernel: same logic, forwarded as `redirect_ports`.) |
 | `oauth_client_secret` (OAuth M2M)                   | `str`                |   ❌   |   ✅   | `None`                    | **Kernel-only in practice.** The Thrift auth path never reads `oauth_client_secret`; use `credentials_provider` or an Azure service principal for M2M on Thrift. |

src/databricks/sql/backend/kernel/auth_bridge.py

@@ -21,16 +21,14 @@
   ``oauth_client_id`` / ``oauth_redirect_port`` overriding it) is
   forwarded to the kernel's ``auth_type='oauth-u2m'`` and the kernel
   runs the browser flow itself.
-- **Azure Entra (Azure AD)** — both Azure auth types forward the selector
-  and Azure credentials to the KERNEL, which is the Azure-aware auth core
-  (it owns the endpoints, scopes, app ids, and tenant discovery). The
-  binding stays thin — it does not construct endpoints or scopes:
-
-  - ``azure-oauth`` (U2M) → forward ``auth_type='azure-oauth'`` (plus any
-    optional ``oauth_client_id`` / ``oauth_redirect_port`` passthrough). The
-    kernel pins the workspace v2.0 authorize/token endpoints, the Azure app
-    client id (``96eecda7-…``), port ``8030``, and the
-    ``{app_id}/user_impersonation offline_access`` scope (PECOBLR-4120).
+- **Azure Entra (Azure AD)** — the KERNEL is the Azure-aware auth core (it
+  owns the endpoints, scopes, app ids, and tenant discovery); the binding
+  forwards the selector + credentials and does not construct endpoints:
+
+  - ``azure-oauth`` (U2M) → **not supported on the kernel path**; rejected with
+    a pointer to ``databricks-oauth``, whose in-house U2M browser flow works
+    against Azure workspaces (the workspace federates login to Entra). A
+    dedicated Azure U2M flow may return later (PECOBLR-4120).
   - ``azure-sp-m2m`` (M2M) → forward ``auth_type='azure-sp-m2m'`` with the
     Azure service-principal ``azure_client_id`` / ``azure_client_secret``.
     The kernel builds the Entra v2.0 token endpoint and the
@@ -170,9 +168,10 @@ def kernel_auth_kwargs(
        - a U2M ``auth_type`` (``databricks-oauth``) *and*
          ``oauth_client_secret`` together.
 
-    (The Azure Entra auth types — ``azure-oauth`` and ``azure-sp-m2m`` —
-    are forwarded to the kernel's Azure-aware flows up front, before these
-    guards; see the module docstring.)
+    (The Azure Entra auth types are handled up front, before these guards:
+    ``azure-sp-m2m`` forwards to the kernel's Azure SP flow; ``azure-oauth``
+    is rejected with a pointer to ``databricks-oauth``. See the module
+    docstring.)
     1. **OAuth M2M** — ``oauth_client_id`` + ``oauth_client_secret``
        both present → forward raw creds to the kernel's ``oauth-m2m``.
     2. **PAT** — the built provider is (or wraps) an
@@ -211,24 +210,20 @@ def kernel_auth_kwargs(
     # creds in azure_* kwargs, not oauth_client_id/secret, so it would
     # otherwise fall through to the final "unsupported" error).
 
-    # azure-oauth (Azure AD U2M): forward the selector; the KERNEL owns Azure
-    # resolution (it is the auth core). The kernel pins the workspace v2.0
-    # authorize/token endpoints (`{host}/oidc/oauth2/v2.0/{authorize,token}` —
-    # NOT the discovered `/oidc/v1/authorize`, which the workspace redirects to
-    # a malformed Entra URL), the Azure app client id, port 8030, and the
-    # `{app_id}/user_impersonation offline_access` scope. So this binding does
-    # NOT construct endpoints/scopes — it just passes `auth_type='azure-oauth'`
-    # plus any optional client_id / redirect_port passthrough. PECOBLR-4120.
+    # azure-oauth (Azure AD U2M) is NOT supported on the kernel path. The
+    # in-house `databricks-oauth` browser flow works against Azure workspaces
+    # (the workspace federates the login to Entra), so it is the U2M path on the
+    # kernel — reject `azure-oauth` with a clear pointer rather than silently
+    # changing the user's selected flow. (A dedicated Azure U2M flow may return
+    # later; for now Azure U2M = `databricks-oauth`.)
     if auth_type == "azure-oauth":
-        kwargs = {"auth_type": "azure-oauth"}
-        if client_id:
-            kwargs["client_id"] = client_id
-            redirect_port = opts.get("oauth_redirect_port")
-            if redirect_port is not None:
-                kwargs["redirect_ports"] = [_coerce_redirect_port(redirect_port)]
-        if federation_client_id:
-            kwargs["identity_federation_client_id"] = federation_client_id
-        return kwargs
+        raise NotSupportedError(
+            "auth_type='azure-oauth' is not supported on use_kernel=True. Use "
+            "auth_type='databricks-oauth' instead — the in-house OAuth U2M "
+            "browser flow works against Azure Databricks workspaces (the "
+            "workspace federates the login to Microsoft Entra). Or use the "
+            "Thrift backend (default) for the dedicated Azure AD U2M flow."
+        )
 
     # azure-sp-m2m (Azure service principal, client-credentials): forward the
     # selector + Azure SP credentials; the KERNEL owns Azure resolution (it is

tests/unit/test_kernel_auth_bridge.py

@@ -258,8 +258,10 @@ class TestKernelOAuthU2M:
     may override ``oauth_scopes``; absent one, ``PYSQL_OAUTH_SCOPES`` is
     forwarded as the default.
 
-    ``azure-oauth`` (Azure AD U2M) routes here too — see
-    ``test_azure_oauth_routes_to_kernel_u2m`` (PECOBLR-4120)."""
+    ``azure-oauth`` (Azure AD U2M) is NOT supported on the kernel path and is
+    rejected with a pointer to ``databricks-oauth`` — see
+    ``test_azure_oauth_rejected_with_pointer_to_databricks_oauth``
+    (PECOBLR-4120)."""
 
     def test_bare_databricks_oauth_forwards_full_python_bundle(self):
         # No overrides → forward the databricks-sql-python bundle in full
@@ -277,34 +279,16 @@ def test_bare_databricks_oauth_forwards_full_python_bundle(self):
             "oauth_scopes": list(PYSQL_OAUTH_SCOPES),
         }
 
-    def test_azure_oauth_forwards_selector_kernel_owns_resolution(self):
-        # azure-oauth (Azure AD U2M): the bridge forwards ONLY the selector.
-        # The kernel owns Azure resolution — it pins the workspace v2.0
-        # authorize/token endpoints, the Azure client id, port 8030, and the
-        # {app_id}/user_impersonation scope. So the bridge must NOT construct
-        # client_id / redirect_ports / oauth_scopes here. PECOBLR-4120.
-        kwargs = kernel_auth_kwargs(
-            _FakeOAuthProvider(),
-            {"auth_type": "azure-oauth"},
-        )
-        assert kwargs == {"auth_type": "azure-oauth"}
-
-    def test_azure_oauth_honors_custom_client_id_and_port(self):
-        # A caller override still passes through (client_id + its coupled port),
-        # but no scopes/endpoints are synthesised by the bridge.
-        kwargs = kernel_auth_kwargs(
-            _FakeOAuthProvider(),
-            {
-                "auth_type": "azure-oauth",
-                "oauth_client_id": "custom-azure-app",
-                "oauth_redirect_port": 9100,
-            },
-        )
-        assert kwargs == {
-            "auth_type": "azure-oauth",
-            "client_id": "custom-azure-app",
-            "redirect_ports": [9100],
-        }
+    def test_azure_oauth_rejected_with_pointer_to_databricks_oauth(self):
+        # azure-oauth (Azure AD U2M) is not supported on the kernel path. The
+        # in-house databricks-oauth browser flow works against Azure workspaces,
+        # so the bridge rejects azure-oauth with a clear pointer rather than
+        # silently changing the user's selected flow. PECOBLR-4120.
+        with pytest.raises(NotSupportedError, match="databricks-oauth"):
+            kernel_auth_kwargs(
+                _FakeOAuthProvider(),
+                {"auth_type": "azure-oauth"},
+            )
 
     def test_u2m_custom_client_id_port_and_scopes_honored(self):
         # A caller may override the coupled client_id + redirect port and the