diff --git a/knowledge_base/run_as_service_principal/README.md b/knowledge_base/run_as_service_principal/README.md new file mode 100644 index 0000000..0f03681 --- /dev/null +++ b/knowledge_base/run_as_service_principal/README.md @@ -0,0 +1,46 @@ +# Run a job as a service principal + +This Declarative Automation Bundles example runs a prod job as a service principal. +In dev, the job runs as the user who deploys it. +The serverless `whoami` job prints its identity, so you can see the effect of `run_as`. + +For more information, see the [run_as documentation](https://docs.databricks.com/dev-tools/bundles/run-as.html). + +## Prerequisites + +* Databricks CLI v1.20.0 or above +* A workspace with Unity Catalog and serverless jobs. +* A service principal in the workspace. Copy its application ID. +* A deployer group (default `bundle-deployers`) that contains the deployers and the service principal. +* Deployers need the `Service Principal: User` role on the service principal. + +## Usage + +1. Deploy the bundle to dev: `databricks bundle deploy -t dev --profile ` +2. Run the job in dev: `databricks bundle run -t dev whoami --profile ` +3. In CI, authenticate as the service principal with OAuth M2M and set the application ID: + + ```sh + export DATABRICKS_HOST=https:// + export DATABRICKS_CLIENT_ID= + export DATABRICKS_CLIENT_SECRET= + export BUNDLE_VAR_service_principal_application_id= + ``` + +4. In CI, deploy the bundle to prod: `databricks bundle deploy -t prod` +5. In CI, run the job in prod: `databricks bundle run -t prod whoami` +6. Read the job output. In dev, it shows your user name. In prod, it shows the service principal: + + ``` + current_user(): + SDK user_name: + SDK display_name: + ``` + +7. Remove the dev resources: `databricks bundle destroy -t dev --profile ` +8. In CI, remove the prod resources: `databricks bundle destroy -t prod` + +| Target | Runs as | +| --- | --- | +| `dev` | The user who deploys | +| `prod` | The service principal | diff --git a/knowledge_base/run_as_service_principal/databricks.yml b/knowledge_base/run_as_service_principal/databricks.yml new file mode 100644 index 0000000..6865a2c --- /dev/null +++ b/knowledge_base/run_as_service_principal/databricks.yml @@ -0,0 +1,53 @@ +# This is a Declarative Automation Bundle definition for run_as_service_principal. +# See https://docs.databricks.com/dev-tools/bundles/run-as.html for documentation. +bundle: + name: run_as_service_principal + +include: + - resources/*.yml + +variables: + service_principal_application_id: + description: >- + Application ID (a UUID) of the service principal that runs the prod job. + Find it in the workspace admin settings under Identity and access > Service principals. + deployer_group: + description: Workspace group whose members (and CI) can deploy and manage the prod bundle + default: bundle-deployers + viewer_group: + description: Workspace group that can view the prod job and its runs + default: users + +# The workspace host is not set here. Use a profile (--profile / -p) or the +# DATABRICKS_HOST environment variable, so the same file works locally and in CI. + +targets: + dev: + # Development mode prefixes resource names with "[dev ]", pauses + # schedules, and runs everything as the deploying user. + # Do not set run_as here: dev deployments are per-user by design. + mode: development + default: true + variables: + # Dev does not use run_as, so this value is never read. The variable has no + # default on purpose: a prod deploy fails fast if you forget to set it. + service_principal_application_id: not-used-in-dev + workspace: + # Each developer gets a separate dev copy under the shared example folder. + root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}/${workspace.current_user.short_name} + + prod: + mode: production + workspace: + # One stable prod copy. Deploy it from one identity (CI as the service + # principal): a different identity that is not a workspace admin cannot + # take over ownership of the job. + root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target} + run_as: + service_principal_name: ${var.service_principal_application_id} + permissions: + # Grant access to groups only. No individual user gets CAN_MANAGE. + - group_name: ${var.deployer_group} + level: CAN_MANAGE + - group_name: ${var.viewer_group} + level: CAN_VIEW diff --git a/knowledge_base/run_as_service_principal/resources/whoami.job.yml b/knowledge_base/run_as_service_principal/resources/whoami.job.yml new file mode 100644 index 0000000..b1af1de --- /dev/null +++ b/knowledge_base/run_as_service_principal/resources/whoami.job.yml @@ -0,0 +1,19 @@ +resources: + jobs: + whoami: + name: run_as_service_principal_whoami + description: Prints the identity that runs the job, so you can see the effect of run_as. + + # No trigger or schedule: run the job on demand with `databricks bundle run whoami`. + + tasks: + - task_key: whoami + environment_key: default + spark_python_task: + python_file: ../src/whoami.py + + environments: + - environment_key: default + # See https://docs.databricks.com/api/workspace/jobs/create#environments-spec + spec: + environment_version: "6" diff --git a/knowledge_base/run_as_service_principal/src/whoami.py b/knowledge_base/run_as_service_principal/src/whoami.py new file mode 100644 index 0000000..0eac950 --- /dev/null +++ b/knowledge_base/run_as_service_principal/src/whoami.py @@ -0,0 +1,13 @@ +# Prints the identity that runs this task. +# In dev this is the user who deployed the bundle; in prod it is the run_as service principal. +from databricks.sdk import WorkspaceClient +from pyspark.sql import SparkSession + +spark = SparkSession.builder.getOrCreate() +sql_user = spark.sql("SELECT current_user() AS user").first()["user"] + +me = WorkspaceClient().current_user.me() + +print(f"current_user(): {sql_user}") +print(f"SDK user_name: {me.user_name}") +print(f"SDK display_name: {me.display_name}")