Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions knowledge_base/bind_existing_resources/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.databricks
38 changes: 38 additions & 0 deletions knowledge_base/bind_existing_resources/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Bind existing resources

This Declarative Automation Bundles example puts an existing job and dashboard under bundle management.
It generates the configuration, binds it to the workspace resources, deploys it, and then unbinds it.

For more information, see the [bind documentation](https://docs.databricks.com/dev-tools/bundles/migrate-resources.html).

## Prerequisites

* Databricks CLI v1.20.0 or above
* A workspace with serverless jobs and a SQL warehouse named `Serverless Starter Warehouse`.
* An existing job and an existing AI/BI dashboard. Copy their IDs from the workspace URLs.

## Usage

1. Set the folder of the existing dashboard: `export BUNDLE_VAR_dashboard_parent_path=/Workspace/Users/<user>/<folder>`
2. Generate the job configuration. This replaces the sample files in `resources/` and `src/`:
`databricks bundle generate job --existing-job-id <job-id> --key existing_job --force --profile <profile>`
3. Generate the dashboard configuration:
`databricks bundle generate dashboard --existing-id <dashboard-id> --key existing_dashboard --force --profile <profile>`
4. In `resources/existing_dashboard.dashboard.yml`, replace the literal `warehouse_id` with `${var.warehouse_id}`.
5. In the same file, set `parent_path: ${var.dashboard_parent_path}`.
6. Bind the job: `databricks bundle deployment bind existing_job <job-id> -t prod --profile <profile>`
7. Bind the dashboard: `databricks bundle deployment bind existing_dashboard <dashboard-id> -t prod --profile <profile>`
8. Show the changes that the deploy makes: `databricks bundle plan -t prod --profile <profile>`
9. Deploy the bundle: `databricks bundle deploy -t prod --profile <profile>`
10. Run the job: `databricks bundle run existing_job -t prod --profile <profile>`
11. Unbind the job: `databricks bundle deployment unbind existing_job -t prod --profile <profile>`
12. Unbind the dashboard: `databricks bundle deployment unbind existing_dashboard -t prod --profile <profile>`
13. Destroy the bundle deployment: `databricks bundle destroy -t prod --profile <profile>`

| Step | Effect on the workspace resource |
| --- | --- |
| `bind` | No change. The bundle state records the resource ID. |
| `plan` | Shows drift. Changes made in the UI appear as `update`. |
| `deploy` | Overwrites the resource with the bundle configuration. The ID stays the same. |
| `unbind` | No change. The bundle state forgets the resource ID. |
| `destroy` | Deletes 0 resources and the `root_path` folder. The unbound job and dashboard stay. |
31 changes: 31 additions & 0 deletions knowledge_base/bind_existing_resources/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This is a Declarative Automation Bundle definition for bind_existing_resources.
# See https://docs.databricks.com/dev-tools/bundles/migrate-resources.html for documentation.
bundle:
name: bind_existing_resources

include:
- resources/*.yml

variables:
warehouse_id:
description: SQL warehouse that runs the dashboard queries
lookup:
warehouse: Serverless Starter Warehouse
dashboard_parent_path:
description: Workspace folder of the existing dashboard, for example /Workspace/Users/<user>/<folder>

# The workspace host is not set here. Use a profile (--profile / -p) or the
# DATABRICKS_HOST environment variable.

targets:
dev:
# Development mode adds a "[dev <user>]" prefix to the bound job name and pauses its schedules.
mode: development
default: true
workspace:
root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}/${workspace.current_user.short_name}

prod:
mode: production
workspace:
root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
resources:
dashboards:
existing_dashboard:
display_name: "Existing dashboard"
# Generated as a literal warehouse ID. Replaced with a variable so the bundle works in any workspace.
warehouse_id: ${var.warehouse_id}
# The parent path cannot change. Keep the folder of the existing dashboard, or deploy recreates it with a new ID.
parent_path: ${var.dashboard_parent_path}
file_path: ../src/existing_dashboard.lvdash.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
resources:
jobs:
existing_job:
name: Existing job
tasks:
- task_key: hello
disabled: false
email_notifications: {}
environment_key: default
notebook_task:
notebook_path: ../src/hello.py
source: WORKSPACE
run_if: ALL_SUCCESS
timeout_seconds: 0
email_notifications: {}
environments:
- environment_key: default
spec:
environment_version: "6"
max_concurrent_runs: 1
queue:
enabled: true
timeout_seconds: 0
webhook_notifications: {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"datasets": [
{
"displayName": "trips",
"name": "d1",
"queryLines": [
"SELECT * FROM samples.nyctaxi.trips LIMIT 10"
]
}
],
"pages": [
{
"displayName": "Trips",
"name": "p1",
"pageType": "PAGE_TYPE_CANVAS"
}
]
}
2 changes: 2 additions & 0 deletions knowledge_base/bind_existing_resources/src/hello.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Databricks notebook source
print("Hello from an existing job!")