Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions knowledge_base/job_git_source/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.databricks
54 changes: 54 additions & 0 deletions knowledge_base/job_git_source/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Job with Git source

This Declarative Automation Bundles example runs a notebook directly from a public Git repository.
See the [Git source documentation](https://docs.databricks.com/aws/en/jobs/how-to/use-git-with-jobs) for supported providers and tasks.

| Source | Behavior |
| --- | --- |
| Git source | The job checks out the configured repository and branch at run time. |
| Synced source | The bundle uploads local files to the workspace during deployment. |

## Prerequisites

* Install Databricks CLI v1.20.0 or above.
* Configure Databricks CLI authentication.
* Use a workspace that supports serverless jobs.

## Usage

1. Validate the development target.

```sh
databricks bundle validate -t dev
```

2. Deploy and run the job.

```sh
databricks bundle deploy -t dev
databricks bundle run -t dev git_source
```

3. Override a Git setting when needed.

```sh
databricks bundle deploy -t dev --var git_branch=main
```

4. Remove the development deployment.

```sh
databricks bundle destroy -t dev --auto-approve
```

5. Deploy production from the `main` branch.

```sh
databricks bundle deploy -t prod
```

6. Remove the production deployment.

```sh
databricks bundle destroy -t prod --auto-approve
```
31 changes: 31 additions & 0 deletions knowledge_base/job_git_source/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
bundle:
name: job_git_source
databricks_cli_version: ">= 1.20.0"

include:
- resources/*.yml

variables:
git_url:
description: Git repository URL for the job source.
default: https://github.com/databricks/bundle-examples
git_provider:
description: Git provider for the job source.
default: gitHub
git_branch:
description: Git branch for the job source.
default: main

targets:
dev:
default: true
mode: development
workspace:
root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}/${workspace.current_user.short_name}
prod:
mode: production
bundle:
git:
branch: main
workspace:
root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}
18 changes: 18 additions & 0 deletions knowledge_base/job_git_source/resources/git_source.job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
resources:
jobs:
git_source:
name: Job with Git source
git_source:
git_url: ${var.git_url}
git_provider: ${var.git_provider}
git_branch: ${var.git_branch}
tasks:
- task_key: hello
environment_key: default
notebook_task:
notebook_path: knowledge_base/development_cluster/src/hello.py
source: GIT
environments:
- environment_key: default
spec:
environment_version: "6"
1 change: 1 addition & 0 deletions knowledge_base/job_git_source/src/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@