Skip to content

Allow managed storage roots for DMS acceptance catalogs #7106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
shreyas-goenka wants to merge 1 commit into
base: main
Choose a base branch
from
+13 −4
Draft

Allow managed storage roots for DMS acceptance catalogs #7106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion acceptance/bundle/dms/declined-deploy/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ title "A destructive change without --auto-approve is declined: this console can
# Changing the catalog recreates the schema, which needs approval.
# Unity Catalog has no catalog called "other", so make one to move the schema into.
other_catalog="dms_other_${UNIQUE_NAME}"
$CLI catalogs create "$other_catalog" &> LOG.catalog
dms_create_catalog "$other_catalog" &> LOG.catalog
trap '$CLI catalogs delete "$other_catalog" --force >> LOG.catalog 2>&1; rm -f "$OUT_REQUESTS"' EXIT
trace update_file.py databricks.yml "catalog_name: main" "catalog_name: ${other_catalog}"
trace musterr $CLI bundle deploy
Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/dms/failed-recreate/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ title "A recreate deletes and then fails to create. The delete went through, so
# Move the schema to a different catalog to force a recreate (catalog_name is immutable), then
# fault the schema create so the delete goes through but the create that follows it fails.
other_catalog="dms_other_${UNIQUE_NAME}"
$CLI catalogs create "$other_catalog" &> LOG.catalog
dms_create_catalog "$other_catalog" &> LOG.catalog
trap '$CLI catalogs delete "$other_catalog" --force >> LOG.catalog 2>&1; rm -f "$OUT_REQUESTS"' EXIT
trace update_file.py databricks.yml "catalog_name: main" "catalog_name: ${other_catalog}"
trace fault.py "POST /api/2.1/unity-catalog/schemas" 400 0 1 INVALID_PARAMETER_VALUE
Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/dms/operation-upload-fails-once/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ title "A recreate records twice, and the first one fails: the second must still
trace fault.py "PATCH /api/2.0/bundle/*" 500 0 1
# Unity Catalog has no catalog called "other", so make one to move the schema into.
other_catalog="dms_other_${UNIQUE_NAME}"
$CLI catalogs create "$other_catalog" &> LOG.catalog
dms_create_catalog "$other_catalog" &> LOG.catalog
trap '$CLI catalogs delete "$other_catalog" --force >> LOG.catalog 2>&1' EXIT
trace update_file.py databricks.yml "catalog_name: main" "catalog_name: ${other_catalog}"
trace musterr $CLI bundle deploy --auto-approve
Expand Down
9 changes: 9 additions & 0 deletions acceptance/bundle/dms/script.prepare
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,12 @@ dms_plan() {
deployment_history_telemetry() {
jq -r 'select(.path? == "/telemetry-ext") | .body.protoLogs[] | fromjson | .entry.databricks_cli_log.bundle_deploy_event.experimental.bool_values[] | select(.key == "deployment_history_enabled") | "\(.key) \(.value)"' "$OUT_REQUESTS"
}

# Metastores without a storage root need an explicit managed location for each catalog.
dms_create_catalog() {
local storage_args=()
if [[ -n ${TEST_UC_MANAGED_STORAGE_ROOT:-} ]]; then
storage_args=(--storage-root "${TEST_UC_MANAGED_STORAGE_ROOT%/}/$1")
fi
$CLI catalogs create "$1" "${storage_args[@]}"
}
2 changes: 1 addition & 1 deletion acceptance/bundle/dms/successful-recreate/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ title "Recreate writes state twice - the entry is dropped, then the new resource
# The drop opens it IN_PROGRESS, the save patches it to SUCCEEDED.
# Unity Catalog has no catalog called "other", so make one to move the schema into.
other_catalog="dms_other_${UNIQUE_NAME}"
$CLI catalogs create "$other_catalog" &> LOG.catalog
dms_create_catalog "$other_catalog" &> LOG.catalog
trap '$CLI catalogs delete "$other_catalog" --force >> LOG.catalog 2>&1' EXIT
trace update_file.py databricks.yml "catalog_name: main" "catalog_name: ${other_catalog}"
trace $CLI bundle deploy --auto-approve
Expand Down
Loading