Skip to content

Add --state-path flag to bundle config-remote-sync - #6367

Open
ilyakuz-db wants to merge 3 commits into
databricks:mainfrom
ilyakuz-db:config-remote-sync-state-path
Open

Add --state-path flag to bundle config-remote-sync#6367
ilyakuz-db wants to merge 3 commits into
databricks:mainfrom
ilyakuz-db:config-remote-sync-state-path

Conversation

@ilyakuz-db

@ilyakuz-db ilyakuz-db commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Changes

Add a --state-path flag to bundle config-remote-sync that overrides workspace.state_path, so the command reads the deployment state from an explicit workspace location instead of the one it derives from the bundle config.

The value must be an absolute workspace path; relative, ~-prefixed, and /Volumes/ values are rejected up front. A value without a /Workspace prefix is normalized the same way a configured state_path is.

Scoped to this command: no change to bundle deploy or to any shared bundle code.

Why

When workspace.root_path is not set explicitly, it defaults to ~/.bundle/<name>/<target>, and ~ expands to the home directory of whoever runs the command. That makes the resolved state location a property of the caller rather than of the deployment: a user who did not deploy the bundle resolves an empty state folder under their own home, so --select-ids finds no deployed resource and the sync fails.

The state folder is the single lever for this — resources.json, terraform.tfstate, the config snapshot and metadata.json are all siblings under workspace.state_path and all read through the same filer — so one flag pins every read to the folder the deployment actually wrote.

Two details worth calling out, both found in review:

  • The override is applied in PostInitFunc, after phases.Initialize. Applying it earlier means ValidateTargetMode sees it, and for mode: development findNonUserPath rejects any state_path that does not contain the current user's name — which is exactly the cross-user case this flag exists to enable. This command only ever reads that folder, so dev-mode uniqueness of it is not meaningful here. It is assigned through a mutator so the value also reaches the dynamic config tree; later phases convert dyn to typed on entry and would otherwise restore the default before the state snapshot is read.
  • State has to be on disk to be opened, and the default location for that is keyed on bundle root and target alone — so a redirected read would otherwise leave another deployment's state in this bundle's own cache, where later commands, including deploy, would pick it up as their own. With the flag set, the state is materialized into a per-invocation scratch directory that is removed on exit. Nothing needs it to persist: AlwaysPull re-reads the remote state every run and the state is opened read-only.
  • deployment.metadata_file_path is derived from state_path by the annotators at the end of phases.Initialize, which runs before the override. It is recomputed afterwards, otherwise the diff reports the default location as a change on every job and pipeline — and --save would write that into the user's YAML.

Tests

New hermetic acceptance test bundle/config-remote-sync/state_path, covering both engines. It deploys two bundles so the cross-deployment cases are real, and deliberately never flushes .databricks, so the override has to win over a populated cache rather than being handed an empty one:

  • --state-path at the deployment's own state folder detects remote drift; the same folder without a /Workspace prefix resolves identically.
  • Reading another deployment's state reports that deployment's resource ids, and afterwards the bundle's own unflagged sync still reads its own state.
  • Reading a second deployment that declares the same resource identically reports no changes, pinning that nothing is derived from this bundle's own defaults.
  • A dev-mode target is not blocked from a state folder under another user's home, or a shared one.
  • Relative, ~-prefixed and /Volumes/ values are rejected before any state is read.

Verified the test fails without each fix: sharing the local cache reproduces lineage mismatch in state files and then a silent fallback to it, applying the override before phases.Initialize reproduces the dev-mode rejection, and skipping the annotator replay reproduces metadata_file_path: add.

This PR was written by Claude Code.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

8 files changed
Suggested: @denik
Also eligible: @andrewnester, @anton-107, @janniklasrose, @lennartkats-db, @pietern, @shreyas-goenka

/cmd/bundle/ - needs approval

Files: cmd/bundle/config_remote_sync.go
Suggested: @denik
Also eligible: @andrewnester, @anton-107, @janniklasrose, @lennartkats-db, @pietern, @shreyas-goenka

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.

ilyakuz-db and others added 2 commits August 24, 2026 16:17
…cope its cache

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 6367
  • Commit SHA: 3cfddf89cff884b30e304c4f6578b7330839d94c

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant