Skip to content

config: ForceSendFields on CLI config structs - #7117

Merged
denik merged 9 commits into
mainfrom
denik/force-send-fields-config-structs
Oct 20, 2026
Merged

denik merged 9 commits into
mainfrom
denik/force-send-fields-config-structs

Conversation

@denik

@denik denik commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Add ForceSendFields to Presets, Lifecycle and Deployment, so explicit zero values (false, 0, "") written in a target or include replace non-zero base values.

Fixes on main: targets.dev.resources.jobs.j.lifecycle.prevent_destroy: false was ignored and the base true stayed.

Visible change: bundle validate -o json now shows explicit zero presets (e.g. name_prefix: "") instead of presets: {}.

Not added to Target, Workspace, Bundle, Experimental, Lock, Artifact or the other structs: they have pointer/map/slice fields, and ForceSendFields there turns explicit null overrides (e.g. run_as:) into forced values.

Prerequisite for removing libs/dyn (#7098).

Tests

This pull request and its description were written by Isaac.

denik and others added 2 commits October 20, 2026 11:01
…rides

Add ForceSendFields []string fields to key CLI-owned config structs so that explicitly set zero values can override non-zero base values.

Affected structs: Presets, Lifecycle, LifecycleWithStarted, JobRunLifecycle, Experimental, Bundle, Deployment, Lock, Workspace, Artifact, AppEnvVar, Target, Script, Python.

The ForceSendFields field is marked json:"-" to exclude it from JSON marshaling. The libs/dyn/convert layer already has support for populating and using this field.

Co-authored-by: Isaac <no-reply@databricks.com>
Test that explicit zero values in target overrides win over non-zero base values:
- target_lifecycle_override: prevent_destroy false overrides true
- target_experimental_override: experimental flags false override true

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the DABs DABs related issues label Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 5322045d6 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0

test branch main (8743718) latest (v1.21.0)
TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS= ✅ ❌ ❌
TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=true ✅ ❌ ❌
TestAccept/bundle/validate/lifecycle_override/DMS= ✅ ❌ ❌
TestAccept/bundle/validate/lifecycle_override/DMS=true ✅ ❌ ❌
TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=: branch ✅ | main (8743718) ❌ | latest (v1.21.0) ❌

main (8743718):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {\n    \"artifacts_dynamic_version\": false,\n    \"jobs_max_concurrent_runs\": 0,\n    \"name_prefix\": \"\"\n  },\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {},\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,7 +3,3 @@
        	            	 {
        	            	-  "presets": {
        	            	-    "artifacts_dynamic_version": false,
        	            	-    "jobs_max_concurrent_runs": 0,
        	            	-    "name_prefix": ""
        	            	-  },
        	            	+  "presets": {},
        	            	   "job": {
        	Test:       	TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=
        	Messages:   	bundle/validate/includes/clears_unforced_preset_values/output.txt vs /tmp/TestAcceptbundlevalidateincludesclears_unforced_preset_value1924664639/002/output.txt

latest (v1.21.0):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {\n    \"artifacts_dynamic_version\": false,\n    \"jobs_max_concurrent_runs\": 0,\n    \"name_prefix\": \"\"\n  },\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {},\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,7 +3,3 @@
        	            	 {
        	            	-  "presets": {
        	            	-    "artifacts_dynamic_version": false,
        	            	-    "jobs_max_concurrent_runs": 0,
        	            	-    "name_prefix": ""
        	            	-  },
        	            	+  "presets": {},
        	            	   "job": {
        	Test:       	TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=
        	Messages:   	bundle/validate/includes/clears_unforced_preset_values/output.txt vs /tmp/TestAcceptbundlevalidateincludesclears_unforced_preset_value3876985452/002/output.txt
TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=true: branch ✅ | main (8743718) ❌ | latest (v1.21.0) ❌

main (8743718):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {\n    \"artifacts_dynamic_version\": false,\n    \"jobs_max_concurrent_runs\": 0,\n    \"name_prefix\": \"\"\n  },\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {},\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,7 +3,3 @@
        	            	 {
        	            	-  "presets": {
        	            	-    "artifacts_dynamic_version": false,
        	            	-    "jobs_max_concurrent_runs": 0,
        	            	-    "name_prefix": ""
        	            	-  },
        	            	+  "presets": {},
        	            	   "job": {
        	Test:       	TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=true
        	Messages:   	bundle/validate/includes/clears_unforced_preset_values/output.txt vs /tmp/TestAcceptbundlevalidateincludesclears_unforced_preset_value2257172509/002/output.txt

latest (v1.21.0):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {\n    \"artifacts_dynamic_version\": false,\n    \"jobs_max_concurrent_runs\": 0,\n    \"name_prefix\": \"\"\n  },\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json\n{\n  \"presets\": {},\n  \"job\": {\n    \"name\": \"job\",\n    \"max_concurrent_runs\": 1\n  }\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,7 +3,3 @@
        	            	 {
        	            	-  "presets": {
        	            	-    "artifacts_dynamic_version": false,
        	            	-    "jobs_max_concurrent_runs": 0,
        	            	-    "name_prefix": ""
        	            	-  },
        	            	+  "presets": {},
        	            	   "job": {
        	Test:       	TestAccept/bundle/validate/includes/clears_unforced_preset_values/DMS=true
        	Messages:   	bundle/validate/includes/clears_unforced_preset_values/output.txt vs /tmp/TestAcceptbundlevalidateincludesclears_unforced_preset_value3959135939/002/output.txt
TestAccept/bundle/validate/lifecycle_override/DMS=: branch ✅ | main (8743718) ❌ | latest (v1.21.0) ❌

main (8743718):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": false,\n  \"cluster_prevent_destroy\": false\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": true,\n  \"cluster_prevent_destroy\": true\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,4 +3,4 @@
        	            	 {
        	            	-  "job_prevent_destroy": false,
        	            	-  "cluster_prevent_destroy": false
        	            	+  "job_prevent_destroy": true,
        	            	+  "cluster_prevent_destroy": true
        	            	 }
        	Test:       	TestAccept/bundle/validate/lifecycle_override/DMS=
        	Messages:   	bundle/validate/lifecycle_override/output.txt vs /tmp/TestAcceptbundlevalidatelifecycle_overrideDMS=4044609458/002/output.txt

latest (v1.21.0):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": false,\n  \"cluster_prevent_destroy\": false\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": true,\n  \"cluster_prevent_destroy\": true\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,4 +3,4 @@
        	            	 {
        	            	-  "job_prevent_destroy": false,
        	            	-  "cluster_prevent_destroy": false
        	            	+  "job_prevent_destroy": true,
        	            	+  "cluster_prevent_destroy": true
        	            	 }
        	Test:       	TestAccept/bundle/validate/lifecycle_override/DMS=
        	Messages:   	bundle/validate/lifecycle_override/output.txt vs /tmp/TestAcceptbundlevalidatelifecycle_overrideDMS=2805509378/002/output.txt
TestAccept/bundle/validate/lifecycle_override/DMS=true: branch ✅ | main (8743718) ❌ | latest (v1.21.0) ❌

main (8743718):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": false,\n  \"cluster_prevent_destroy\": false\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": true,\n  \"cluster_prevent_destroy\": true\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,4 +3,4 @@
        	            	 {
        	            	-  "job_prevent_destroy": false,
        	            	-  "cluster_prevent_destroy": false
        	            	+  "job_prevent_destroy": true,
        	            	+  "cluster_prevent_destroy": true
        	            	 }
        	Test:       	TestAccept/bundle/validate/lifecycle_override/DMS=true
        	Messages:   	bundle/validate/lifecycle_override/output.txt vs /tmp/TestAcceptbundlevalidatelifecycle_overrideDMS=true3156401632/002/output.txt
FAIL

latest (v1.21.0):

acceptance_test.go:1308: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1308
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1078
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:656
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": false,\n  \"cluster_prevent_destroy\": false\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -t dev -o json\n{\n  \"job_prevent_destroy\": true,\n  \"cluster_prevent_destroy\": true\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,4 +3,4 @@
        	            	 {
        	            	-  "job_prevent_destroy": false,
        	            	-  "cluster_prevent_destroy": false
        	            	+  "job_prevent_destroy": true,
        	            	+  "cluster_prevent_destroy": true
        	            	 }
        	Test:       	TestAccept/bundle/validate/lifecycle_override/DMS=true
        	Messages:   	bundle/validate/lifecycle_override/output.txt vs /tmp/TestAcceptbundlevalidatelifecycle_overrideDMS=true3982315901/002/output.txt
FAIL

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 5322045

Run: 39404423509

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 280 17 4:13
✅​ aws-windows-latest-4core-16gb 282 15 3:10
✅​ azure linux-2core-8gb 279 17 4:01
✅​ azure-windows-latest-4core-16gb 281 15 4:54
✅​ gcp linux-2core-8gb 280 17 3:59
🔄​ gcp-windows-latest-4core-16gb 1 281 15 3:29
Test Name gcp-windows-latest-4core-16gb
🔄​ TestSyncIncrementalFileSync 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:47 azure-windows-latest-4core-16gb TestAccept
3:07 gcp-windows-latest-4core-16gb TestAccept
3:04 aws-windows-latest-4core-16gb TestAccept

denik and others added 6 commits October 20, 2026 11:28
Remove FSF from structs with pointer/map/slice/struct fields to avoid breaking
null overrides:
- Target (has pointers like RunAs, Workspace, etc.)
- Script, Python (speculative, unused)
- LifecycleWithStarted, JobRunLifecycle (have pointer/slice fields)
- Workspace (has CurrentUser *User)
- Experimental (has map, struct, pointer fields)
- Bundle (has Terraform pointer, Git struct)
- Lock (has Enabled *bool)
- Artifact (has slice, pointer fields)

Remove broken acceptance tests:
- target_experimental_override (Target has no experimental field)
- target_lifecycle_override (moved to cleaner single test)

Keep only on truly safe structs: Presets, Lifecycle, Deployment, AppEnvVar

Co-authored-by: Isaac <no-reply@databricks.com>
With ForceSendFields on Presets, explicit zero values (name_prefix: "",
jobs_max_concurrent_runs: 0, artifacts_dynamic_version: false) are now preserved
in bundle validate -o json output instead of being collapsed to an empty presets object.

This is a more faithful representation of the bundle configuration and enables
zero-value overrides to work in includes.

Co-authored-by: Isaac <no-reply@databricks.com>
When creating AppEnvVar literals, the ForceSendFields field must be initialized
even though it's empty. This fixes the exhaustruct_v5 lint error.

Co-authored-by: Isaac <no-reply@databricks.com>
Test verifies that prevent_destroy: false in an include override wins over
prevent_destroy: true in root - a key zero-value override scenario.

Changelog fragment documents the user-visible change: bundle validate now
preserves explicit zero values in presets output.

Test FAILS on origin/main (shows null for overrides) and PASSES on this branch
(shows false), proving the fix works.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…esources

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik marked this pull request as ready for review October 20, 2026 09:55
@denik
denik requested review from a team as code owners October 20, 2026 09:55
…catenation

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik enabled auto-merge October 20, 2026 10:12
@denik
denik added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit a1cf984 Oct 20, 2026
27 checks passed
@denik
denik deleted the denik/force-send-fields-config-structs branch October 20, 2026 10:48
denik added a commit that referenced this pull request Oct 20, 2026
#7117 added ForceSendFields to Presets, so a written zero is recorded.

Co-authored-by: Isaac <no-reply@databricks.com>
denik added a commit that referenced this pull request Oct 20, 2026
#7117 added ForceSendFields to Presets, so a written zero is recorded.

Co-authored-by: Isaac <no-reply@databricks.com>
denik added a commit that referenced this pull request Oct 21, 2026
#7117 added ForceSendFields to Presets, so a written zero is recorded.

Co-authored-by: Isaac <no-reply@databricks.com>
denik added a commit that referenced this pull request Oct 21, 2026
#7117 added ForceSendFields to Presets, so a written zero is recorded.

Co-authored-by: Isaac <no-reply@databricks.com>
denik added a commit that referenced this pull request Oct 21, 2026
#7117 added ForceSendFields to Presets, so a written zero is recorded.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants