validate: check required and enum fields with generated typed walkers by denik · Pull Request #7143 · databricks/cli

denik · 2026-10-21T11:20:32Z

Stack, 3 of 4. Based on #7145 (after #7130); next: #7098, which replaces the dyn bridge in this PR.

Changes

validate:required and validate:enum now run generated typed walkers (bundle/config/validate/{required,enum}_walk.go) instead of matching every node of the dyn tree against a trie built from the generated pattern tables. The walkers only go down the pattern paths of the typed config.

The typed config can't tell unset from null, zero or a pure ${...} reference, and it has no locations. For those, each walker node also follows the dyn.Value at its path. This bridge lives in one file, walk_node.go, and gets replaced once config no longer uses dyn.

This is a slice of #7098 (#7098). libs/dyn stays in place.

Behavior

No change: same diagnostics, wording, paths, locations and ordering. Enum diagnostics are first put in dyn-walk order so that ties in the existing sort come out as before.

Tests

New unit tests in walk_test.go.
All TestAccept/bundle acceptance tests pass with unchanged goldens. During that run a temporary check (not committed) compared every result with the old trie implementation and found no mismatch.
The same check passed on 16k randomly mutated acceptance configs (null/""/0/false/{}/[]/refs).

Benchmarks

1000 jobs (bundletest.Bundle), median of 6 runs, macOS M-series, load ~5:

	main	this PR
`Required().Apply`	41.0 ms	20.7 ms
`Enum().Apply`	23.0 ms	4.0 ms
`BenchmarkValidateRequired1000` (via `bundle.Apply`)	133 ms	113 ms
`BenchmarkValidateEnum1000` (via `bundle.Apply`)	114 ms	95 ms
`bundle validate` end to end, `acceptance/bundle/benchmarks` `--jobs 1000`	11.12 s	11.04 / 11.18 s

The required check is now dominated by the unchanged sortDiagnostics (it calls fmt.Sprintf in the comparator). End-to-end bundle validate shows no measurable difference: these two mutators are under 1% of it.

This pull request and its description were written by Isaac.

eng-dev-ecosystem-bot · 2026-10-21T11:37:10Z

Integration test report

Commit: 2f2e8ac

Run: 39633019933

	Env	✅pass	🙈skip	Time
✅	aws linux-2core-8gb	281	20	4:51
✅	aws-windows-latest-4core-16gb	283	18	3:22
✅	azure linux-2core-8gb	280	20	4:08
✅	azure-windows-latest-4core-16gb	282	18	3:38
✅	gcp linux-2core-8gb	281	20	4:29
✅	gcp-windows-latest-4core-16gb	283	18	3:20

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
3:31	azure-windows-latest-4core-16gb	TestAccept
3:14	aws-windows-latest-4core-16gb	TestAccept
3:14	gcp-windows-latest-4core-16gb	TestAccept

github-actions · 2026-10-21T15:58:35Z

Commit: 2f2e8ac06 | Workflow run

Regression test report

Selected 5 changed tests (limit=20, 0 not selected)

10 tests: regression=2 unreleased=0 coverage=8 failing=0

test	branch	main (`bef6321`)	latest (v1.21.0)
TestAccept/bundle/override/cluster_id_clears_new_cluster_reference/DMS=	✅	✅	✅
TestAccept/bundle/override/cluster_id_clears_new_cluster_reference/DMS=true	✅	✅	✅
TestAccept/bundle/override/preset_clears_forced_schedule_reference/DMS=	✅	✅	✅
TestAccept/bundle/override/preset_clears_forced_schedule_reference/DMS=true	✅	✅	✅
TestAccept/bundle/resource_deps/whole_map_reference_hides_entries/DMS=	✅	✅	✅
TestAccept/bundle/resource_deps/whole_map_reference_hides_entries/DMS=true	✅	✅	✅
TestAccept/bundle/validate/required_struct_reference/DMS=	✅	✅	✅
TestAccept/bundle/validate/required_struct_reference/DMS=true	✅	✅	✅
TestAccept/bundle/validate/secret_scope_invalid_permission_level/DMS=	✅	❌	❌
TestAccept/bundle/validate/secret_scope_invalid_permission_level/DMS=true	✅	❌	❌

TestAccept/bundle/validate/secret_scope_invalid_permission_level/DMS=: branch ✅ | main (bef6321) ❌ | latest (v1.21.0) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/validate/secret_scope_invalid_permission_level/output.txt
        +++ /tmp/TestAcceptbundlevalidatesecret_scope_invalid_permission_level2755409355/002/output.txt
        @@ -2,11 +2,9 @@
         >>> [CLI] bundle validate
         Warning: invalid value "12" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[0].level
        -  in databricks.yml:10:18
         
         Warning: invalid value "fuzz_badlevel" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[1].level
        -  in databricks.yml:12:18
         
         Name: test-bundle
         Target: default
        @@ -19,11 +14,9 @@
         >>> [CLI] bundle validate --strict
         Warning: invalid value "12" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[0].level
        -  in databricks.yml:10:18
         
         Warning: invalid value "fuzz_badlevel" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[1].level
        -  in databricks.yml:12:18
         
         Name: test-bundle
         Target: default

latest (v1.21.0): same output as main (bef6321)

TestAccept/bundle/validate/secret_scope_invalid_permission_level/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.21.0) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/validate/secret_scope_invalid_permission_level/output.txt
        +++ /tmp/TestAcceptbundlevalidatesecret_scope_invalid_permission_level15001421/002/output.txt
        @@ -2,11 +2,9 @@
         >>> [CLI] bundle validate
         Warning: invalid value "12" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[0].level
        -  in databricks.yml:10:18
         
         Warning: invalid value "fuzz_badlevel" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[1].level
        -  in databricks.yml:12:18
         
         Name: test-bundle
         Target: default
        @@ -19,11 +14,9 @@
         >>> [CLI] bundle validate --strict
         Warning: invalid value "12" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[0].level
        -  in databricks.yml:10:18
         
         Warning: invalid value "fuzz_badlevel" for enum field. Valid values are [READ WRITE MANAGE]
           at resources.secret_scopes.bad_levels.permissions[1].level
        -  in databricks.yml:12:18
         
         Name: test-bundle
         Target: default
        
FAIL

latest (v1.21.0): same output as main (bef6321)

Replace the generated pattern tables, matched against every node of the dynamic configuration with a trie, by generated walkers that descend only along the patterns' paths of the typed configuration. Null, zero, pure-reference values and locations still come from the dyn.Value at each node (walk_node.go), so the diagnostics are unchanged. Slice of #7098. Co-authored-by: Isaac <no-reply@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>

…res resource Co-authored-by: Isaac <no-reply@databricks.com>

github-actions Bot added the DABs DABs related issues label Oct 21, 2026

denik changed the title ~~Generate typed walkers for the required and enum field validators~~ validate: check required and enum fields with generated typed walkers Oct 21, 2026

denik force-pushed the denik/required-enum-typed-walkers branch 2 times, most recently from 8df35af to 50e9336 Compare October 21, 2026 15:50

denik changed the base branch from main to denik/typed-mutators-simple October 21, 2026 15:50

denik added this pull request to stack #7150 October 21, 2026 15:50

This was referenced Oct 21, 2026

bundle: delete libs/dyn; typed configuration with generated walkers #7098

Draft

acc: record main's behaviour for reference, override and required-field edge cases #7130

Draft

bundle: read and edit the typed config in five mutators #7145

Draft

denik force-pushed the denik/required-enum-typed-walkers branch from 50e9336 to ab6c0ac Compare October 21, 2026 16:03

denik and others added 3 commits October 21, 2026 18:11

Name the generated enum value variables after the enum type

da6e830

Co-authored-by: Isaac <no-reply@databricks.com>

Regenerate the walkers after rebasing on the SDK update and the featu…

2f2e8ac

…res resource Co-authored-by: Isaac <no-reply@databricks.com>

denik force-pushed the denik/required-enum-typed-walkers branch from ab6c0ac to 2f2e8ac Compare October 21, 2026 16:12

Navigation Menu

Repository navigation

validate: check required and enum fields with generated typed walkers - #7143

validate: check required and enum fields with generated typed walkers#7143
denik wants to merge 3 commits into
denik/typed-mutators-simplefrom
denik/required-enum-typed-walkers

denik commented Oct 21, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Oct 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

denik commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Behavior

Tests

Benchmarks

Uh oh!

eng-dev-ecosystem-bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

github-actions Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression test report

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

denik commented Oct 21, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Oct 21, 2026 •

edited

Loading

github-actions Bot commented Oct 21, 2026 •

edited

Loading