Add job_runs lifecycle.triggers.on_value_change - #6341
Open
radakam wants to merge 74 commits into
Open
Conversation
Re-fire a run when matched file contents change, using stable hash fingerprints so mtime-only updates (touch) do not recreate.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
The modernize linter rejects the manual m[k]=v copy loop, failing CI lint.
Honor .gitignore and sync.exclude when hashing matches. Cover mysubdir globs, READPLAN, and keep LF on Windows so restored files hash cleanly.
The glob test edits migrations/ignored.txt, but the test .gitignore excluded it from the commit so CI clones would miss the file.
Plan serialization is already covered by on_file_change; the glob cases only assert the hash map.
Cover deploying glob hashes from a saved plan, not only from a live re-plan.
Keep the gitignore fixture as dot_gitignore so it does not affect the repo while developing, drop unit tests that duplicate acceptance coverage, and allocate trigger state once when a trigger is armed.
The previous simplification omitted OnBundleDeploy from the struct literal, which the linter rejects.
Always persist the same lifecycle.triggers shape so trigger fingerprints compare at the same paths. Wrap on_file_change hashes in files so dropping the trigger updates instead of recreating.
Nest lifecycle by value so structdiff descends to the leaf that changed instead of reporting the whole subtree, and let RemapState supply the empty shape. GetRun never returns the fingerprints, so the remote type no longer advertises a lifecycle it cannot fill.
radakam
marked this pull request as ready for review
August 21, 2026 15:59
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: 3865b6b
Top 12 slowest tests (at least 2 minutes):
|
Drop the files wrapper so plan diffs land on lifecycle.triggers.on_file_change, matching the config path without an extra state-only level.
The trigger, PrepareState and OverrideChangeDesc cases are asserted end to end by on_bundle_deploy and on_file_change, so the unit copies only duplicated goldens. Keep the ones acceptance cannot reach without stubbing GetRun.
Flattening on_file_change to a map made an unarmed trigger drop out of the serialized state, so the three plans that print an unarmed job_run now report an empty triggers object.
The unit test staged an asynchronous cancellation by hand; the acceptance test shows the real request order a user's destroy produces. Also drops two unit tests that the on_file_change acceptance tests already cover.
Re-fire a run when a resolved expression changes, including resource refs that become deploy-graph edges. Removing the trigger is a state-only update.
Include the on_value_change input and state paths required by generated-file validation.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
August 24, 2026 09:36
0db1f4b to
e431088
Compare
Re-fire a run when matched file contents change, using stable hash fingerprints so mtime-only updates (touch) do not recreate.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
on_value_change_ref already asserts depends_on and persisted resolved ids.
failed_run, on_bundle_deploy, wait, and interrupted_run already assert the user-visible wait and trigger behavior. Add destroy_unfinished_run so cancel-before-delete is covered the same way.
Re-fire a run when matched file contents change, using stable hash fingerprints so mtime-only updates (touch) do not recreate.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
The modernize linter rejects the manual m[k]=v copy loop, failing CI lint.
Honor .gitignore and sync.exclude when hashing matches. Cover mysubdir globs, READPLAN, and keep LF on Windows so restored files hash cleanly.
The glob test edits migrations/ignored.txt, but the test .gitignore excluded it from the commit so CI clones would miss the file.
Plan serialization is already covered by on_file_change; the glob cases only assert the hash map.
Cover deploying glob hashes from a saved plan, not only from a live re-plan.
Keep the gitignore fixture as dot_gitignore so it does not affect the repo while developing, drop unit tests that duplicate acceptance coverage, and allocate trigger state once when a trigger is armed.
The previous simplification omitted OnBundleDeploy from the struct literal, which the linter rejects.
Always persist the same lifecycle.triggers shape so trigger fingerprints compare at the same paths. Wrap on_file_change hashes in files so dropping the trigger updates instead of recreating.
Nest lifecycle by value so structdiff descends to the leaf that changed instead of reporting the whole subtree, and let RemapState supply the empty shape. GetRun never returns the fingerprints, so the remote type no longer advertises a lifecycle it cannot fill.
Drop the files wrapper so plan diffs land on lifecycle.triggers.on_file_change, matching the config path without an extra state-only level.
Flattening on_file_change to a map made an unarmed trigger drop out of the serialized state, so the three plans that print an unarmed job_run now report an empty triggers object.
Directory-only and sync-excluded matches silently disarmed the trigger. Treat them as errors, warn that ** is not recursive, and cover both in acc.
The goldens were captured against a one-line-taller databricks.yml, so CI failed after the source settled on line 24.
Clearing a trigger is now a skip, so the plan reports the run as unchanged and the deploy makes no API call.
Patterns must still land under the sync root. Reject **, warn when a glob matches directories, and skip only when the trigger itself is cleared.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
from
August 25, 2026 08:12
17857f9 to
45c4ccd
Compare
filepath.Join rebases an absolute glob under the bundle root, so reject those up front. Key missing-file placeholders to the sync root, and skip a cleared trigger from path.String() instead of prefix matching.
…s-lifecycle-on-value-change The base branch was rebased and gained YAML-relative on_file_change paths, absolute-pattern rejection, and a reworked prevent_destroy diagnostic. Adapt on_value_change to the new shape: extend preventDestroyError to name every armed trigger instead of listing two by hand, fold on_value_change into the base's path.String() switch for cleared triggers, and drop the jobRunTriggerLocalPaths helper the base replaced.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
from
August 25, 2026 10:12
d65e640 to
b0437a3
Compare
…s-lifecycle-on-value-change The base was rebased onto a newer main and softened the on_file_change "nothing to hash" error into a warning. Adapt on_value_change: follow the base in dropping the unreachable "lifecycle.triggers" case from OverrideChangeDesc, keeping on_value_change alongside on_file_change on the leaf paths, and take the zero-value emptyJobRunLifecycleState now that it no longer lists fields by hand. Also restore the initialize.go ApplySeqContext formatting that the previous merge reverted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add
lifecycle.triggers.on_value_changeforresources.job_runs(direct engine). The run is recreated when the resolved expression changes.${resources.*}refs become deploy-graph dependencies. Multiple triggers are separate list entries (one key each). Removing the trigger is unchanged: it does not recreate or start a new run (same ason_file_change/on_bundle_deploy).Why
job_runsalready supporton_bundle_deployandon_file_change. Watching a resolved value (variable or resource id) is the remaining trigger from the job_runs design, so a run can re-fire when an upstream resource or interpolated value changes.Tests
validate_job_run_triggers(empty/whitespace/two-keys/prevent_destroy). Wait/GetRun stubs stay injob_run_test.go; PrepareInputConfig / DropJobRunValueChangeConfigRefs are covered by acceptance.on_value_change: two expressions; changing one var recreates (other fingerprint stays); removing both triggers is unchanged with no run-now.on_value_change_ref: concatenated${resources.jobs.other.id}-${resources.jobs.extra.id};depends_onincludes both jobs; persisted state is[OTHER_ID]-[EXTRA_ID];READPLAN=""andREADPLAN=1.