Report PyDABs deploy telemetry per resource type by Sankalp-Mittal · Pull Request #6890 · databricks/cli

Sankalp-Mittal · 2026-09-30T11:52:13Z

Changes

Add a top-level pydabs section to the bundle deploy telemetry event (BundleDeployPydabs) with:
- resource_loaders_count, resource_mutators_count
- added_resources_count, updated_resources_count
- resources: one entry per resource type added or updated by Python (resource_type, added_count, updated_count), sorted by type.
- config_section: where the user declared Python support: PYTHON, EXPERIMENTAL_PYTHON, or BOTH.
The section is sent only when the bundle declares Python resource loaders or mutators.
The Python mutator now records per-type counts (bundle.Metrics.PythonAddedResources / PythonUpdatedResources) instead of two totals.
config_section is recorded at the start of the load_resources phase, before applyBackwardsCompatibilityFixes copies python into experimental.python. After that copy the two sections can't be told apart.
experimental.python_* is no longer sent: pydabs carries the same counts. Readers use pydabs for new CLIs and fall back to experimental.python_* for older ones.

Why

The only per-type resource data in deploy telemetry today is resources_metadata, which covers the merged YAML + Python config. It can't show which resource types are actually defined or modified in Python. This adds counts limited to those resources. config_section shows how many bundles still use experimental.python rather than python.

updated_count includes Python-defined resources that a mutator then modifies, so it can overlap with added_count.

This PR is a draft until the telemetry backend schema accepts the new pydabs field.

Tests

Unit: TestPydabsMetrics (a type only updated, never added) and TestPythonConfigSection (none / both); everything else is covered by the acceptance test.
Acceptance: acceptance/bundle/python/telemetry deploys a YAML job and schema, a Python loader adding a job and a pipeline, and a job mutator, once with python: and once with experimental.python:. It asserts jobs 1 added / 2 updated, pipelines 1 added / 0 updated, no schemas, 1 loader / 1 mutator, and config_section PYTHON then EXPERIMENTAL_PYTHON.
go test ./acceptance -run 'TestAccept/bundle/(telemetry|python)' passes.

This pull request and its description were written by Isaac.

Add a top-level `pydabs` section to the bundle deploy event with loader and mutator counts plus per-resource-type added/updated counts, covering only resources defined or modified in Python. The experimental.python_* fields are still populated and marked deprecated. Also fix loader and mutator counts being reported as 0 for bundles using the top-level `python` section; only `experimental/python` was read. Co-authored-by: Isaac <no-reply@databricks.com>

Add pydabs.config_section (PYTHON, EXPERIMENTAL_PYTHON, BOTH), recorded before the Python mutator copies 'python' into 'experimental/python'. Read loader and mutator counts from 'experimental/python' only, as before: that copy means it already covers the 'python' section. Co-authored-by: Isaac <no-reply@databricks.com>

eng-dev-ecosystem-bot · 2026-09-30T12:35:09Z

Integration test report

Commit: 35c13f8

Run: 39964508354

	Env	✅pass	🙈skip	Time
✅	aws linux-2core-8gb	281	16	4:11
✅	aws-windows-latest-4core-16gb	283	14	4:09
✅	azure linux-2core-8gb	280	16	4:02
✅	azure-windows-latest-4core-16gb	282	14	3:41
✅	gcp linux-2core-8gb	281	16	4:05
✅	gcp-windows-latest-4core-16gb	283	14	3:18

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
4:05	aws-windows-latest-4core-16gb	TestAccept
3:35	azure-windows-latest-4core-16gb	TestAccept
3:12	gcp-windows-latest-4core-16gb	TestAccept

…dabs-telemetry

acceptance/bundle/python/telemetry{,-experimental} assert per-type counts and config_section for both sections, and acceptance/bundle/telemetry/deploy asserts the pydabs section is omitted without Python. Co-authored-by: Isaac <no-reply@databricks.com>

Group the deprecation under one comment, as for the Resource*Count fields, and mark the remaining use with nolint while both are populated. Co-authored-by: Isaac <no-reply@databricks.com>

Deploy once with 'python' and once with 'experimental/python' in the same test instead of two near-identical directories, and run only the current databricks-bundles version: the telemetry doesn't depend on it. Co-authored-by: Isaac <no-reply@databricks.com>

github-actions · 2026-10-20T13:14:00Z

Commit: 35c13f8d0 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0

test	branch	main (`edebd84`)	latest (v1.22.1)
TestAccept/bundle/python/telemetry/DMS=/PYDAB_VERSION=current	✅	❌	❌

TestAccept/bundle/python/telemetry/DMS=/PYDAB_VERSION=current: branch ✅ | main (edebd84) ❌ | latest (v1.22.1) ❌

main (edebd84):

acceptance_test.go:1311: Diff:
        --- bundle/python/telemetry/output.txt
        +++ /tmp/TestAcceptbundlepythontelemetryDMS=PYDAB_VERSION=current2443699480/002/output.txt
        @@ -9,25 +9,7 @@
         Files: 0 uploaded, 0 deleted
         Resources: 4 created, 0 changed, 0 deleted, 0 unchanged
         {
        -  "pydabs": {
        -    "resource_loaders_count": 1,
        -    "resource_mutators_count": 1,
        -    "added_resources_count": 2,
        -    "updated_resources_count": 2,
        -    "resources": [
        -      {
        -        "resource_type": "jobs",
        -        "added_count": 1,
        -        "updated_count": 2
        -      },
        -      {
        -        "resource_type": "pipelines",
        -        "added_count": 1,
        -        "updated_count": 0
        -      }
        -    ],
        -    "config_section": "PYTHON"
        -  },
        +  "pydabs": null,
           "all_resource_types": [
             "jobs",
             "pipelines",
        @@ -43,25 +25,7 @@
         Files: 0 uploaded, 0 deleted
         Resources: 0 created, 0 changed, 0 deleted, 4 unchanged
         {
        -  "pydabs": {
        -    "resource_loaders_count": 1,
        -    "resource_mutators_count": 1,
        -    "added_resources_count": 2,
        -    "updated_resources_count": 2,
        -    "resources": [
        -      {
        -        "resource_type": "jobs",
        -        "added_count": 1,
        -        "updated_count": 2
        -      },
        -      {
        -        "resource_type": "pipelines",
        -        "added_count": 1,
        -        "updated_count": 0
        -      }
        -    ],
        -    "config_section": "EXPERIMENTAL_PYTHON"
        -  },
        +  "pydabs": null,
           "all_resource_types": [
             "jobs",
             "pipelines",
        
FAIL

latest (v1.22.1): same output as main (edebd84)

janniklasrose · 2026-10-23T09:56:35Z

+	PydabsConfigSectionUnspecified        PydabsConfigSection = "TYPE_UNSPECIFIED"
+	PydabsConfigSectionPython             PydabsConfigSection = "PYTHON"
+	PydabsConfigSectionExperimentalPython PydabsConfigSection = "EXPERIMENTAL_PYTHON"
+	PydabsConfigSectionBoth               PydabsConfigSection = "BOTH"


what's the purpose of BOTH, can't this be consolidated as a union of the other two?

this is an enum and not bool so union doesn't work over here, while I could change it to a bool that would need a proto change and I don't think there is any major advantage to doing that over just having an enum

Fill one entry per type from the added and updated maps and sort once, instead of concatenating, sorting and compacting the key lists. Co-authored-by: Isaac <no-reply@databricks.com>

…dabs-telemetry

The direct engine is now the only engine, and the acceptance harness rejects EnvMatrix.DATABRICKS_BUNDLE_ENGINE. Co-authored-by: Isaac <no-reply@databricks.com>

The pydabs section carries the same counts. Consumers read pydabs for new CLIs and fall back to experimental.python_* for older ones. Co-authored-by: Isaac <no-reply@databricks.com>

github-actions Bot added DABs DABs related issues PyDABs labels Sep 30, 2026

Sankalp-Mittal and others added 2 commits October 19, 2026 09:02

Merge remote-tracking branch 'origin/main' into sankalp-mittal/fix-py…

ab2c541

…dabs-telemetry

Sankalp-Mittal marked this pull request as ready for review October 20, 2026 13:02

Sankalp-Mittal requested review from a team as code owners October 20, 2026 13:02

Sankalp-Mittal requested review from janniklasrose, maksim-zuev-db and rugpanov and removed request for janniklasrose, maksim-zuev-db and rugpanov October 20, 2026 13:02

Sankalp-Mittal and others added 2 commits October 20, 2026 13:04

Fix SA1019 lint on deprecated experimental.python_* telemetry fields

53f5215

Group the deprecation under one comment, as for the Resource*Count fields, and mark the remaining use with nolint while both are populated. Co-authored-by: Isaac <no-reply@databricks.com>

janniklasrose reviewed Oct 23, 2026

View reviewed changes

andrewnester reviewed Oct 23, 2026

View reviewed changes

Comment thread libs/telemetry/protos/bundle_deploy.go

Comment thread bundle/bundle.go

Comment thread bundle/phases/telemetry.go Outdated

Build pydabs per-type counts with two loops

c84081b

Fill one entry per type from the added and updated maps and sort once, instead of concatenating, sorting and compacting the key lists. Co-authored-by: Isaac <no-reply@databricks.com>

Sankalp-Mittal requested review from andrewnester and janniklasrose October 23, 2026 12:53

andrewnester approved these changes Oct 23, 2026

View reviewed changes

Sankalp-Mittal and others added 3 commits October 23, 2026 13:11

Merge remote-tracking branch 'origin/main' into sankalp-mittal/fix-py…

edb9983

…dabs-telemetry

Drop the engine matrix from the PyDABs telemetry test

3509dd3

The direct engine is now the only engine, and the acceptance harness rejects EnvMatrix.DATABRICKS_BUNDLE_ENGINE. Co-authored-by: Isaac <no-reply@databricks.com>

Stop sending experimental.python_* deploy telemetry

35c13f8

The pydabs section carries the same counts. Consumers read pydabs for new CLIs and fall back to experimental.python_* for older ones. Co-authored-by: Isaac <no-reply@databricks.com>

Navigation Menu

Repository navigation

Report PyDABs deploy telemetry per resource type - #6890

Report PyDABs deploy telemetry per resource type#6890
Sankalp-Mittal wants to merge 10 commits into
mainfrom
sankalp-mittal/fix-pydabs-telemetry

Sankalp-Mittal commented Sep 30, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Sep 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 20, 2026 •

edited

Loading

Uh oh!

janniklasrose Oct 23, 2026

Uh oh!

Sankalp-Mittal Oct 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Sankalp-Mittal commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

eng-dev-ecosystem-bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression test report

Uh oh!

janniklasrose Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

Sankalp-Mittal Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Sankalp-Mittal commented Sep 30, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Sep 30, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading