Skip to content

Recover corrupted DMS deployment-history markers - #7065

Draft
shreyas-goenka wants to merge 2 commits into
shreyas-goenka_data/shreyas-goenka/dms-marker-upload-stackfrom
shreyas-goenka_data/shreyas-goenka/dms-marker-recovery-stack
Draft

shreyas-goenka wants to merge 2 commits into
shreyas-goenka_data/shreyas-goenka/dms-marker-upload-stackfrom
shreyas-goenka_data/shreyas-goenka/dms-marker-recovery-stack

Conversation

@shreyas-goenka

Copy link
Copy Markdown
Contributor

Changes

Repair a missing or malformed resources.json marker only after confirming its existing DMS deployment. Reuse a valid surviving recorded header; otherwise create a new lineage and warn that old saved plans are invalidated. Upload the repaired marker through the approved deployment path.

Reuse the deployment lookup during initialization. Preserve direct-state resources and schema, feature, WAL, access, and service error checks. Fresh non-DMS invocations do not probe DMS.

Stacked above #7056; migration remains before the three follow-up PRs.

Why

DMS holds the resources and versions, so a damaged marker should be recoverable without recreating resources. This supplies the third write case after first deployment and migration.

Tests

Focused package tests/lint and state/DMS/user-agent acceptance pass. Recovery tests cover surviving/fresh lineage and negative gates; upload assertions verify one repair write, unchanged resource IDs, and advancing service versions.

Repository-wide quick checks, Go lints, doctests, and unit tests pass (the invalid local pip index was removed for the test process only). Full acceptance still reports migration-baseline expectation failures and unrelated dependency/SSH failures; Python formatting/lint could not fetch pinned Ruff because PyPI DNS is unavailable. No unrelated expectations were rewritten.

Implementation and PR description written with Codex.

@shreyas-goenka
shreyas-goenka added this pull request to stack #7047 October 14, 2026 15:45
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: c43801a

Run: 38592154945

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 7 4 292 28 28:58
❌​ aws-windows-latest-4core-16gb 7 2 296 26 25:55
❌​ azure linux-2core-8gb 7 2 293 28 26:30
❌​ azure-windows-latest-4core-16gb 7 4 293 26 30:02
❌​ gcp linux-2core-8gb 7 296 28 23:29
❌​ gcp-windows-latest-4core-16gb 7 2 296 26 28:12
19 interesting tests: 12 flaky, 7 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
❌​ TestAccept ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/dms/declined-deploy ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/dms/declined-deploy/DMS=true ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/dms/depends-on ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/dms/depends-on/DMS=true ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/dms/existing-state ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/dms/existing-state/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/dms/marker-writes ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/dms/marker-writes/DMS=true ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/dms/no-drift 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/dms/no-drift/DMS=true 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/dms/repeated-migration ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/dms/repeated-migration/DMS=true ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/dms/stale-local-state ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/dms/stale-local-state/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/dms/stale-plan 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN= 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestSyncIncrementalFileSync ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestSyncIncrementalSyncPythonNotebookDelete ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 25 slowest tests (at least 2 minutes):
duration env testname
4:54 gcp linux-2core-8gb TestAccept/bundle/dms/marker-writes/DMS=true
4:16 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/marker-writes/DMS=true
4:01 aws-windows-latest-4core-16gb TestAccept/bundle/dms/marker-writes/DMS=true
3:46 aws linux-2core-8gb TestAccept/bundle/dms/marker-writes/DMS=true
3:32 azure-windows-latest-4core-16gb TestAccept/bundle/dms/marker-writes/DMS=true
3:28 aws-windows-latest-4core-16gb TestTelemetryEndpoint
3:12 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:53 aws-windows-latest-4core-16gb TestImportFileFormatSource
2:53 azure linux-2core-8gb TestAccept/bundle/dms/marker-writes/DMS=true
2:52 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:51 aws-windows-latest-4core-16gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=
2:45 gcp linux-2core-8gb TestAccept/bundle/dms/no-drift/DMS=true
2:44 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:38 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:32 aws-windows-latest-4core-16gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=1
2:30 azure-windows-latest-4core-16gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=
2:24 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=
2:17 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:16 azure linux-2core-8gb TestAccept/bundle/dms/declined-deploy/DMS=true
2:16 azure linux-2core-8gb TestFilerRecursiveDelete/workspace_files_extensions
2:13 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
2:11 azure-windows-latest-4core-16gb TestAccept/bundle/dms/multiple-resources/DMS=true
2:05 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:02 gcp linux-2core-8gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=1
2:00 azure-windows-latest-4core-16gb TestAccept/bundle/dms/depends-on/DMS=true

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants