Skip to content

Recover batched Excel writes onto main - #1118

Merged
ebhills merged 5 commits into
mainfrom
codex/recover-excel-batch-writes
Aug 12, 2026
Merged

Recover batched Excel writes onto main#1118
ebhills merged 5 commits into
mainfrom
codex/recover-excel-batch-writes

Conversation

@ebhills

@ebhills ebhills commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • prevent recipe batching from writing partial intermediate results
  • accumulate repeated Excel sheet writes in both append and overwrite modes
  • switch subsequent overwrite batches to append so earlier output is retained
  • union dynamic or reordered batch columns in first-seen order
  • align every row by column name rather than by positional order
  • add focused connector and recipe regression coverage for all of these cases

Why

Recipe batches can write repeatedly to the same Excel sheet. Without this chain, intermediate batches can be written independently, overwrite mode can discard earlier rows, and dynamic dictionary keys can shift values into the wrong columns when batches are combined positionally. These are silent data-loss or data-alignment risks.

Recovery provenance

This is a clean recovery of the five commits previously merged to legacy dev in #1112.

  • branch created from current origin/main
  • verified zero commits behind main before recovery
  • replayed the complete prerequisite chain in its original order
  • git range-diff shows all five recovered commits are patch-equivalent to the preserved Preserve batched Excel writes across recipe batches #1112 chain
  • no dev branch, synchronization commit, or merge commit was included
  • final scope is limited to five Excel/recipe implementation and test files

Original PR: #1112

Compatibility and risk

The intended behavior change is limited to repeated writes produced by recipe batching. Single writes retain their existing path. The main risk is batch accumulation or column-union behavior, covered directly by the focused tests.

Rollback is the normal revert of this five-commit PR; no data migration or configuration change is required.

Validation

  • 8 passed: every newly added regression test
  • 37 passed, 2 deselected: complete Excel connector tests, recipe connector tests, and TestBatch
  • the two deselected tests require live authentication to sso.wrangle.works and failed only because that external service is unavailable from the local sandbox
  • existing pandas SettingWithCopyWarning warnings remain; no new test failure was attributed to this recovery
  • git diff --check passed
  • branch is zero commits behind main
  • changed-file scope against main:
    • tests/connectors/test_excel.py
    • tests/connectors/test_recipe.py
    • tests/recipes/wrangles/test_main.py
    • wrangles/connectors/excel.py
    • wrangles/recipe_wrangles/main.py

mborodii-prog and others added 5 commits August 11, 2026 19:44
Handle repeated `excel.sheet.write` batches with changing or reordered columns by aligning rows on column names instead of position. This preserves accumulated Excel output for dynamic dictionary splits and keeps overwrite batches in the original column order, with tests covering both connector-level and recipe batch behavior.
@ebhills ebhills self-assigned this Aug 12, 2026
@ebhills
ebhills marked this pull request as ready for review August 12, 2026 00:56
@ebhills
ebhills merged commit abb5fdf into main Aug 12, 2026
13 checks passed
@ebhills
ebhills deleted the codex/recover-excel-batch-writes branch August 12, 2026 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants