Skip to content

CDC Lakehouse Assignment - Rupali Pipare - #19

Open
Rupalinaitam wants to merge 13 commits into
Robustrade:mainfrom
Rupalinaitam:feature/rupali-cdc-lakehouse
Open

CDC Lakehouse Assignment - Rupali Pipare#19
Rupalinaitam wants to merge 13 commits into
Robustrade:mainfrom
Rupalinaitam:feature/rupali-cdc-lakehouse

Conversation

@Rupalinaitam

Copy link
Copy Markdown

CDC Lakehouse Assignment

Completed the CDC Lakehouse implementation as requested.

Highlights

  • CDC event generation for INSERT, UPDATE and DELETE
  • Durable append-only change log
  • Duplicate event handling
  • Checkpoint-based processing and restart support
  • JSONL CDC lake
  • DuckDB current-state warehouse
  • Historical recovery
  • Schema safety validation
  • Warehouse validation parity
  • Dataset catalog
  • Source SQLite reader and CDC generator

Validation

  • All 44 automated tests are passing.
  • Working tree is clean.
  • Submission branch: feature/rupali-cdc-lakehouse

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant