apache-hudi

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.

Lakehouse storage system benchmark

Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work

A Smart Traffic Management System for Ho Chi Minh City, Vietnam leveraging batch and real-time data processing, intuitive dashboards, and monitoring tools to optimize traffic flow, enhance safety, and support sustainable urban mobility through advanced analytics and user-friendly applications.

Change Data Capture (CDC) pipeline that streams data from MariaDB to S3-compatible object storage using Apache Flink and Apache Hudi.

Master Azure Data Engineering with this Basic to Advance guide! Covers SQL, PySpark, Kafka, Databricks, Snowflake & Airflow. Build 15+ industrial projects using Azure (ADF, Synapse, Event Hubs), GCP & modern table formats (Delta Lake, Iceberg, Hudi). Learn real-time streaming, Medallion architecture, and cloud data warehousing with hands-on labs.

Apache Hudi examples designed to be run on AWS Elastic Map Reduce (EMR) via. EMR Studio or EMR Notebooks

First iteration of an entire repository representing a polylithic enterprise DataLake POC in AWS.

Explore Azure data engineering from basic SQL to advanced cloud pipelines, real-time streaming, and scalable data workflows.

Nearby Beverage Explorer currently at City Hall MRT Station (25.041171, 121.565227). It is used to track beverage store ratings on Google Maps.

Diploma thesis for ECE NTUA in the course 3189 Advanced Topics in Database Systems under the supervision of Prof. Dimitrios Tsoumakos.

This project demonstrates a real-time Change Data Capture (CDC) pipeline using Apache Hudi, where inventory changes from PostgreSQL are streamed through Kafka and materialized into a lakehouse storage layer on MinIO. Built using an open-source lakehouse stack with Apache Hudi as the core storage layer.

Onehouse is a fully managed universal data lakehouse platform, built by the original creators of Apache Hudi. It provides near real-time data ingestion (CDC and streaming), incremental ETL/ELT pipelines, automated table optimization (compaction, clustering, cleaning), and open table-format interoperability across Apache Hudi, Apache Iceberg, and…

PySpark pipeline benchmarking Delta Lake, Apache Hudi, and Apache Iceberg open table formats — performance comparison with 36 tests passing.

Production-grade Meta Ads analytics pipeline — Airbyte → S3 Bronze → Apache Hudi Silver → PostgreSQL Gold star schema, built on PySpark and AWS Glue.

Open Table Format is a category of open standards for organizing and managing data in data lakehouses. The three dominant formats are Apache Iceberg (the emerging industry standard with snapshot-based metadata and broad engine support), Delta Lake (Databricks-originated, transaction-log-based), and Apache Hudi (upsert-optimized with Copy-on-Write…

Apache Hudi examples designed to be run on AWS Glue via. Glue Jobs

Irodori Table Apache Hudi connector extension