Skip to content
View LuciferDataEngineer's full-sized avatar

Block or report LuciferDataEngineer

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse

Hi, I'm LuciferDataEngineer 👋

Data Engineer · Analytics Engineer · Melbourne, Australia

I build production-style data pipelines on open data: ingestion, lakehouse modelling, data quality and decision-focused dashboards.


🛠️ Tech stack

Python SQL dbt DuckDB PostgreSQL Dagster Pandas Kafka Apache Iceberg Apache Spark Delta Lake Debezium Airflow Apache Flink Feast MLflow Kubernetes OpenTofu pgvector Hugging Face ECharts Docker GitHub Actions Plotly Streamlit

Patterns I work with: medallion and lakehouse architecture · incremental loads and high-watermarks · dimensional modelling · data contracts · dbt tests and data quality · orchestration · CI/CD for data · static dashboards on GitHub Pages


🚀 Featured projects

Project What it answers Stack Live
supply-chain-control-tower Logistics control tower on 100k real e-commerce orders: on-time delivery, late lanes, freight burden, seller delays and review impact, with recommendations DuckDB · dbt · Dagster · ECharts · GitHub Actions Live dashboard
rba-rate-shock-copilot An AI copilot answering questions about the RBA's 2026 cash-rate rises and what they cost mortgage holders, with RAG over Reserve Bank statements Postgres + pgvector · sentence-transformers embeddings · RAG · MetricFlow semantic layer · dbt-duckdb · Dagster · OpenTofu Live dashboard
vic-heat-health-sentinel Which Victorian areas face dangerous heat in the next 16 days, who lives there, and where should councils prepare cooling centres and outreach? Redpanda · PyFlink · Delta Lake · Feast · scikit-learn + MLflow · Evidently · dbt-duckdb · Dagster · OpenTofu · Kubernetes (kind) Live dashboard
fuel-shock-radar Australia's 2026 fuel price shock: which day is cheapest to fill up, are retail margins fair, and how much more do regional drivers pay? Postgres · Debezium CDC · Redpanda · PySpark Structured Streaming · Delta Lake (MERGE/SCD2) · Airflow Datasets · OpenLineage/Marquez · Great Expectations · DuckDB Live dashboard
aus-hazard-stream Is the 2026 Gippsland earthquake swarm unusual, and where do quakes and bushfire hotspots threaten people and infrastructure? (near real-time) Redpanda/Kafka · JSON Schema registry · Apache Iceberg · H3 + DuckDB spatial · dbt-duckdb · Dagster · Soda · MapLibre Live map
nem-grid-pulse Will Australia's AI data-centre boom strain the electricity grid? Prices, negative-price hours, renewables and load scenarios across the NEM Python · dbt-duckdb · Dagster · data contracts · ECharts · GitHub Actions Dashboard
nt-education-insights Why is Aboriginal school attendance in the NT falling, where is it worst, and what should policymakers act on? Python · DuckDB · pandas · medallion pipeline · Plotly · Streamlit Dashboard
victorian-home-data-platform A free-first medallion platform for Victorian property market research Python · medallion architecture

All projects use free, openly licensed public data (AEMO, ACARA, ABS, NT Department of Education and others) and document their sources, cleaning, transformations and limitations in the README.


📊 GitHub stats

🤝 Let's connect

Open to data engineering and analytics engineering roles in Australia. Feel free to open an issue on any project or reach out.

Popular repositories Loading

  1. victorian-home-data-platform victorian-home-data-platform Public

    Free-first medallion data engineering platform for Victorian property market research

    Python 1

  2. gdp-dashboard gdp-dashboard Public

    Python 1

  3. nt-education-insights nt-education-insights Public

    Public evidence on NT school attendance, NAPLAN, retention and resourcing for education policymakers and NT Department of Education stakeholders.

    Python

  4. nem-grid-pulse nem-grid-pulse Public

    Electricity price, demand and generation mix in Australia's National Electricity Market, built as a dbt + DuckDB lakehouse on free AEMO data, with a focus on AI data-centre load.

    Python

  5. LuciferDataEngineer LuciferDataEngineer Public

    Profile README

  6. aus-hazard-stream aus-hazard-stream Public

    Near real-time streaming lakehouse for Australian earthquakes and bushfire hotspots: Redpanda (Kafka) → Apache Iceberg → dbt-duckdb, with geospatial (H3) analytics and a live map dashboard on free …

    Python