Data Engineering Newsletter#25
Data Engineering News
1. 2025 Guide to Architecting an Iceberg Lakehouse
"Is your data architecture ready for Iceberg in 2025?”
The author says that designing an Apache Iceberg lakehouse is not just about tools; it’s about rethinking how you govern, scale, and serve data across teams. With major 2024 shifts (like Databricks acquiring Tabular and Snowflake’s Polaris Catalog), this 2025 guide offers a clear, honest look at how to architect a modern Iceberg lakehouse, from storage and catalogs to ingestion and consumption.
2. Building a Data Lakehouse with Iceberg, Spark, and AWS Glue
"Can Apache Iceberg, Spark, and AWS Glue be the open stack you need for your next-gen data lakehouse?”
Ritam Mukherjee says that yes, you can, by combining Apache Iceberg, Spark, and AWS Glue into a clean, modern architecture. This short and practical guide walks you through every step, from raw S3 data to Iceberg tables and Athena queries, all without managing infrastructure.
3. Open Source Data Engineering Landscape 2025
"Is your data stack ready for the open source shift happening in 2025?"
Alireza Sadeghi discusses that open-source data engineering is not just growing; it’s reshaping itself. From zero-disk architectures and native Python libraries to catalog wars and single-node processing, this 2025 landscape maps the most important tools, trends, and tectonic shifts shaping the future of data infrastructure.
https://medium.com/@ApacheDolphinScheduler/open-source-data-engineering-landscape-2025-db53ce18d53d
4. Why Walmart Chose Apache Hudi for Their Lakehouse
"Why did one of the world’s biggest retailers pick Apache Hudi for their lakehouse over Iceberg or Delta?”
Vu Trinh discusses Walmart’s decision to ignore the hype. They chose Apache Hudi for its low-latency upserts, real-time processing, and open-source flexibility across clouds. Through deep benchmarking, they found Hudi best suited to their scale, control needs, and hybrid workloads.
https://blog.det.life/why-walmart-chose-apache-hudi-for-their-lakehouse-c0a3574db0ba
5. Step-by-Step Guide - Replicating PostgreSQL to Iceberg with OLake & AWS Glue
"How do you replicate Postgres data to Iceberg tables using OLake and AWS Glue, step by step?”
Rohan Khameshra explains that Postgres-to-Iceberg replication doesn’t need to be hard or expensive. Using OLake and AWS Glue, he walks you through a step-by-step setup that supports real-time CDC, schema evolution, and cost-efficient analytics at scale.
https://olake.io/iceberg/postgres-to-iceberg-using-glue
All rights reserved Den Digital, India. I have provided links for informational purposes and do not suggest endorsement. All views expressed in this newsletter are my own and do not represent current, former, or future employer” opinions.