Advancing Analytics: Data Engineering

SEPTEMBER 7, 2021

Without descriptive metadata and a mechanism to maintain it, the data lake risks turning into a data swamp." - Gartner 2014 - [link] So here is where these next few blog articles come in. See The Data Warehouse Toolkit - Kimball & Ross for more information. Here is where the Delta Lake comes in.

Data Lake 52

Data Lake Datasets Data Warehouse SQL 52

Maxime Beauchemin

JANUARY 20, 2017

The fact that ETL tools evolved to expose graphical interfaces seems like a detour in the history of data processing, and would certainly make for an interesting blog post of its own. In retrospect, complex SCD modeling techniques are not intuitive and reduce accessibility.

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130

Christophe Blefari

DECEMBER 30, 2022

Analytics engineers provide clean data sets to end users, modeling data in a way that empowers end users to answer their own questions. [.], 1 Analytics Engineering brought back light on data modeling. 1 Analytics Engineering brought back light on data modeling. Preset wrote a gentle introduction to data modeling.

Data Warehouse 130

Data Warehouse BI Data Machine Learning 130

Data Engineering Weekly

SEPTEMBER 11, 2023

Niels Claeys: Head-to-head comparison of 3 dbt SQL engines The blog post compares benchmarks for the popular dbt SQL engines Trino, DuckDB, and Spark. link] Faith Lierheimer: An appropriately unhinged deep dive into Kimball's dimensional modeling primer An interesting article about Kimball’s dimensional modeling.

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Monte Carlo

FEBRUARY 9, 2023

In fact, with increasingly strict data regulations like GDPR and a renewed emphasis on optimizing technology costs, we’re now seeing a revitalization of “ Data Vault 2.0 ” data modeling. In this blog post we’ll dive into data vault architecture; challenges and best practices for maintaining data quality; and how data observability can help.

Architecture 52

Architecture Raw Data Metadata Data Warehouse 52

Christophe Blefari

AUGUST 4, 2023

As a disclaimer, this may not quite make sense in a corporate context, but since this is my blog, I'll do what I want. How to model: Kimball vs One Big Table — This is one of the main topic of discussion in the data space. 💡 If you just want a few articles to read, just go to the bottom of the email.

Google Cloud 225

Google Cloud MongoDB NoSQL Data 225

Data Science Blog: Data Engineering

NOVEMBER 15, 2023

The Event Log Data Model for Process Mining Process Mining as an analytical system can very well be imagined as an iceberg. The creation of this data model requires the data connection to the source system (e.g. SAP ERP), the extraction of the data and, above all, the data modeling for the event log.

Architecture 52

Architecture Database-centric Process BI 52