Data Engineering Weekly

JANUARY 22, 2023

Editor’s Note: Update on our blog series One of the promises I made toward the end of 2022 is to publish more of my thoughts and industry observation of data engineering trends. Data Catalog - A broken promise A classic blog triggers a few conversations about Data Catalog and its future. So far, we have published.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Snowflake

AUGUST 31, 2023

In this blog post, we’ll dive into the details of these features and the benefits for customers. Recapping our journey with Iceberg Last year we announced two Iceberg Table previews based on functionality already available in Snowflake. In 2022, We announced added support for Iceberg for External Tables to easily read Iceberg datasets.

Metadata 80

Metadata AWS Data Lake Datasets 80

Data Engineering Weekly

JANUARY 15, 2023

We are seeing embedded data catalog tools like reCap emerging to address the concerns. 3:23 PM ∙ Jan 6, 2022 18 Likes 4 Retweets But I like the term “Lots of Islands of Simplicity, but it’s a complex toolchain.” The conversation around the Cost of Idle CPU in Snowflake is exciting to me in the blog.

Data Engineering 52

Data Engineering Data Engineer Engineering Metadata 52

Big Data Tools

JULY 13, 2022

The authors claim that Spark, with its current architecture, lacks 4 important traits: built-in remote connectivity, a rich developer experience, stability, and upgradability. The State of Data Engineering 2022 – I like this kind of content. Big Data Event: London – This is going to be a huge data event in London.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Big Data Tools

JULY 13, 2022

The authors claim that Spark, with its current architecture, lacks 4 important traits: built-in remote connectivity, a rich developer experience, stability, and upgradability. The State of Data Engineering 2022 – I like this kind of content. Big Data Event: London – This is going to be a huge data event in London.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52