Christophe Blefari

MARCH 22, 2024

The process works well, but as you can see, because I use fresh news, it's just-in-time. ❤️ I rarely say it, if Data News helps you save time you should consider taking a paid subscription (60€/year) to help me covers the blog fees and my writing Fridays. Save posts without reading. on April 10.

Electronics 130

Electronics Media Data Python 130

Maxime Beauchemin

JANUARY 20, 2017

In relation to previously existing roles , the data engineering field could be thought of as a superset of business intelligence and data warehousing that brings more elements from software engineering. This includes tasks like setting up and operating platforms like Hadoop/Hive/HBase, Spark, and the like. Data Engineering?

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130

Rockset

JANUARY 23, 2019

Have you ever wanted to run SQL queries on Amazon DynamoDB tables without impacting your production workloads? In this blog, I will discuss how Rockset integrates with DynamoDB and continuously updates a collection automatically as new objects are added to a DynamoDB table. Each field in the dataset is described here.

SQL 40

SQL NoSQL AWS Datasets 40

Picnic Engineering

JULY 26, 2022

Since the publication of the first blog post in this series, we have received numerous questions via social media, direct messages, public posts, and meet-up discussions. Though many of the ongoing questions were already covered in the previous blog posts, a few warranted their own article. Data Mesh, or Hub-and-Spoke? Any advice?

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Maxime Beauchemin

FEBRUARY 28, 2017

You might be wondering, “What is data engineering and why does it matter?” In The Rise of the Data Engineer , Maxime defined data engineering as: The data engineering field could be thought of as a superset of business intelligence and data warehousing that brings more elements from software engineering. We should expect 1.8.0

Data Engineering 130

Data Engineering Data Engineer Engineering Data Pipeline 130

Airbnb Tech

APRIL 13, 2021

Minerva , Apache Druid , DataPortal , Apache Superset , SLA monitoring ) to make data-informed decisions. A foundation of robust and actionable data is essential for a successful efficiency program. More importantly, cost data has become a lever for long-term control. Most teams at Airbnb rely on the data warehouse (i.e.,

AWS 52

AWS Raw Data Amazon Web Services Cloud 52

Rockset

FEBRUARY 25, 2021

For example, you might have to develop a real-time data pipeline using a tool like Kafka just to get the data in a format that allows you to aggregate or join data in a performant manner. Two of these real-time analytics solutions are Elasticsearch and Rockset. Instead, this data is often semi-structured in JSON or arrays.

SQL 40

SQL Data Pipeline Kafka Database 40