Data Engineering Weekly

NOVEMBER 26, 2023

🤔 Reflecting on the past is crucial; it gives us a solid foundation to understand how trends have evolved and what's worked (or not). 🕵️‍♂️📈 Before we delve into the patterns, it's important to remember that data infrastructure maturity model. Let me know your thoughts in the comments.

Data Engineering 70

Data Engineering Data Engineer Engineering Kafka 70

Data Engineering Weekly

APRIL 2, 2023

Contribute to the Rudderstack Transformations Library, Win $1000 RudderStack Transformations lets you customize event data in real-time with your own JavaScript or Python code. Meta: Presto - A Decade of SQL Analytics at Meta Presto and Kafka are the two systems that greatly impacted data infrastructure in the last decade.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

DoorDash Engineering

OCTOBER 17, 2023

Experimentation isn’t just a cornerstone for innovation and sound decision-making; it’s often referred to as the gold standard for problem-solving, thanks in part to its roots in the scientific method. High-fives are exchanged and your team is already setting its sights on the next big project. between control and treatment groups.

Education 87

Education Kafka Algorithm Data Warehouse 87

Cloudera

SEPTEMBER 28, 2020

The customer had a few primary reasons for the upgrade: Utilize existing hardware resources and avoid the expensive resources, time and cost of adding new hardware for migrations. . Support Kafka connectivity to HDFS, AWS S3 and Kafka Streams. Cluster management and replication support for Kafka clusters.

Cloud 130

Cloud Kafka Professional Services Metadata 130

Data Engineering Weekly

MARCH 19, 2023

Contribute to the Rudderstack Transformations Library, Win $1000 RudderStack Transformations lets you customize event data in real time with your own JavaScript or Python code. The author makes an interesting analogy, if you buy your favorite cereal without the box, ingredient details, and other relevant information, do you trust it?

Data Engineering 57

Data Engineering Data Engineer Engineering Media 57

ProjectPro

DECEMBER 7, 2021

Data pipelines are a significant part of the big data domain, and every professional working or willing to work in this field must have extensive knowledge of them. To understand the working of a data pipeline, one can consider a pipe that receives input from a source that is carried to give output at the destination.

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

Rockset

APRIL 29, 2019

In this blog post I compare options for real-time analytics on DynamoDB - Elasticsearch , Athena, and Spark - in terms of ease of setup, maintenance, query capability, latency. However, as an operational database optimized for transaction processing, DynamoDB is not well-suited to delivering real-time analytics.

NoSQL 52

NoSQL PostgreSQL AWS SQL 52