Monte Carlo

JANUARY 16, 2024

In this blog post, we’ll look at six innovations that are shaping the future of the data warehousing, as well as challenges and considerations that organizations should keep in mind. Data lake and data warehouse convergence 2. At the center of it all is the data warehouse, the lynchpin of any modern data stack.

Data Lake 52

Data Lake Data Warehouse Unstructured Data AWS 52

Picnic Engineering

SEPTEMBER 8, 2022

We continue our story on the Analytics Platform setup in Picnic. In the “Picnic Analytics Platform: Migration from AWS Kinesis to Confluent Cloud” we described why and how we migrated from AWS Kinesis to Confluent Cloud. Our pipeline captures these events and sends them to Confluent Cloud.

Kafka 52

Kafka Metadata AWS Java 52

Rockset

SEPTEMBER 14, 2021

We’re introducing a new Rockset Integration for Apache Kafka that offers native support for Confluent Cloud and Apache Kafka, making it simpler and faster to ingest streaming data for real-time analytics. We’ve been able to go from raw event streams to a fast SQL query in 5 minutes ?.

Kafka 52

Kafka SQL MongoDB Computer Science 52

Lyft Engineering

NOVEMBER 29, 2023

In this particular blog post, we explain how Druid has been used at Lyft and what led us to adopt ClickHouse for our sub-second analytic system. For new platform engineers, it presented a steeper learning curve. For example: How many rides in the last 2 hours in the SF region?) An example of how we use Druid rollup at Lyft.

Kafka 104

Kafka Data Ingestion Datasets Architecture 104

Rockset

AUGUST 28, 2023

Here we track every click to every product in our webshop, and then ingest this data into Rockset via Confluent Cloud. Use case 2: Creating materialized views or development datasets Rockset supports automatic real-time rollups on ingestion for some data sources. Let’s now configure the schedule for our query lambda.

AWS 52

AWS SQL Datasets Metadata 52

ProjectPro

AUGUST 24, 2021

And, out of these professions, this blog will discuss the data engineering job role. Source Code: Smart IoT Infrastructure Data Engineering Project with Source Code 2) Aviation Data Analysis Aviation Data can segment passengers, observe their behavioral patterns, and reach out to them with relevant, targeted promotions.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

Confluent

MAY 29, 2019

In part 1 , we discussed an event streaming architecture that we implemented for a customer using Apache Kafka ® , KSQL from Confluent, and Kafka Streams. Now in part 2, we’ll discuss the challenges we faced developing, building, and deploying the KSQL portion of our application and how we used Gradle to address them. KSQL primer.

Kafka 95

Kafka Management Bytes SQL 95