Data Engineering Weekly

FEBRUARY 4, 2024

The user journey, sales process, marketing campaign, everything falls under a state machine. Data modeling is a collaborative process across business units to capture state changes in business activity. The solution centered around Notebook opens a Flink Session for the Kafka stream and continues the exploration.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

ProjectPro

FEBRUARY 6, 2023

With over 10 million active subscriptions, 50 million active topics, and a trillion messages processed per day, Google Cloud Pub/Sub makes it easy to build and manage complex event-driven systems. Instead of using synchronous remote procedure calls (RPCs), publishers connect with subscribers asynchronously by streaming events.

Google Cloud 52

Google Cloud Cloud Cloud Storage Data Ingestion 52

Big Data Tools

JULY 13, 2022

The process of returning to active maintenance is not even described in the docs. There are also multiple improvements for streaming support (for Kafka and Kinesis ), along with many other changes. In a nutshell, InLong is a SaaS-based streaming platform that scales. However, a miracle happened!

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Big Data Tools

JULY 13, 2022

The process of returning to active maintenance is not even described in the docs. There are also multiple improvements for streaming support (for Kafka and Kinesis ), along with many other changes. In a nutshell, InLong is a SaaS-based streaming platform that scales. However, a miracle happened!

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

ProjectPro

JULY 23, 2021

This blog presents some of the most unique and innovative AWS projects from beginner to advanced levels. Serverless Web App Intermediate Level AWS Projects 6. Real-time Data Processing Application 7. Serverless Image Recognition Engine 13. AWS Lambda is another foundation service for serverless computing.

AWS 52

AWS Project Amazon Web Services Cloud Computing 52

Jesse Anderson

OCTOBER 5, 2023

Confluent had two moats (replication and Confluent Cloud), and now they anticipate three moats (replication, Confluent Cloud, serverless Flink). Everything should be streaming – No, it shouldn’t. There should be a clear and compelling need for streaming to deal with its various downsides.

Kafka 195

Kafka Finance Cloud Designing 195

Rockset

MARCH 18, 2024

Today, Confluent announced the general availability of its serverless Apache Flink service. Flink is one of the most popular stream processing technologies, ranked as a top five Apache project and backed by a diverse committer community including Alibaba and Apple. What are the challenges building RAG pipelines?

Cloud 64

Cloud Building Metadata Kafka 64