Confluent

DECEMBER 7, 2022

Learn how to build a real-time streaming application using Apache Kafka® and.NET producer and consumer clients.

Building 57

Building Kafka Process 57

Rockset

OCTOBER 21, 2022

It’s part of building a stellar customer experience, where customers want to keep coming back, and you start building customer champions. Customer experience is part of the journey of building loyal customers. When building a real-time customer 360 app, you’ll definitely need event data from a streaming data source, like Kafka.

Kafka 52

Kafka Building SQL Database 52

Confluent

DECEMBER 2, 2022

With over 4,700 stores, learn how Walmart used Kafka to build an event-driven architecture for real-time inventory management, providing a seamless omnichannel experience.

Kafka 116

Kafka Systems Architecture Building 116

Confluent

NOVEMBER 25, 2020

As part of this year’s Kafka Summit, the Confluent Community team hosted a community hackathon named Kafkathon 2020. The event provided an opportunity for participants to learn, build, and showcase […].

Kafka 104

Kafka Building 104

Cloudera

SEPTEMBER 26, 2023

These solutions include on-premises and public cloud offerings with the data management, security and governance needed to build an enterprise data and analytic big data solution. The post IBM Technology Chooses Cloudera as its Preferred Partner for Addressing Real Time Data Movement Using Kafka appeared first on Cloudera Blog.

Kafka 91

Kafka Technology IT Government 91

Confluent

MARCH 11, 2020

In the Apache Kafka® ecosystem, ksqlDB and Kafka Streams are two popular tools for building event streaming applications that are tightly integrated with Apache Kafka. While ksqlDB and Kafka Streams […].

Kafka 112

Kafka Process Building 112

Lyft Engineering

JUNE 28, 2023

On the flip side, there was a substantial appetite to build real-time ML systems from developers at Lyft. The Event Driven Decisions capability in particular turned out to be general enough as to be applicable to a wide range of use cases. One key component is the Analytics Event Abstraction layer.

Machine Learning 115

Machine Learning Building Metadata Kafka 115

Data Engineering Podcast

MARCH 2, 2020

Summary Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing.

Kafka 100

Kafka Process PostgreSQL MySQL 100

ProjectPro

FEBRUARY 8, 2023

Did you know thousands of businesses, including over 80% of the Fortune 100, use Apache Kafka to modernize their data strategies? Apache Kafka is the most widely used open-source stream-processing solution for gathering, processing, storing, and analyzing large amounts of data. What is Apache Kafka Used For?

Kafka 52

Kafka Banking Medical Healthcare 52

Rockset

MARCH 5, 2020

Apache Kafka has seen broad adoption as the streaming platform of choice for building applications that react to streams of data in real time. In many organizations, Kafka is the foundational platform for real-time event analytics, acting as a central location for collecting event data and making it available in real time.

Kafka 40

Kafka Data Warehouse Data Lake Relational Database 40

Confluent

MAY 23, 2019

Let us cut to the chase: Kafka Summit London session videos are available! This year’s London event was almost double the size of last year, so I’m especially happy to see these numbers on the page: 2 days of Apache Kafka ®. World-class Kafka keynotes. And if you weren’t there? Well, dig in and start learning! 3 keynotes.

Kafka 109

Kafka Cloud Database Engineering 109

Rockset

AUGUST 16, 2022

Streams for Everyone If you have come this far it means you have already considered or are considering using event streaming in your data architecture for the wide variety of benefits it can offer. In either case, both Amazon Kinesis and Apache Kafka can help but which one is the right fit for you and your goals.

Kafka 52

Kafka AWS Cloud Java 52

Afterpay Tech

SEPTEMBER 6, 2022

Photo by Leon S on Unsplash By: Jing Li Summary This article articulates the challenges, innovation and success of the Kafka implementation in Afterpay’s Global Payments Platform in the PCI zone. Context The asynchronous processing capability that Kafka offers opens up numerous innovation opportunities to interact with other services.

Kafka 52

Kafka AWS Metadata Data Warehouse 52

Confluent

MARCH 26, 2019

Apache-Kafka ® -based applications stand out for their ability to decouple producers and consumers using an event log as an intermediate layer. This enables choreographed service collaborations, where many components can subscribe to events stored in the event log and react to them asynchronously.

Kafka 111

Kafka Transportation Metadata Consulting 111

Confluent

MAY 13, 2019

In the last year, we’ve experienced enormous growth on Confluent Cloud, our fully managed Apache Kafka ® service. As Confluent Cloud has grown, we’ve noticed two gaps that very clearly remain to be filled in managed Apache Kafka services. Five seconds to Kafka (or, never make another cluster again!).

Kafka 100

Kafka Cloud Management Building 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Building reliable data pipelines is a complex and costly undertaking with many layered requirements. In order to reduce the amount of time and effort required to build pipelines that power critical insights Manish Jethani co-founded Hevo Data. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

JULY 27, 2021

When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. How do you handle migrating existing projects, particularly if they are using Kafka currently?

Building 100

Building Management Kafka Data Warehouse 100

Confluent

JUNE 27, 2019

Confluent’s clients for Apache Kafka ® recently passed a major milestone—the release of version 1.0. Magnus Edenhill first started developing librdkafka about seven years ago, later joining Confluent in the very early days to help foster the community of Kafka users outside the Java ecosystem. Build()) {. }. In the v0.11.6

Kafka 106

Kafka Designing Java Coding 106

Data Engineering Podcast

JULY 1, 2019

Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he could push Kafka and Cassandra for this use case. Coming up this fall is the combined events of Graphorum and the Data Architecture Summit.

Kafka 100

Kafka Finance Media Architecture 100

Rockset

DECEMBER 14, 2022

Rockset’s native connector for Amazon Managed Streaming for Apache Kafka (MSK) makes it simpler and faster to ingest streaming data for real-time analytics. Amazon MSK is a fully managed AWS service that gives users the ability to build and run applications using Apache Kafka.

Kafka 52

Kafka MongoDB SQL AWS 52

Confluent

APRIL 23, 2019

When it was first created, Apache Kafka ® had a client API for just Scala and Java. Since then, the Kafka client API has been developed for many other programming languages which enables you to pick the language you want. At Confluent, we have an engineering team dedicated to the development of these Kafka clients.

Programming Language 101

Programming Language Kafka Programming Scala 101

Confluent

FEBRUARY 28, 2019

Only a little more than one month after the first release, we are happy to announce another milestone for our Kafka integration. Today, you can grab the Kafka Connect Neo4j Sink from Confluent Hub. . Neo4j extension – Kafka sink refresher. Then it is really up to you what you want to with the event data.

Kafka 86

Kafka Java Programming Language Data Pipeline 86

Data Engineering Weekly

MAY 16, 2023

Since Kafka is almost synonymous with real-time data processing, we often call this a “Fronting Kafka” pattern. The Fronting Kafka pattern follows a two-cluster approach. Like the staging environment, Fronting Kafka receives all the events without validation. Should We Build a New Tool?

Engineering 110

Engineering Kafka Data Pipeline Data Warehouse 110

Rockset

JULY 28, 2022

CDC enables true real-time analytics on your application data, assuming the platform you send the data to can consume the events in real time. Options For Change Data Capture on MongoDB Apache Kafka The native CDC architecture for capturing change events in MongoDB uses Apache Kafka.

MongoDB 52

MongoDB Kafka NoSQL Data Lake 52

Data Engineering Podcast

MAY 15, 2022

Srivatsan Sridharan has had the opportunity to design, build, and run data lake platforms for both Yelp and Robinhood, with many valuable lessons learned from each experience. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Building BI Architecture 100

Data Engineering Podcast

OCTOBER 28, 2019

In an effort to create a better abstraction for building data applications Nick Schrock created Dagster. In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and his vision for the future of data programming.

Building 100

Building Data Pipeline Programming Language Metadata 100

Confluent

SEPTEMBER 25, 2020

Twitter, one of the most popular social media platforms today, is well known for its ever-changing environment—user behaviors evolve quickly; trends are dynamic and versatile; and special and emergent events […].

Machine Learning 95

Machine Learning Kafka Building Media 95

Netflix Tech

MARCH 10, 2023

This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts. We build the data pipeline to persist the assets data in the iceberg in parallel with cassandra and elasticsearch DB.

Management 94

Management Kafka Metadata Media 94

Rockset

SEPTEMBER 14, 2021

We’re introducing a new Rockset Integration for Apache Kafka that offers native support for Confluent Cloud and Apache Kafka, making it simpler and faster to ingest streaming data for real-time analytics. With the Kafka Integration, users no longer need to build, deploy or operate any infrastructure component on the Kafka side.

Kafka 52

Kafka SQL MongoDB Computer Science 52

Confluent

APRIL 10, 2019

Instead of having many point-to-point connections between sites, the Confluent Platform provides an integrated event streaming architecture with frictionless data replication between sites. Useful to know in case of a disaster event). How far behind is data replication?

Kafka 84

Kafka Metadata Java Cloud 84

Data Engineering Podcast

DECEMBER 30, 2019

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Process 100

Process Building Hadoop Java 100

Netflix Tech

JUNE 1, 2023

To do this, we devised a novel way to simulate the projected traffic weeks ahead of launch by building upon the traffic migration framework described here. New content or national events may drive brief spikes, but, by and large, traffic is usually smoothly increasing or decreasing.

Algorithm 136

Algorithm Metadata Kafka Systems 136

Netflix Tech

AUGUST 30, 2021

In the Device Management Platform, this is achieved by having device updates be event-sourced through the control plane to the cloud so that NTS will always have the most up-to-date information about the devices available for testing. The RAE is configured to be effectively a router that devices under test (DUTs) are connected to.

Management 85

Management Kafka Transportation Cloud 85

Pipeline Data Engineering

FEBRUARY 18, 2022

furthermore - and this might be the most frequent setup - some people come without any clue about what they really wan’t to build. some come with a data stack problem that they are currently facing at work, and they need professional guidance and clarity about how to go about solving it. And that’s totally fine.

Building 52

Building Portfolio Kafka Machine Learning 52

Afterpay Tech

NOVEMBER 23, 2020

What did we set out to build? Motivated by this and other similar findings, we decided to build a completely new search platform to power the Shop Directory. When a consumer completes an order in a store website, Afterpay Checkout Service sends an order completion event to Kafka. The journey is only 1% complete.

Engineering 52

Engineering Building Kafka Retail 52

Data Engineering Weekly

APRIL 21, 2024

Grab narrates how it integrated Debeizium, Kafka, and Apache Hudi to enable near real-time data analytics on the data lake. link] GoodData: Building a Modern Data Service Layer with Apache Arrow GoodData writes about using Apache Arrow to build an efficient service layer.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Kafka 245

Kafka Data Lake High Quality Data SQL 245