Remove kafka-listeners-explained
article thumbnail

Deploying Data Pipelines using the Saga pattern

Picnic Engineering

In our previous blog, Dima Kalashnikov explained how we configure our Internal services pipeline in the Analytics Platform. In this post, we will explain how our team automates the creation of new data pipeline deployments. Wrapping up: In this post, we have explained how we automate our pipeline deployments at Picnic.

article thumbnail

Metadata Management And Integration At LinkedIn With DataHub

Data Engineering Podcast

In this episode Mars Lan and Pardhu Gunnam explain how they designed the platform, how it integrates into their data platforms, and how it is being used to power data discovery and analytics at LinkedIn. Closing Announcements Thank you for listening! Don’t forget to check out our other show, Podcast.__init__

Metadata 100
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Kafka Listeners – Explained

Confluent

Put another way, courtesy of Spencer Ruport: LISTENERS are what interfaces Kafka binds to. Is anyone listening? Apache Kafka ® is a distributed system. You need to tell Kafka how the brokers can reach each other but also make sure that external clients (producers/consumers) can reach the broker they need to reach.

Kafka 100
article thumbnail

Fast Analytics On Semi-Structured And Structured Data In The Cloud

Data Engineering Podcast

In this episode CEO Venkat Venkataramani and SVP of Product Shruti Bhat explain the origins of Rockset, how it is architected to allow for fast and flexible SQL analytics on your data, and how their serverless platform can save you the time and effort of implementing portions of your own infrastructure.

article thumbnail

Cloudera DataFlow Designer: The Key to Agile Data Pipeline Development

Cloudera

In our previous DataFlow Designer blog post , we introduced you to the new user interface and highlighted its key capabilities. In this blog post we will put these capabilities in context and dive deeper into how the built-in, end-to-end data flow life cycle enables self-service data pipeline development.

article thumbnail

Change Data Capture: What It Is and How to Use It

Rockset

This post will explain some common CDC implementations and discuss the benefits and drawbacks of using each. The target system simply needs to listen out for changes and apply them instead of constantly polling the source and keeping track of what it's already captured.

IT 40
article thumbnail

The Top 25 Data Engineering Influencers and Content Creators on LinkedIn

Databand.ai

Deepak regularly shares blog content and similar advice on LinkedIn. She also runs dutchengineer.org, which features a blog and newsletter full of tips for landing your dream job in data science, and offers digital courses and one-on-one mentoring for data scientists and data engineers.