Data Engineering Podcast

MARCH 2, 2020

Summary Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing. Developers can work with the SQL constructs that they are familiar with while automatically getting the durability and reliability that Kafka offers.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Confluent

MARCH 4, 2020

The Elasticsearch sink connector helps you integrate Apache Kafka® and Elasticsearch with minimum effort. You can take data you’ve stored in Kafka and stream it into Elasticsearch to then be […].

Kafka 119

Kafka IT Data 119

KDnuggets

MARCH 6, 2020

Fines from the GDPR have been rolling in since its inception in 2018. This article investigates who are the largest penalty recipients by country, the amounts, and private individuals.

IT 109

IT 109

Netflix Tech

MARCH 5, 2020

By Kevin Glisson, Marc Vilanova, Forest Monsen Netflix is pleased to announce the open-source release of our crisis management orchestration framework: Dispatch! Okay, but what is Dispatch? Put simply, Dispatch is: All of the ad-hoc things you’re doing to manage incidents today, done for you, and a bunch of other things you should’ve been doing, but have not had the time!

Metadata 81

Metadata AWS Management Architecture 81

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Teradata

MARCH 4, 2020

Many Teradata customers are interested in integrating Vantage with Microsoft Azure First Party Services. Check out this guide to help you get started.

52

52

Confluent

MARCH 3, 2020

When building API-driven web applications, there is one key metric that engineering teams should minimize: the blocked factor. The blocked factor measures how much time developers spend in the following […].

Engineering 95

Engineering Building 95

Netflix Tech

MARCH 5, 2020

Netflix has a program in our Information Security department for quantifying the risk of deliberate (attacker-driven) and accidental… Continue reading on Netflix TechBlog ».

Programming 71

Programming 71

Teradata

MARCH 2, 2020

Learn how Teradata's hashing algorithm is used to enhance the performance and ease-of-use of the Advanced SQL Engine.

Database 52

Database Algorithm SQL Engineering 52

Rockset

MARCH 5, 2020

Apache Kafka has seen broad adoption as the streaming platform of choice for building applications that react to streams of data in real time. In many organizations, Kafka is the foundational platform for real-time event analytics, acting as a central location for collecting event data and making it available in real time. While Kafka has become the standard for event streaming, we often need to analyze and build useful applications on Kafka data to unlock the most value from event streams.

Kafka 40

Kafka Data Warehouse Data Lake Relational Database 40

Airbnb Tech

MARCH 3, 2020

One of the most common ways to store results from a Spark job is by writing the results to a Hive table stored on HDFS. While in theory, managing the output file count from your jobs should be simple, in reality, it can be one of the more complex parts of your pipeline. Author : Zachary Ennenga Airbnb’s new office building, 650 Townsend Background At Airbnb, our offline data processing ecosystem contains many mission-critical, time-sensitive jobs — it is essential for us to maximize the stabilit

Datasets 52

Datasets Bytes Scala Data Engineering 52

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

Netflix Tech

MARCH 3, 2020

By Ben Sykes Continue reading on Netflix TechBlog ».

Kafka 98

Kafka 98