Data Engineering Podcast

OCTOBER 15, 2021

Summary The binding element of all data work is the metadata graph that is generated by all of the workflows that produce the assets used by teams across the organization. The DataHub project was created as a way to bring order to the scale of LinkedIn’s data needs. How is the governance of DataHub being managed?

Metadata 100

Metadata BI Data Warehouse Government 100

François Nguyen

MARCH 22, 2021

TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. How do we build data products ? How can we interoperate between the data domains ? Data As Code is a very strong choice : we do not want any UI because it is an heritage of the ETL period.

Architecture 130

Architecture Technology Google Cloud Metadata 130

Data Engineering Podcast

APRIL 5, 2021

It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more. Their SDKs and plugins make event streaming easy, and their integrations with cloud applications like Salesforce and ZenDesk help you go beyond event streaming.

Data Warehouse 100

Data Warehouse Data Pipeline BI Metadata 100

Cloudera

AUGUST 21, 2020

For example, if a credit card was used in the United States and shortly afterward the same card was used in Spain to withdraw the same amount, these two events in isolation could appear legitimate. However, in the context of time and geography, these two events point to a pattern of fraud.

Banking 60

Banking Kafka Cloud Storage Government 60

Monte Carlo

SEPTEMBER 20, 2022

Here’s how Prefect , Series B startup and creator of the popular data orchestration tool, harnessed the power of data observability to preserve headcount, improve data quality and reduce time to detection and resolution for data incidents. Scaling data governance.

Big Data 52

Big Data Data Warehouse Data Data Governance 52

Data Engineering Podcast

NOVEMBER 26, 2023

Developing event-driven pipelines is going to be a lot easier - Meet Functions! Data lakes are notoriously complex. Memphis Logo]([link] Developing event-driven pipelines is going to be a lot easier - Meet Functions! Developing event-driven pipelines is going to be a lot easier - Meet Functions!

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Monte Carlo

JUNE 14, 2023

Airbyte – An open source platform that easily allows you to sync data from applications. Data streaming ingestion solutions include: Apache Kafka – Confluent is the vendor that supports Kafka, the open source event streaming platform to handle streaming analytics and data ingestion.

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52