Monte Carlo

APRIL 24, 2023

If streaming data is a priority for your platform, you might also choose to leverage a system like Confluent’s Apache Kafka along with some of the above mentioned technologies. And data orchestration tools are generally easy to stand-up for initial use-cases. Missed Nishith’s 5 considerations?

Data Pipeline 52

Data Pipeline Building Data Ingestion BI 52

DataKitchen

JUNE 19, 2023

The Data Journey, in this sense, provides transparency about the status and outcomes of individual tasks, offers insights into potential bottlenecks or inefficiencies in the sequence of operations, and helps ensure that scheduled tasks are executed as planned. Consider a data pipeline orchestrated by Airflow.

Data 52

Data Data Validation Utilities High Quality Data 52

Monte Carlo

JANUARY 25, 2023

There is, however, an added dimension to this relationship: data producers are often consumers of upstream data sources. Data warehouse producers wear both hats working with upstream producers so they can consume high-quality data and producing high-quality data to provide to their consumers.

Data Warehouse 52

Data Warehouse Data High Quality Data Metadata 52