How we think about Data Pipelines is changing
The goal is to reliably and efficiently release data into production
Published in
6 min readNov 8, 2023
Data Pipelines are series of tasks organised in a directed acyclic graph or “DAG”. Historically, these are run on open-source workflow orchestration packages like Airflow or Prefect, and require infrastructure managed by data engineers or platform teams. These data…
