project-use-case etl-pipeline-with-snowflake-dbt-airflow-example 
Data Engineering Podcast
FEBRUARY 8, 2021
In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify.
Data Engineering Podcast
MARCH 22, 2021
In this episode Raghu Murthy, founder and CEO of Datacoral, does a deep dive on how he and his team manage change data capture pipelines in production. Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code. RudderStack’s smart customer data pipeline is warehouse-first.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
APRIL 19, 2021
Summary Most of the time when you think about a data pipeline or ETL job what comes to mind is a purely mechanistic progression of functions that move data from point A to point B. Sometimes, however, one of those transformations is actually a full-fledged machine learning project in its own right.
Data Engineering Weekly
MARCH 19, 2023
link] Uber: Setting Uber’s Transactional Data Lake in Motion with Incremental ETL Using Apache Hudi Uber writes a comprehensive guide on running incremental ETL using Apache Hudi. link] Open Questions on Type-2 modeling I keep thinking about the Type-2 SCD and the complexity of the data pipeline.
dbt Developer Hub
NOVEMBER 28, 2021
Airflow and dbt are often framed as either / or: You either build SQL transformations using Airflow’s SQL database operators (like SnowflakeOperator ), or develop them in a dbt project. You either orchestrate dbt models in Airflow, or you deploy them using dbt Cloud.
Monte Carlo
APRIL 24, 2023
Sometimes the build versus buy debate can include layers of its own, with decisions to buy often evolving into a decision to use open-source tooling to retain flexibility or leverage fully-managed solutions for faster time-to-value. Missed Nishith’s 5 considerations? Check out Part 1 of the build vs buy guide to catch up.
Data Engineering Podcast
MARCH 15, 2021
Summary The team at DoorDash has a complex set of optimization challenges to deal with using data that they collect from a multi-sided marketplace. Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code. RudderStack’s smart customer data pipeline is warehouse-first.
Expert insights. Personalized for you.
100 
Let's personalize your content