dbt Developer Hub

AUGUST 17, 2022

When running a job that has over 1,700 models, how do you know what a “good” runtime is? While there are many possible answers depending on dataset size, complexity of modeling, and historical run times, the crux of the matter is normally “did you hit your SLAs”? The model fct_dbt_invocations takes, on average, 1.5 hours to run.

Data Warehouse 59

Data Warehouse Datasets Cloud Coding 59

DoorDash Engineering

MARCH 7, 2023

While most engineering tooling at DoorDash is focused on making safe incremental improvements to existing systems, in part by testing in production (learn more about our end-to-end testing strategy ), this is not always the best approach when launching an entirely new business line.

AWS 119

AWS PostgreSQL Database SQL 119

dbt Developer Hub

NOVEMBER 28, 2021

My models process the data so that it's easy to perform analysis and spot trends. Here are the tools I chose to use: Google Bigquery acts as the main database, holding all the source data, intermediate models, and data marts. That's why I built a mini-warehouse for studying community growth.

Raw Data 52

Raw Data Metadata Datasets Database 52

Data Engineering Podcast

JANUARY 28, 2018

Happening April 29th to the 30th in New York it will give you a solid understanding of the latest breakthroughs and best practices in AI for business. In New York, it will give you a solid understanding of the latest breakthroughs and best practices in AI for business. To the 30th.

Data 100

Data Project Electronics Data Management 100

ProjectPro

NOVEMBER 30, 2021

This guide provides definitions, a step-by-step tutorial, and a few best practices to help you understand ETL pipelines and how they differ from data pipelines. When working on real-time business problems, data scientists build models using various Machine Learning or Deep Learning algorithms.

Process 52

Process Data Pipeline Data Warehouse AWS 52

phData: Data Engineering

JULY 18, 2023

Business-Focused Operation Model: Teams can shed countless hours of managing long-running and complex ETL pipelines that do not scale. Transparent Pricing Model: Say goodbye to tedious cost adjustments for hardware, software, platform maintenance, upgrade costs, etc. Why Migrate to a Modern Data Stack?

Data Warehouse 52

Data Warehouse Pipeline-centric Government Data 52