Remove best-practices dbt-unity-catalog-best-practices
article thumbnail

5 Lessons Learned from Testing Databricks SQL Serverless + DBT

Towards Data Science

We ran a $12K experiment to test the cost and performance of Serverless warehouses and dbt concurrent threads, and obtained unexpected results. So let’s drive into what we explored: Experiment Setup We tried to take a “practical” approach to this study, and simulate what a real company might do when they want to run a SQL warehouse.

SQL 52
article thumbnail

The Future of Big Data Analytics & Data Science: 6 Trends of Tomorrow

Monte Carlo

On the data lake side, Databricks has launched Unity Catalog to help bring more metadata, structure, and governance to data assets. It’s always best, of course, to diagnose a problem rather than treating the symptom. Beyond that, data catalogs can be used to outline how stakeholders can (and can’t) use data.