Data Engineering Weekly

SEPTEMBER 29, 2024

Airbnb: Sandcastle - data/AI apps for everyone Product ideas powered by data and AI must go through rapid iteration on shareable, lightweight live prototypes instead of static proposals. However, hosting an internal application for fast prototyping is always a challenging platform to build and maintain. Airbnb writes about Sandcastle, an Airbnb-internal prototyping platform that enables data scientists, engineers, and product managers to bring data/AI ideas to life.

Data Engineering 62

Data Engineering Data Engineer Engineering SQL 62

dbt Developer Hub

SEPTEMBER 29, 2024

One of the most important things that dbt does is unlock the ability for teams to collaborate on creating and disseminating organizational knowledge. In the past, this primarily looked like a team working in one dbt Project to create a set of transformed objects in their data platform. As dbt was adopted by larger organizations and began to drive workloads at a global scale, it became clear that we needed mechanisms to allow teams to operate independently from each other, creating and sharing da

Metadata 52

Metadata Cloud Project Accessible 52

Hevo

SEPTEMBER 29, 2024

Have you ever opened the billing section of a BigQuery account and got a shocking surprise? You are not alone. BigQuery is a powerful tool, but this power does not come for free all the time. It can quickly deplete your budget if you do not practice good cost management.

Management 52

Management IT Data 52

Hevo

SEPTEMBER 29, 2024

Today most organizations are of the opinion that public APIs should be tapped into and useful information extracted there from. The same, however triggers a sound ETL solution to handle the data correctly.

ETL Tools 52

ETL Tools Data Integration Data 52

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Hevo

SEPTEMBER 29, 2024

If you are a data-driven business, then you must know how crucial it is to extract meaningful insights from your data. That’s where Reverse ETL comes into play. I’m guessing you might know what ETL (Extract, Transform, Load) is. It is the process of bringing data into your warehouses.

ETL Tools 52

ETL Tools Process Data IT 52