Christophe Blefari

JULY 8, 2023

Aujourd'hui il s'occupe du produit chez Kestra, un orchestrateur open-source développé en France. The new dbt Semantic Layer Following the acquisition of Transform by dbt Labs a few months ago, dbt Core integrates MetricsFlow. In a dbt model, you can define primary and foreign entities.

Kafka 130

Kafka PostgreSQL Transportation Data 130

Data Engineering Podcast

OCTOBER 15, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. When is Decodable the wrong choice?

Process 182

Process Building SQL BI 182

Christophe Blefari

APRIL 21, 2023

Analytics engineering future This week Tristan Handy—dbt Labs CEO—wrote a post about the future of analytics engineering: The next big step forwards for analytics engineering. As introduction Tristan gives the original vision of dbt that became mainstream, today. In dbt Core 1.5 CoreDB raises $6.5m seed round.

Raw Data 130

Raw Data Data Datasets SQL 130

Christophe Blefari

MARCH 4, 2023

This week I've published an compact article about how to get started with dbt. The idea behind this article is to define every dbt concept and objects from the CLI to the Jinja templating or models and sources. Rad my article — How to get started with dbt Machine Learning Saturday 🤖 Was it a boost ride?

Machine Learning 130

Machine Learning AWS Data Data Lake 130

Christophe Blefari

APRIL 8, 2023

I've tried to run open-source models locally on my own laptop. The only normalisation I did was back at the engineering school while learning SQL with Normal Forms. Which is easier to conceptually understand but also to use in machine learning. When it comes to modeling it's hard not to mention dbt.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Christophe Blefari

APRIL 8, 2023

I've tried to run open-source models locally on my own laptop. The only normalisation I did was back at the engineering school while learning SQL with Normal Forms. Which is easier to conceptually understand but also to use in machine learning. When it comes to modeling it's hard not to mention dbt.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Data Engineering Podcast

OCTOBER 8, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate.

Insurance 162

Insurance BI SQL Machine Learning 162

Towards Data Science

OCTOBER 25, 2023

This approach was effective for small organizations and startups with limited data sources and use cases. However, as organizations prioritized being data-driven, there was an increased need for more machine learning, analytics, and business intelligence data use cases. Your organization may be undergoing the decentralization of data.

Data Pipeline 82

Data Pipeline SQL Data Architecture Data 82

Data Engineering Podcast

OCTOBER 22, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. What are the most interesting, unexpected, or challenging lessons that you have learned while working on defining and implementing data product strategies? When is a data product strategy overkill?

BI 162

BI SQL Machine Learning Programming Language 162

Data Engineering Podcast

OCTOBER 1, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. When is AI the wrong choice for ETL applications?

Building 162

Building BI SQL Machine Learning 162

Christophe Blefari

JUNE 16, 2023

Yann's vision goes toward AI systems learning and reasoning like animals and humans. Deep multi-task learning and real-time personalisation for closeup recommendations — Pinterest still doing deep learning. Mainly he unit tests macros (the logic) with his framework and test data with soda and dbt contracts.

Programming Language 130

Programming Language SQL PostgreSQL Data 130

Christophe Blefari

SEPTEMBER 15, 2023

Gen) AI News 🤖 Reinforcement Learning: an easy introduction to value iteration — Title says easy, but the article contains maths formula. Dimensional data modeling with dbt — A great 6-steps process to create a simple dim-fact model with dbt. It also works well for machine learning use-cases.

Data Warehouse 130

Data Warehouse Data SQL Python 130

Data Engineering Podcast

AUGUST 27, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data.

Database 130

Database Building PostgreSQL BI 130

Data Engineering Podcast

OCTOBER 30, 2022

Summary One of the most impactful technologies for data analytics in recent years has been dbt. To help simplify the adoption and management of dbt projects Nandam Karthik helped create Optimus. To help simplify the adoption and management of dbt projects Nandam Karthik helped create Optimus.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Weekly

MAY 12, 2024

Gloss Genius: How We Migrated From dbt Cloud and Scaled Our Data Development Gloss Genius describes its migration journey from dbt cloud to Airflow + custom Github actions. The post highlights how Gloss Genuis established feature parity with SlimCI and enhanced post-migration with open-source tooling.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Christophe Blefari

MARCH 17, 2023

Glaze, protecting artists from style mimicry — A tool developed by researchers at Chicago university will help digital artists by cloaking their art to avoid mimicry from deep learning trainings. Tracking the fake GitHub star black market with Dagster, dbt and BigQuery — Things are getting spicy here. dbt Reimagined.

Data 130

Data SQL Deep Learning Kafka 130

Data Engineering Podcast

AUGUST 20, 2023

Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. When is Illumidesk the wrong choice?

Education 130

Education BI SQL Database 130

Towards Data Science

AUGUST 19, 2023

By asking for data analytics projects, I learned how to use dbt and further developed my Python skills. Rather than on job title, I started to focus on roles that had a mix of skills that I already had and those that I wanted to learn. dbt is a SQL-based tool, so if you already know SQL, it is fairly easy to learn.

Data Engineering 83

Data Engineering Data Engineer Engineering SQL 83

Towards Data Science

JULY 10, 2023

A simple framework for building dbt models that actually get used. When I was researching the Ultimate Guide to dbt , I was shocked by the lack of material around actually building models from scratch. This article is the result of research and experiments into how best to design and implement a dbt model.

Designing 98

Designing Electronics Engineering Coding 98

Data Engineering Podcast

APRIL 2, 2023

Join The RudderStack Transformation Challenge today for a chance to win a $1,000 cash prize just by submitting a Transformation to the open-source RudderStack Transformation library. What are the most interesting, unexpected, or challenging lessons that you have learned while working on the MAD landscape over the years?

Hadoop 130

Hadoop Machine Learning Python Architecture 130

Workfall

JULY 18, 2023

But then, a game-changer emerged – DBT (Data Build Tool). With DBT’s materializations, our data transformations underwent a magical transformation themselves. In this blog, we’ll whisk you away on an enchanting journey through DBT materializations. In this blog, we will cover: What is DBT? DBT , the Data Build Tool.

Datasets 77

Datasets Entertainment Data Workflow Data 77

Data Engineering Weekly

APRIL 7, 2024

dbt: 2024 State of Analytics Engineering The 2024 dbt’s state of analytical engineering report is out. link] Matt Turck: Full Steam Ahead: The 2024 MAD (Machine Learning, AI & Data) Landscape Coninue the week of insights into the world of data & AI landscape, the 2024 MAD landscape is out.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Data Engineering Podcast

JULY 9, 2023

dbt vs. informatica vs. ETL scripts, etc.) What are some examples of data sources or problem domains for which this approach is well suited? What are the most interesting, unexpected, or challenging lessons that you have learned while working on ECM? dbt vs. informatica vs. ETL scripts, etc.) When is ECM the wrong choice?

Database-centric 130

Database-centric Machine Learning SQL Data Engineering 130

Christophe Blefari

SEPTEMBER 28, 2023

A lot of companies started with Postgres + dbt + Metabase, don't be ashamed. Your data stack won't explode if you don't use dbt. 3 lessons to learn before creating your own data team — Christelle wrote 3 lessons learned about a survey that has been run in a private French data community.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Data Engineering Podcast

JULY 17, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. What are the categories of information that organizations use external data sources for?

Data Management 130

Data Management Management Metadata MongoDB 130

Workfall

JULY 4, 2023

Reading Time: 8 minutes In the world of data engineering, a mighty tool called DBT (Data Build Tool) comes to the rescue of modern data workflows. With DBT, they weave powerful SQL spells to create data models that capture the essence of their organization’s information. In DBT, transformations are like these artisans.

Data Warehouse 69

Data Warehouse SQL PostgreSQL Database 69

Data Engineering Podcast

OCTOBER 22, 2021

One of the driving forces for that change has been the rise of analytics engineering powered by dbt. Lightdash has fully embraced that shift by building an entire open source business intelligence framework that is powered by dbt models. What was your motivation for releasing Lightdash as open source?

Engineering 100

Engineering Business Intelligence BI Data Warehouse 100

Marc Lamberti

SEPTEMBER 24, 2023

As a best practice, you should group tasks with a common attribute (same source/actions etc.) Suppose you have two “paths” with the same steps to process and store data from two sources. You might want a DAG like this: You can see in the DAG above that the two task groups A and B are in the taskgroup sources.

Coding 130

Coding Python Process Data Pipeline 130

Workfall

AUGUST 14, 2023

These pipelines ensure that data flows smoothly from various sources to its intended destinations, enabling businesses to make informed decisions and gain valuable insights. Two powerful tools that have emerged to simplify the management of data pipelines are DBT (Data Build Tool) and Airflow. What is Airflow? What is Airflow?

Data Pipeline 62

Data Pipeline Raw Data Data Database 62

Data Engineering Weekly

MARCH 3, 2024

Visit rudderstack.com to learn more. link] Zendesk: dbt at Zendesk The Zendesk team shares their journey of migrating legacy data pipelines to dbt, focusing on making them more reliable, efficient, and scalable. There will be food, networking, and real-world talks around data engineering.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Christophe Blefari

JULY 15, 2023

They use Fivetran, dbt and Census. Introduction to dimensionality reduction — I've gave up on machine learning a few years ago, so I really like every article explaining with visual machine learning concepts. Make Python free-threading — This is how open-source is made.

Datasets 130

Datasets Python Machine Learning Data 130

Data Engineering Podcast

FEBRUARY 8, 2021

In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify. This is a great conversation about the lessons learned from real world use of a specific technology and how well it lives up to its promises. What kinds of data sources are you working with?

Data Warehouse 100

Data Warehouse Building BI SQL 100

Data Engineering Podcast

JUNE 19, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

NOVEMBER 6, 2022

Entity resolution and fuzzy matching are powerful utilities for cleaning up data from disconnected sources, but it has typically required custom development and training machine learning models. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Weekly

AUGUST 6, 2023

the best way to run dbt Core in Airflow. The blog narrates how to schedule dbt jobs in Airflow by parsing dbt's manifest.json file, and auto construct Airflow tasks. At Data Engineering Weekly, I wrote an article 15 months back narrating how dbt is not a separate model but part of the larger orchestration system.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Towards Data Science

APRIL 5, 2023

A Guide to Maximize Your dbt Productivity in Visual Studio Code (Image from Unsplash ) If you are struggling to get VS Code and dbt to work well together, you are not alone. or higher and a virtual environment with dbt installed. A dbt project. This article does not go into detail on how to set up a dbt project.

Coding 79

Coding SQL Python Data Science 79

dbt Developer Hub

JULY 2, 2023

Have you experienced a data warehousing project with 50+ data sources, with 25+ data developers working on the same data platform, or data spanning 5+ years with two or more generations of source systems? Just find another water source and fill up the reservoir. So why Data Vault? A new development in the city? No problem!

Cloud 52

Cloud Data Warehouse Data BI 52