Data Engineering Podcast

MAY 22, 2022

Summary Machine learning has become a meaningful target for data applications, bringing with it an increase in the complexity of orchestrating the entire data flow. Flyte is a project that was started at Lyft to address their internal needs for machine learning and integrated closely with Kubernetes as the execution manager. In this episode Ketan Umare and Haytham Abuelfutuh share the story of the Flyte project and how their work at Union is focused on supporting and scaling the code and communi

Machine Learning 130

Machine Learning Data Engineering Data Engineer Cloud 130

dbt Developer Hub

MAY 22, 2022

We’ve likely been here: Table A has 56 columns and we want to select all but one of them ( column_56 ). So here we go, let’s get started… select column_1 , column_2 , column_3 , please_save_me… from {{ ref ( 'table_a' ) }} At this point, you realize your will to continue typing out the next 52 columns has essentially dwindled down to nothing and you’re probably questioning the life choices that led you here.

Utilities 52

Utilities SQL Coding Project 52

Data Engineering Podcast

MAY 22, 2022

Summary The interfaces and design cues that a tool offers can have a massive impact on who is able to use it and the tasks that they are able to perform. With an eye to making data workflows more accessible to everyone in an organization Raj Bains and his team at Prophecy designed a powerful and extensible low-code platform that lets technical and non-technical users scale data flows without forcing everyone into the same layers of abstraction.

Scala 100

Scala SQL Data Data Workflow 100