Data Engineering Podcast

MARCH 24, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Can you describe what the focus of Dagster+ is and the story behind it? Don't forget to check out our other shows.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

JANUARY 28, 2024

In this episode he shares the story of how it got started, how it works, and how you can get started with it. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Scanner is and the story behind it? for the prroduct?

Data Lake 147

Data Lake Building High Quality Data AWS 147

The Pragmatic Engineer

APRIL 21, 2024

I managed our entire Applied Engineering org from its earliest days through the launch and scaling of ChatGPT.    We cover more on Evan’s story in  Inside OpenAI: how does ChatGPT ship so quickly?   For those of us who have not spent the past few years building ChatGPT from the ground up, how does it work?

Engineering 345

Engineering Software Engineer Software Engineering Programming 345

Data Engineering Podcast

JUNE 25, 2023

In this episode Toby Mao explains how it works, the importance of automatic column-level lineage tracking, and how you can start using it today. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

The Pragmatic Engineer

OCTOBER 10, 2023

We don’t frequently have CEOs on The Pragmatic Engineer: in fact, today is the first issue when we’re talking with a CEO – partially – about their CEO job. But when Facebook entered Australia, they pretty much crushed us. In high school, in Australia, Prashant Varanasi was one of my good friends.

Software Engineer 249

Software Engineer Software Engineering Architecture Media 249

Data Engineering Podcast

MAY 5, 2024

This has prompted numerous debates about the possibility of, and timeline for, artificial general intelligence (AGI). Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Links Aigo.ai

Building 147

Building Data Lake High Quality Data Machine Learning 147

The Pragmatic Engineer

MARCH 19, 2024

There’s been a number of news stories about “AI developers,” recently. Press reports say Devin is ready to do the job of developers, but even Cognition AI admits that the tool only solved about 1 in 7 GitHub issues unassisted in tests. Devin is spreading a lot of fear about job security.

Software Engineer 257

Software Engineer Software Engineering Programming Language Media 257

Data Engineering Podcast

MARCH 3, 2024

Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization. Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

FEBRUARY 4, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what RisingWave is and the story behind it? Don't forget to check out our other shows. Email hosts@dataengineeringpodcast.com ) with your story.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

MARCH 17, 2024

In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data. Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino.

Database 147

Database Data Lake High Quality Data Data Workflow 147

Data Engineering Podcast

SEPTEMBER 3, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Can you describe what dlt is and the story behind it? Don't forget to check out our other shows. If you've learned something or tried out a project from the show then tell us about it!

Data Integration 162

Data Integration BI SQL Python 162

Data Engineering Podcast

FEBRUARY 25, 2024

In this episode he explains how he used the combination of Apache Arrow, Flight, Datafusion, and Parquet to lay the foundation of the newest version of his time-series database. Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino.

Database 162

Database Technology Data Lake High Quality Data 162

The Pragmatic Engineer

MAY 1, 2024

There’s been a number of news stories about “AI developers,” recently. Press reports say Devin is ready to do the job of developers, but even Cognition AI admits that the tool only solved about 1 in 7 GitHub issues unassisted in tests. Devin is spreading a lot of fear about job security.

Software Engineer 191

Software Engineer Software Engineering Programming Language Media 191

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Can you describe what Estuary is and the story behind it? Don't forget to check out our other shows.

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

Jesse Anderson

OCTOBER 5, 2022

So in my book, I talk about three key elements, the data, the narrative, and visuals, and how those elements combined to help us tell stories or communicate our insights that we have more effectively to a targeted audience.” Much of a good data story revolves around the narrative. “I We want to be concise.”

Consulting 130

Consulting Data IT 130

Knowledge Hut

MAY 6, 2024

Over the years, I've heard so many different opinions and read so many articles about what people think of certifications. Let us think about getting certified from an Agile Mindset : Agile Team: The candidate is the Customer (Product Owner), Scrum Master & Development Team all in one. How can they improve?

Certification 97

Certification Project Management Technology 97

Data Engineering Podcast

APRIL 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Cloudera

APRIL 11, 2024

To delve deeper into the importance of partnerships, we spoke with Lan about her experience as a senior partner marketing leader at Cloudera. What makes strong partnerships so vital? I’m in lockstep with our partners daily, ensuring we all have the plans to succeed. How do they impact the work happening at Cloudera?

Food 87

Food Data Architecture Architecture Programming 87

Data Engineering Podcast

MARCH 31, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Validate consistency between source and target at any scale, and receive alerts about any discrepancies. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold.

Project 130

Project Data Lake High Quality Data Data Workflow 130

The Pragmatic Engineer

JULY 20, 2023

Today’s Thursday issue is a bit different than usual: we’ll share the story of a startup acquisition that had a happy outcome to showcase the viability of this exit path. At the center of the story is Jean Yang, who interned at Google, Microsoft Research, and Facebook as a software engineer. People loved our fuzzer demos.

Building 201

Building Programming Language Programming Python 201

Data Engineering Podcast

APRIL 14, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Data Engineering Podcast

MAY 7, 2023

In this episode Kevin Niparko and Hanhan Wang share the details of how it is implemented and how you can use it to build and maintain rich customer profiles. Can you describe what Segment Unify is and the story behind it? However, that's also something we're re-thinking with our warehouse-centric strategy.

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Data Engineering Podcast

JULY 9, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Can you describe what entity-centric modeling (ECM) is and the story behind it? feature engineering) What role does the tooling of a team have in the ways that they end up thinking about modeling?

Database-centric 130

Database-centric Machine Learning SQL Data Engineering 130

Data Engineering Podcast

MAY 14, 2023

Summary All of the advancements in our technology is based around the principles of abstraction. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Don't forget to check out our other shows.

Data Lake 147

Data Lake Machine Learning Data Warehouse AWS 147

The Pragmatic Engineer

MARCH 30, 2023

Disclaimer: I worked at Uber, Lyft's US competitor, between 2016-2020. As always, I aim to remain independent in my analysis: I hold no positions in any of the companies mentioned in this article, and have not been paid to write about any of the companies. To get full issues twice a week, subscribe here.

Software Engineer 286

Software Engineer Software Engineering Retail Media 286

Robinhood

MARCH 12, 2024

Robinhood was founded on a simple idea: that our financial markets should be accessible to all. With customers at the heart of our decisions, Robinhood is lowering barriers and providing greater access to financial information and investing. Tell us about it! Tell us more about that mission.

Finance 71

Finance Programming Accessible Accessibility 71

Data Engineering Podcast

DECEMBER 24, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. What motivated to write a book about how to manage Kafka in production? Don't forget to check out our other shows. Email hosts@dataengineeringpodcast.com ) with your story.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Engineering at Meta

SEPTEMBER 7, 2023

The story of its development and initial launch is the story of Meta’s ability to scale with speed and impact. Instagram uses Python (Django) for its backend. The Threads launch With everything moving ahead at full speed toward a July launch, our team saw and took an opportunity to release the app early.

Media 143

Media Engineering Coding Python 143

Data Engineering Podcast

APRIL 16, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Can you describe what Zenlytic is and the story behind it? Don't forget to check out our other shows.

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Data Engineering Podcast

JANUARY 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino.

Data Process 147

Data Process Process Data Lake High Quality Data 147

Knowledge Hut

APRIL 9, 2024

It is more about a mindset change. Based on our research findings and conversations with top executives, we discovered that Agile methodologies could help spur growth and support digital transformation in an era of high customer demand and fast-emerging market trends. It can be a combination of stories, incidents, defects, etc.

IT 98

IT Certification Project Consulting 98

Knowledge Hut

JANUARY 11, 2024

In our organization, I, as a product manager, have the onus of converting all user needs, wants, and challenges into our vision and decomposing the same into strategy, roadmap, and task-level activities. This is where the agile solution of an “Epic” comes to my aid.

Certification 97

Certification Programming Project Management 97

The Pragmatic Engineer

MAY 11, 2023

A JP Morgan stock analyst did just this, quickly crunching numbers and asking the question: “David, looking at the math on this large upfront bill that did not recur, it seems to be about $65 million, if I'm running that correctly. We had always talked about some of the industries that were most affected and optimized.

AWS 304

AWS Software Engineer Software Engineering Google Cloud 304

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. Rudderstack : !

Process 182

Process Building SQL BI 182

Data Engineering Podcast

JANUARY 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Don't forget to check out our other shows. __init__ covers the Python language, its community, and the innovative ways it is being used. Closing Announcements Thank you for listening!

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

FEBRUARY 11, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Your host is Tobias Macey and today I'm interviewing Andy Jefferson about how to solve the problem of data sharing Interview Introduction How did you get involved in the area of data management?

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Cloudera

APRIL 26, 2024

This month is Allyship April—a time dedicated to deepening our understanding of allyship and its profound impact on fostering inclusive cultures. Allyship isn’t merely a buzzword; it’s a fundamental commitment to actively support and advocate for marginalized individuals and communities within our organization.

Education 93

Education Engineering IT Management 93