Data Engineering Podcast

NOVEMBER 6, 2022

In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications. Can you share your definition of "behavioral data" and how it is differentiated from other sources/types of data?

Building 130

Building IT Metadata MongoDB 130

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data.

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

SEPTEMBER 11, 2022

In this episode he shares the design of the project and how it fits into your development practices. Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

SEPTEMBER 25, 2022

In this episode Tom Baeyens explains their reasons for creating a new syntax for expressing and validating checks for data assets and processes, as well as how to incorporate it into your own projects. What are the ways that reliability is measured for data assets? Can you describe what SodaCL is and the story behind it?

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

MAY 15, 2022

Srivatsan Sridharan has had the opportunity to design, build, and run data lake platforms for both Yelp and Robinhood, with many valuable lessons learned from each experience. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Building BI Architecture 100

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

SEPTEMBER 3, 2018

Using master data management to build a data catalog helps you answer these questions reliably and simplify the process of building your business intelligence reports. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Knowledge Hut

OCTOBER 27, 2023

Data visualization and presentation techniques help you present information and make faster decisions. Top Power BI Project Ideas Microsoft is always making Power BI better, and now that it's part of Fabric, the updates are really big. I have put together a list of Power BI projects for practice and to learn from.

BI 52

BI Project Datasets Healthcare 52

Data Engineering Podcast

AUGUST 13, 2023

Summary Data pipelines are the core of every data product, ML model, and business intelligence dashboard. The folks at Rivery distilled the seven principles of modern data pipelines that will help you stay out of trouble and be productive with your data. Closing Announcements Thank you for listening!

Data Pipeline 130

Data Pipeline BI SQL Business Intelligence 130

Data Engineering Podcast

MAY 19, 2019

Summary In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. The data build tool (dbt) is designed to bring battle tested engineering practices to your analytics pipelines. When is it the wrong choice?

Data Analytics 100

Data Analytics Building Engineering Data Warehouse 100

Data Engineering Podcast

OCTOBER 22, 2021

Summary The market for business intelligence has been going through an evolutionary shift in recent years. Lightdash has fully embraced that shift by building an entire open source business intelligence framework that is powered by dbt models. What are the main goals of the project?

Engineering 100

Engineering Business Intelligence BI Data Warehouse 100

Knowledge Hut

MAY 9, 2024

Data Architect ScyllaDB Data architects play a crucial role in designing an organization's data management framework by assessing data sources and integrating them into a centralized plan. They manage data storage and the ETL process.

Data Science 52

Data Science Data Mining Data Architect Programming Language 52

Data Engineering Podcast

MAY 22, 2022

Summary Machine learning has become a meaningful target for data applications, bringing with it an increase in the complexity of orchestrating the entire data flow. Flyte is a project that was started at Lyft to address their internal needs for machine learning and integrated closely with Kubernetes as the execution manager.

Machine Learning 130

Machine Learning Data Engineering Data Engineer Cloud 130

Knowledge Hut

NOVEMBER 16, 2023

JSP certification JSP is a Web page-building tool that allows for dynamic content. The demand for SAS – data analytics is growing day-by-day and the business intelligence domain has emerged as one of the most trusted and lucrative options for science graduates.

Technology 98

Technology MySQL MongoDB Google Cloud 98

Data Engineering Podcast

JUNE 12, 2022

Summary Building a well rounded and effective data team is an iterative process, and the first hire can set the stage for future success or failure. Trupti Natu has been the first data hire multiple times and gone through the process of building teams across the different stages of growth.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JUNE 22, 2020

Summary The majority of analytics platforms are focused on use internal to an organization by business stakeholders. As the availability of data increases and overall literacy in how to interpret it and take action improves there is a growing need to bring business intelligence use cases to a broader audience.

Business Intelligence 100

Business Intelligence Data Engineering Data Engineer SQL 100

Data Engineering Podcast

APRIL 26, 2021

Summary The reason for collecting, cleaning, and organizing data is to make it usable by the organization. One of the most common and widely used methods of access is through a business intelligence dashboard. Once you sign up and create an alert in Datafold for your company data, they will send you a cool water flask.

Business Intelligence 100

Business Intelligence Data Warehouse Hadoop Data Pipeline 100

Knowledge Hut

DECEMBER 26, 2023

According to the World Economic Forum, the amount of data generated per day will reach 463 exabytes (1 exabyte = 10 9 gigabytes) globally by the year 2025. Thus, almost every organization has access to large volumes of rich data and needs “experts” who can generate insights from this rich data.

Data Science 97

Data Science BI Business Intelligence Data Mining 97

Data Engineering Podcast

AUGUST 21, 2022

In this episode Tracy Daniels, CDO of Truist, shares her journey into the position, her responsibilities, and her relationship to the data professionals in her organization. RudderStack helps you build a customer data platform on your warehouse or data lake. Can you describe what your path to CDO of Truist has been?

Metadata 100

Metadata MongoDB MySQL Data Lake 100

Data Engineering Podcast

NOVEMBER 4, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Warehouse 100

Data Warehouse Business Intelligence ETL System Data Lake 100

Knowledge Hut

DECEMBER 29, 2023

A Data Engineer in the Data Science team is responsible for this sort of data manipulation. Big Data is a part of this umbrella term, which encompasses Data Warehousing and Business Intelligence as well. A Data Engineer's primary responsibility is the construction and upkeep of a data warehouse.

Data Science 98

Data Science Data Mining Deep Learning Programming Language 98

Data Engineering Podcast

AUGUST 6, 2022

In this episode Paolo Platter shares the lessons they have learned in that process, the Data Mesh Boost platform that they have built to reduce some of the boilerplate required to make it successful, and some of the considerations to make when deciding if a data mesh is the right choice for you.

Metadata 100

Metadata MongoDB MySQL Business Intelligence 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

APRIL 16, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Business intelligence is a crowded market. Can you describe what Zenlytic is and the story behind it?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Engineering 100

Data Engineering Data Engineer MongoDB Metadata 100

Data Engineering Podcast

DECEMBER 11, 2021

In order to make it easier for developers to build customer profiles in a way that respects their privacy Serge Huber helped to create the Apache Unomi framework as an open source customer data platform. Start trusting your data with Monte Carlo today! Visit dataengineeringpodcast.com/montecarlo to learn more.

Data Warehouse 100

Data Warehouse Raw Data Data Lake BI 100

Data Engineering Podcast

NOVEMBER 10, 2021

Summary A significant source of friction and wasted effort in building and integrating data management systems is the fragmentation of metadata across various tools. This is an ambitious and exciting project, so listen and try it out today. Start trusting your data with Monte Carlo today!

Metadata 100

Metadata Data Warehouse Data Lake BI 100

Data Engineering Podcast

OCTOBER 16, 2022

In this episode Adrian Kosowski explains how the Pathway product got started, how its design simplifies the creation of data products that support supply chain operations, and how developers can help to build an ecosystem of applications that allow businesses to accelerate their time to insight.

Database 100

Database Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 10, 2022

Summary Building a data platform is a journey, not a destination. Beyond the work of assembling a set of technologies and building integrations across them, there is also the work of growing and organizing a team that can support and benefit from that platform. Need to automate data pipelines with less red tape?

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

JUNE 5, 2022

Summary The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

JULY 31, 2022

In this episode Ernie Ostic shares the approach that he and his team at Manta are taking to build a complete view of data lineage across the various data systems in your organization and the useful applications of that information in the work of every data stakeholder. What are the core problems that Manta aims to solve?

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

In this episode Roambee CEO, Sanjay Sharma, shares the types of questions that companies are asking about their logistics, the technical work that they do to provide ways to answer those questions, and how they approach the challenge of data quality in its many forms. Can you describe what Roambee is and the story behind it?

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

MAY 8, 2022

In this episode Jon Herke shares how TigerGraph customers are taking advantage of those capabilities to achieve meaningful discoveries in their fields, the utilities that it provides for modeling and managing your connected data, and some of his own experiences working with the platform before joining the company.

Database 100

Database Data Lake BI Business Intelligence 100

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

JANUARY 29, 2023

Summary Business intelligence has gone through many generational shifts, but each generation has largely maintained the same workflow. Data analysts create reports that are used by the business to understand and direct the business, but the process is very labor and time intensive.

Business Intelligence 130

Business Intelligence PostgreSQL Building BI 130

Data Engineering Podcast

SEPTEMBER 4, 2022

In this episode founders Barr Moses and Lior Gavish rejoin the show to reflect on the evolution and adoption of data observability technologies and the capabilities that are being introduced as the broader ecosystem adopts the practices. RudderStack helps you build a customer data platform on your warehouse or data lake.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. Need to automate data pipelines with less red tape? How do lakehouse/data virtualization systems allow for that?

Architecture 100

Architecture Metadata MongoDB MySQL 100