Data Engineering Podcast

NOVEMBER 6, 2022

In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications. Can you share your definition of "behavioral data" and how it is differentiated from other sources/types of data?

Building 130

Building IT Metadata MongoDB 130

The Modern Data Company

APRIL 15, 2024

With this in mind, let’s explore how to demystify the process of building your data-driven strategy, making it accessible and actionable. We’ll uncover how you can transform data into a strategic asset that propels your organization forward without getting lost in the complexity of its creation. It matters a lot.

Building 52

Building Raw Data Food Data 52

Data Engineering Podcast

JUNE 26, 2022

Summary The most complicated part of data engineering is the effort involved in making the raw data fit into the narrative of the business. Master Data Management (MDM) is the process of building consensus around what the information actually means in the context of the business and then shaping the data to match those semantics.

Data Management 100

Data Management Management MongoDB Scala 100

Data Engineering Podcast

MARCH 27, 2022

The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. Privacera is an enterprise grade solution for cloud and hybrid data governance built on top of the robust and battle tested Apache Ranger project. Closing Announcements Thank you for listening!

Data Governance 130

Data Governance Government Cloud Building 130

Data Engineering Podcast

SEPTEMBER 11, 2022

Summary Any business that wants to understand their operations and customers through data requires some form of pipeline. Building reliable data pipelines is a complex and costly undertaking with many layered requirements. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

JUNE 8, 2020

Summary The landscape of data management and processing is rapidly changing and evolving. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering?

Data Management 100

Data Management Management Machine Learning Portfolio 100

Knowledge Hut

MARCH 27, 2024

In the world of project management, adherence to PMP project compliance standards is not just a requirement but a cornerstone of success. I learned this firsthand through a personal experience where strict adherence to compliance protocols ensured project success. What is Project Compliance?

Project 52

Project Process Government Management 52

Data Engineering Podcast

AUGUST 20, 2021

In this episode Vikrant Dubey discusses his work on the Cuelake project which allows data analysts to build a lakehouse with SQL queries. By building on top of Zeppelin, Spark, and Iceberg he and his team at Cuebook have built an autoscaled cloud native system that abstracts the underlying complexity.

Building 100

Building Data Lake Data Warehouse SQL 100

Data Engineering Podcast

DECEMBER 7, 2020

Summary Building data products are complicated by the fact that there are so many different stakeholders with competing goals and priorities. Different organizations have tried a multitude of organizational strategies to improve the success rate of these data teams with varying levels of success. centralized, embedded, etc.)

Building 100

Building Consulting Big Data Data Engineering 100

Knowledge Hut

JANUARY 9, 2024

SAP frameworks provide a foundation for building custom SAP applications that meet the specific needs of a business. As the demand for SAP is growing, SAP project managers with the right skills and experience are becoming a priority. Who is a SAP Project Manager and What do They Do?

Project 52

Project Management Certification Portfolio 52

Knowledge Hut

OCTOBER 27, 2023

Data visualization and presentation techniques help you present information and make faster decisions. Top Power BI Project Ideas Microsoft is always making Power BI better, and now that it's part of Fabric, the updates are really big. I have put together a list of Power BI projects for practice and to learn from.

BI 52

BI Project Datasets Healthcare 52

Knowledge Hut

OCTOBER 27, 2023

Angular created by Google is an open-source JavaScript framework that is widely used for building dynamic web applications. Its popularity and widespread adoption in the technology world make it an important tool for developers seeking to build robust and cutting-edge applications.

Coding 52

Coding Project Media Portfolio 52

Data Engineering Podcast

AUGUST 10, 2021

In this episode Mark Grover explains what he is building at Stemma, how it expands on the success of the Amundsen project, and why trust is the most important asset for data teams. RudderStack’s smart customer data pipeline is warehouse-first. Go to dataengineeringpodcast.com/census today to get a free 14-day trial.

IT 130

IT Building Data Warehouse Python 130

Data Engineering Podcast

FEBRUARY 15, 2021

In order to be able to truly derive insights from the data that an organization collects, there are certain foundational capabilities that they need to have capacity for. This is a great conversation about the initial steps that every group should be thinking of as they start down the road to making data informed decisions.

Building 100

Building Data Warehouse BI Data Pipeline 100

Data Engineering Podcast

FEBRUARY 8, 2021

Summary With all of the tools and services available for building a data platform it can be difficult to separate the signal from the noise. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify. What kinds of data sources are you working with?

Data Warehouse 100

Data Warehouse Building BI SQL 100

Data Engineering Podcast

FEBRUARY 13, 2022

To simplify the work of managing the full flow of your customer data and keep you in full control the team at Rudderstack created their eponymous open source platform that allows you to work with first and third party data, as well as build and manage reverse ETL workflows. Who are the target users of Rudderstack?

Building 100

Building Hadoop Data Pipeline Metadata 100

Data Engineering Podcast

NOVEMBER 9, 2020

Summary A data catalog is a critical piece of infrastructure for any organization who wants to build analytics products, whether internal or external. While there are a number of platforms available for building that catalog, many of them are either difficult to deploy and integrate, or expensive to use at scale.

Building 100

Building PostgreSQL BI Metadata 100

Data Engineering Podcast

MARCH 16, 2020

In this episode Adam Kocoloski shares the history of the project, how it works under the hood, and how the new design will improve the project for our new era of computation. This was an interesting conversation about the challenges of maintaining a large and mission critical project and the work being done to evolve it.

Building 100

Building Data Warehouse NoSQL Data Lake 100

Data Engineering Podcast

SEPTEMBER 11, 2022

In this episode he shares the design of the project and how it fits into your development practices. Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

SEPTEMBER 25, 2022

In this episode Tom Baeyens explains their reasons for creating a new syntax for expressing and validating checks for data assets and processes, as well as how to incorporate it into your own projects. What are the ways that reliability is measured for data assets? Can you describe what SodaCL is and the story behind it?

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

MAY 15, 2022

Srivatsan Sridharan has had the opportunity to design, build, and run data lake platforms for both Yelp and Robinhood, with many valuable lessons learned from each experience. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Building BI Business Intelligence 100

Data Engineering Podcast

AUGUST 26, 2019

Summary Data engineers are responsible for building tools and platforms to power the workflows of other members of the business. Benn Stancil is the chief analyst at Mode Analytics and in this episode he explains the set of considerations and requirements that data analysts need in their tools and.

Building 100

Building Data Analytics Media Data Engineering 100

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

JANUARY 23, 2022

He also provides some excellent insights into the factors that play into the build vs. buy decision at different organizational sizes. Your host is Tobias Macey and today I’m interviewing Ashish Mrig about his path as a data engineer Interview Introduction How did you get involved in the area of data management?

Building 100

Building Management Data Pipeline Metadata 100

Data Engineering Podcast

APRIL 19, 2020

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. Eventador is a managed platform designed to let you focus on using the data that you collect, without worrying about how to make it reliable.

Building 100

Building PostgreSQL MongoDB SQL 100

Data Engineering Podcast

AUGUST 31, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering? If you hand a book to a new data engineer, what wisdom would you add to it? When is Firebolt the wrong choice?

Data Warehouse 100

Data Warehouse Cloud Building Data Lake 100

Data Engineering Podcast

JULY 27, 2020

In this episode he shares his approach to testing complex systems, the common challenges that are faced by engineers who build them, and why it is important to understand their limitations. I’m working with O’Reilly on a project to collect the 97 things that every data engineer should know, and I need your help.

Systems 100

Systems Building Scala Java 100

Data Engineering Podcast

DECEMBER 14, 2020

In this episode Andrew Gross, Bobby Muldoon, and Anup Segu describe the self service data platform that they have built to allow data analysts to own the end-to-end delivery of data projects and how that has allowed them to scale their output. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Analytics 100

Data Analytics Building MySQL Data 100

Data Engineering Podcast

OCTOBER 28, 2019

In an effort to create a better abstraction for building data applications Nick Schrock created Dagster. In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and his vision for the future of data programming.

Building 100

Building Data Pipeline Programming Language Metadata 100

Data Engineering Podcast

APRIL 6, 2020

By organizing and storing data to emphasize the relationship between entities, we can discover the complex connections between multiple sources of information. If you’re wondering how to extract additional business value from existing data, this episode will provide a way to expand your data resources.

Building 100

Building Big Data Machine Learning Python 100

Knowledge Hut

SEPTEMBER 29, 2023

In the below blog, we are going to read more about some of these AWS Solution Architect projects. In the normal world, architects are responsible for creating structures and various designs of buildings. AWS Solution Architects are similar to them just instead of buildings, they are responsible for creating and designing in AWS.

AWS 52

AWS Project Coding Cloud Computing 52

Data Engineering Podcast

FEBRUARY 3, 2020

If you are curious about data sharing strategies or data collaboratives, then listen now to learn more! With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform.

Building 100

Building Media AWS Data 100

Knowledge Hut

DECEMBER 28, 2023

In this blog, we'll talk about intriguing and real-time sample Hadoop projects with source codes that can help you take your data analysis to the next level. Why Are Hadoop Projects So Important? The Apache Hadoop framework provides tools for efficient analysis, resource management, and parallel processing of datasets.

Hadoop 52

Hadoop Project Datasets Big Data 52

Data Engineering Podcast

SEPTEMBER 3, 2018

Using master data management to build a data catalog helps you answer these questions reliably and simplify the process of building your business intelligence reports. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Data Engineering Podcast

JULY 27, 2021

This is a fascinating conversation with a lot of useful lessons for anyone who wants to understand the operational aspects of Pulsar and the benefits that it can provide to data workloads. RudderStack’s smart customer data pipeline is warehouse-first. Go to dataengineeringpodcast.com/97things today to get your copy!

Building 100

Building Management Kafka Data Warehouse 100

ProjectPro

FEBRUARY 16, 2023

With so much riding on the efficiency of ETL processes for data engineering teams, it is essential to take a deep dive into the complex world of ETL on AWS to take your data management to the next level. This is particularly useful for companies that need to process data in near-real-time.

AWS 52

AWS Data Management ETL Tools Management 52