Data Engineering Podcast

NOVEMBER 6, 2022

In this episode Nick King discusses how you can be intentional about data creation in your applications and services to reduce the friction and errors involved in building data products and ML applications. Can you share your definition of "behavioral data" and how it is differentiated from other sources/types of data?

Building 130

Building IT Metadata MongoDB 130

Data Engineering Podcast

JULY 17, 2022

Summary There are extensive and valuable data sets that are available outside the bounds of your organization. Whether that data is public, paid, or scraped it requires investment and upkeep to acquire and integrate it with your systems. Need to automate data pipelines with less red tape?

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

FEBRUARY 20, 2022

He also explains how he has architected the systems that ingest, process, and distribute the data that he is responsible for and the requirements that are introduced when collaborating with researchers, domain experts, and machine learning developers. RudderStack’s smart customer data pipeline is warehouse-first.

Systems 100

Systems Software Engineer Software Engineering Data Warehouse 100

Data Engineering Podcast

APRIL 24, 2022

WhyLogs is a powerful library for flexibly instrumenting all of your data systems to understand the entire lifecycle of your data from source to productionized model. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Machine Learning 100

Machine Learning Systems Data Lake Metadata 100

Data Engineering Podcast

MARCH 27, 2022

The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. Privacera is an enterprise grade solution for cloud and hybrid data governance built on top of the robust and battle tested Apache Ranger project.

Data Governance 130

Data Governance Government Cloud Building 130

Data Engineering Podcast

JULY 6, 2020

Summary There are an increasing number of use cases for real time data, and the systems to power them are becoming more mature. Once you have a streaming platform up and running you need a way to keep an eye on it, including observability, discovery, and governance of your data. That’s what the Lenses.io

Systems 100

Systems Kafka SQL Government 100

The Modern Data Company

APRIL 15, 2024

With this in mind, let’s explore how to demystify the process of building your data-driven strategy, making it accessible and actionable. We’ll uncover how you can transform data into a strategic asset that propels your organization forward without getting lost in the complexity of its creation. It matters a lot.

Building 52

Building Raw Data Food Data 52

Data Engineering Podcast

JULY 20, 2020

Summary Wind energy is an important component of an ecologically friendly power system, but there are a number of variables that can affect the overall efficiency of the turbines. Michael Tegtmeier founded Turbit Systems to help operators of wind farms identify and correct problems that contribute to suboptimal power outputs.

Systems 100

Systems Manufacturing Machine Learning Algorithm 100

Data Engineering Podcast

AUGUST 20, 2021

In this episode Vikrant Dubey discusses his work on the Cuelake project which allows data analysts to build a lakehouse with SQL queries. By building on top of Zeppelin, Spark, and Iceberg he and his team at Cuebook have built an autoscaled cloud native system that abstracts the underlying complexity.

Building 100

Building Data Lake Data Warehouse SQL 100

Data Engineering Podcast

JULY 5, 2021

SaaSGlue is a managed service that lets you connect all of your systems, across clouds and physical infrastructure, and spanning all of your programming languages. RudderStack’s smart customer data pipeline is warehouse-first. I understand that you are building this company with your 3 brothers.

Systems 100

Systems Management Data Warehouse Programming Language 100

Knowledge Hut

JANUARY 9, 2024

As businesses are becoming more complex, the need for software systems that can help them manage their operations has taken the first place. SAP frameworks provide a foundation for building custom SAP applications that meet the specific needs of a business. Who is a SAP Project Manager and What do They Do?

Project 52

Project Management Certification Portfolio 52

Knowledge Hut

MARCH 27, 2024

In the world of project management, adherence to PMP project compliance standards is not just a requirement but a cornerstone of success. I learned this firsthand through a personal experience where strict adherence to compliance protocols ensured project success. What is Project Compliance?

Project 52

Project Process Government Management 52

Data Engineering Podcast

AUGUST 10, 2021

In this episode Mark Grover explains what he is building at Stemma, how it expands on the success of the Amundsen project, and why trust is the most important asset for data teams. RudderStack’s smart customer data pipeline is warehouse-first. Go to dataengineeringpodcast.com/census today to get a free 14-day trial.

IT 130

IT Building Data Warehouse Python 130

Data Engineering Podcast

MAY 6, 2019

Summary The database market continues to expand, offering systems that are suited to virtually every use case. FoundationDB is a distributed key-value store that provides the primitives that you need to build a custom database platform. Can you explain what FoundationDB is and how you got involved with the project?

Systems 100

Systems MongoDB NoSQL Database 100

Data Engineering Podcast

NOVEMBER 9, 2020

Summary A data catalog is a critical piece of infrastructure for any organization who wants to build analytics products, whether internal or external. While there are a number of platforms available for building that catalog, many of them are either difficult to deploy and integrate, or expensive to use at scale.

Building 100

Building PostgreSQL BI Metadata 100

Knowledge Hut

OCTOBER 27, 2023

Angular created by Google is an open-source JavaScript framework that is widely used for building dynamic web applications. Its popularity and widespread adoption in the technology world make it an important tool for developers seeking to build robust and cutting-edge applications.

Coding 52

Coding Project Media Portfolio 52

Data Engineering Podcast

FEBRUARY 8, 2021

Summary With all of the tools and services available for building a data platform it can be difficult to separate the signal from the noise. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify. What kinds of data sources are you working with?

Data Warehouse 100

Data Warehouse Building BI SQL 100

Data Engineering Podcast

APRIL 19, 2020

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. Eventador is a managed platform designed to let you focus on using the data that you collect, without worrying about how to make it reliable.

Building 100

Building PostgreSQL MongoDB SQL 100

Knowledge Hut

DECEMBER 28, 2023

In this blog, we'll talk about intriguing and real-time sample Hadoop projects with source codes that can help you take your data analysis to the next level. Why Are Hadoop Projects So Important? The Apache Hadoop framework provides tools for efficient analysis, resource management, and parallel processing of datasets.

Hadoop 52

Hadoop Project Datasets Big Data 52

Data Engineering Podcast

FEBRUARY 15, 2021

In order to be able to truly derive insights from the data that an organization collects, there are certain foundational capabilities that they need to have capacity for. This is a great conversation about the initial steps that every group should be thinking of as they start down the road to making data informed decisions.

Building 100

Building Data Warehouse BI Data Pipeline 100

Data Engineering Podcast

FEBRUARY 13, 2022

To simplify the work of managing the full flow of your customer data and keep you in full control the team at Rudderstack created their eponymous open source platform that allows you to work with first and third party data, as well as build and manage reverse ETL workflows. Who are the target users of Rudderstack?

Building 100

Building Hadoop Data Pipeline Metadata 100

Data Engineering Podcast

MAY 15, 2022

Srivatsan Sridharan has had the opportunity to design, build, and run data lake platforms for both Yelp and Robinhood, with many valuable lessons learned from each experience. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Building BI Architecture 100

Data Engineering Podcast

AUGUST 26, 2019

Summary Data engineers are responsible for building tools and platforms to power the workflows of other members of the business. Benn Stancil is the chief analyst at Mode Analytics and in this episode he explains the set of considerations and requirements that data analysts need in their tools and.

Building 100

Building Data Analytics Media Data Engineering 100

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 3, 2023

The dlt project was created to eliminate overhead and bring data integration into your full control as a library component of your overall data system. In this episode Adrian Brudaru explains how it works, the benefits that it provides over other data integration solutions, and how you can start building pipelines today.

Data Integration 162

Data Integration BI SQL Python 162

Data Engineering Podcast

SEPTEMBER 3, 2018

Using master data management to build a data catalog helps you answer these questions reliably and simplify the process of building your business intelligence reports. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Knowledge Hut

OCTOBER 27, 2023

Data visualization and presentation techniques help you present information and make faster decisions. Top Power BI Project Ideas Microsoft is always making Power BI better, and now that it's part of Fabric, the updates are really big. I have put together a list of Power BI projects for practice and to learn from.

BI 52

BI Project Datasets Healthcare 52

Data Engineering Podcast

NOVEMBER 26, 2019

In this episode James Cunningham and Ted Kaemming describe the process of rearchitecting a production system, what they learned in the process, and some useful tips for anyone else evaluating Clickhouse. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, Alluxio, and Data Council.

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

ProjectPro

FEBRUARY 16, 2023

With so much riding on the efficiency of ETL processes for data engineering teams, it is essential to take a deep dive into the complex world of ETL on AWS to take your data management to the next level. AWS refers to Amazon Web Service, the most widely used cloud computing system.

AWS 52

AWS Data Management ETL Tools Management 52

Data Engineering Podcast

OCTOBER 28, 2019

In an effort to create a better abstraction for building data applications Nick Schrock created Dagster. In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and his vision for the future of data programming.

Building 100

Building Data Pipeline Programming Language Metadata 100

Data Engineering Podcast

DECEMBER 14, 2020

In this episode Andrew Gross, Bobby Muldoon, and Anup Segu describe the self service data platform that they have built to allow data analysts to own the end-to-end delivery of data projects and how that has allowed them to scale their output. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Analytics 100

Data Analytics Building MySQL Data 100

Data Engineering Podcast

JULY 27, 2021

This is a fascinating conversation with a lot of useful lessons for anyone who wants to understand the operational aspects of Pulsar and the benefits that it can provide to data workloads. RudderStack’s smart customer data pipeline is warehouse-first. Go to dataengineeringpodcast.com/97things today to get your copy!

Building 100

Building Management Kafka Data Warehouse 100

Data Engineering Podcast

DECEMBER 22, 2019

Between those use cases there are varying levels of support for fast reads on quickly changing data. To address that need more completely the team at Materialize has created an engine that allows for building queryable views of your data as it is continually updated from the stream of changes being generated by your applications.

SQL 100

SQL Building Engineering Java 100

Knowledge Hut

SEPTEMBER 29, 2023

In the below blog, we are going to read more about some of these AWS Solution Architect projects. In the normal world, architects are responsible for creating structures and various designs of buildings. AWS Solution Architects are similar to them just instead of buildings, they are responsible for creating and designing in AWS.

AWS 52

AWS Project Coding Cloud Computing 52

Knowledge Hut

NOVEMBER 2, 2023

As an AWS Cloud Practitioner with experience in delivering multiple AWS cloud practitioner projects, I vividly recall assisting a startup to prove the scalability of their AI solution on AWS during one of my early projects. In this dynamic profession, there is no alternative to project-based learning.

AWS 52

AWS Coding Cloud Project 52

Data Engineering Podcast

DECEMBER 30, 2019

In order to support their customers, they need to capture, process, and analyze massive amounts of timeseries data with a high degree of uptime and reliability. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council. Closing Announcements Thank you for listening!

Process 100

Process Building Hadoop Java 100

Data Engineering Podcast

JULY 2, 2023

In this episode Razi Raziuddin shares how data engineering teams can support the machine learning workflow through the development and support of systems that empower data scientists and ML engineers to build and maintain their own features. What is feature engineering is and why/to whom it matters?

Machine Learning 130

Machine Learning Data Engineering Data Engineer Engineering 130