Monte Carlo

AUGUST 31, 2023

Data is a priority for your CEO, as it often is for digital-first companies, and she is fluent in the latest and greatest business intelligence tools. Your CTO is excited about migrating to the cloud, and constantly sends your team articles highlighting performance measurements against some of the latest technologies.

Data Lake 52

Data Lake Data Pipeline Unstructured Data Data Warehouse 52

Data Engineering Podcast

JUNE 8, 2020

Summary The landscape of data management and processing is rapidly changing and evolving. This is a useful conversation to gain a macro perspective on where businesses are looking to improve their capabilities to work with data. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Management 100

Data Management Management Machine Learning Portfolio 100

Data Engineering Podcast

FEBRUARY 26, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. We feel your pain.

Building 147

Building Metadata Machine Learning Data Integration 147

The Modern Data Company

APRIL 15, 2024

Helps in organizing new data (like new books) in the right place. Understands how different pieces of data (books) relate to each other, helping users find related information. Keeps the library orderly, adjusting as new data arrives. Tracks key performance indicators to understand the use and value of the data.

Government 52

Government Metadata Data Data Lake 52

Data Engineering Podcast

OCTOBER 30, 2022

Summary One of the most impactful technologies for data analytics in recent years has been dbt. It’s hard to have a conversation about data engineering or analysis without mentioning it. Despite its widespread adoption there are still rough edges in its workflow that cause friction for data analysts.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode she shares the story behind the project, the details of how it is implemented, and how you can use it for your own data projects. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Who is the target audience for Zingg?

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Podcast

AUGUST 3, 2020

Graph data models and the applications built on top of them are perfect for representing relationships and finding emergent structures in your information. This was an informative and enlightening conversation with two experts on graph data applications that will help you start on the right track in your own projects.

NoSQL 100

NoSQL Database Algorithm Relational Database 100

Data Engineering Podcast

DECEMBER 9, 2018

Jean George Perrin has been so impressed by the versatility of Spark that he is writing a book for data engineers to hit the ground running. He also discusses what you need to know to get it deployed and keep it running in a production environment and how it fits into the overall data ecosystem. Who is the target audience?

Scala 100

Scala MySQL Kafka Hadoop 100

Data Engineering Podcast

JULY 17, 2022

Summary Data engineering is a large and growing subject, with new technologies, specializations, and "best practices" emerging at an accelerating pace. init started in 2015, and your first episode of Data Engineering was published January 14, 2017. The Data Engineering Podcast covers the latest on modern data management.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JANUARY 30, 2022

He recently wrote a book on effective patterns for Pandas code, and in this episode he shares advice on how to write efficient data processing routines that will scale with your data volumes, while being understandable and maintainable. What are the main tasks that you have seen Pandas used for in a data engineering context?

Data Engineering 100

Data Engineering Data Engineer Engineering Data Pipeline 100

Data Engineering Podcast

JULY 10, 2022

Summary Building a data platform is a journey, not a destination. Beyond the work of assembling a set of technologies and building integrations across them, there is also the work of growing and organizing a team that can support and benefit from that platform. Tired of deploying bad data? What does Riskified do?

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

AUGUST 3, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. What is involved in migrating an existing data lake to use Hudi?

Data Lake 130

Data Lake Data Warehouse Hadoop Architecture 130

Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

DECEMBER 21, 2020

The team at Immuta has built a platform that aims to tackle that problem in a flexible and maintainable fashion so that data teams can easily integrate authorization, data masking, and privacy enhancing technologies into their data infrastructure. What is data governance?

Data Governance 130

Data Governance Government Data Lake Banking 130

Data Engineering Podcast

DECEMBER 11, 2022

Summary One of the most critical aspects of software projects is managing its data. Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. Closing Announcements Thank you for listening!

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

DECEMBER 4, 2022

In this episode Arjun Narayan explains how the technical barriers to adopting real-time data in your analytics and applications have become surmountable by organizations of all sizes. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

SEPTEMBER 4, 2022

In this episode founders Barr Moses and Lior Gavish rejoin the show to reflect on the evolution and adoption of data observability technologies and the capabilities that are being introduced as the broader ecosystem adopts the practices. You were one of the early entrants in the market of data quality/data observability products.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

MARCH 11, 2018

Practices such as version controlled migration scripts and iterative schema evolution provide the necessary mechanisms to ensure that your data layer is as agile as your application. What was the state of software and database system development at the time and why did you find it necessary to write a book on this subject?

Database 100

Database MongoDB NoSQL Insurance 100

Data Engineering Podcast

DECEMBER 7, 2020

If you are struggling to deliver value from big data, or just starting down the path of building the organizational capacity to turn raw information into valuable products then this is a conversation that you don’t want to miss. If you hand a book to a new data engineer, what wisdom would you add to it?

Building 100

Building Big Data Consulting Data Engineering 100

Data Engineering Podcast

JULY 27, 2021

Summary Everyone expects data to be transmitted, processed, and updated instantly as more and more products integrate streaming data. Now there’s a book that captures the foundational lessons and principles that underly everything that you hear about here.

Building 100

Building Management Kafka Data Warehouse 100

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

AUGUST 31, 2020

Summary Data warehouse technology has been around for decades and has gone through several generational shifts in that time. The current trends in data warehousing are oriented around cloud native architectures that take advantage of dynamic scaling and the separation of compute and storage. When is Firebolt the wrong choice?

Data Warehouse 100

Data Warehouse Cloud Building Data Lake 100

Knowledge Hut

NOVEMBER 29, 2023

Azure Solution Architect Expert For the role of an Azure Solution Architect Expert, you need to pass two exams - Exam AZ-300: Microsoft Azure Architect Technologies and AZ-301: Microsoft Azure Architect Design. To AZ - 300: Microsoft Azure Architect Technologies This exam validates your knowledge of secure and reliable architecting solutions.

Certification 52

Certification Big Data Machine Learning Data Lake 52

Data Engineering Podcast

JUNE 29, 2020

The team at Audio Analytic are working to impart a sense of hearing to our myriad devices with their sound recognition technology. This was a great conversation about the complexities of working in a niche domain of data analysis and how to build a pipeline of high quality data from collection to analysis.

Data Collection 100

Data Collection Management High Quality Data Metadata 100

Data Engineering Podcast

AUGUST 6, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. The market for data catalogues is nascent but growing fast.

Database 100

Database SQL BI Business Intelligence 100

Data Engineering Podcast

JULY 20, 2020

This was a great conversation about using data to improve the way the world works. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering?

Systems 100

Systems Machine Learning Manufacturing Algorithm 100

Knowledge Hut

MARCH 15, 2024

To thrive in the IT industry, one must be able to keep up with the swift advancements in technology. Obtaining a certification is a great way to keep up with the most recent trends and learn about the newest developments in technology. It teaches the fundamentals of data principles and how to use Microsoft data services.

Cloud Computing 52

Cloud Computing Algorithm Certification SQL 52

Knowledge Hut

JANUARY 19, 2024

Big Data Technologies Handling large-scale datasets is a challenge addressed by Big Data Technologies. The syllabus should cover technologies such as Apache Hadoop and Spark, introducing concepts of distributed computing and parallel processing for effective big data management.

Data Science 52

Data Science Machine Learning Datasets Algorithm 52

Data Engineering Podcast

JULY 19, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. How are teams identifying and addressing those failures?

Management 100

Management Data Warehouse Data Pipeline Data 100

Data Engineering Podcast

OCTOBER 19, 2020

They also discuss methods for gaining visibility into the flow of data through your infrastructure, how to diagnose and prevent potential problems, and what they are building at Monte Carlo to help you maintain your data’s uptime. If you hand a book to a new data engineer, what wisdom would you add to it?

Machine Learning 100

Machine Learning Data Engineering Data Engineer Data 100

Data Engineering Podcast

FEBRUARY 11, 2019

Summary Machine learning is a class of technologies that promise to revolutionize business. How much of your sales process is spent on educating your clients about what AI or ML are and the benefits that these technologies can provide? What basic technology stack is necessary for putting the first ML models into production?

Machine Learning 100

Machine Learning Deep Learning Recruitment Software Engineer 100

Data Engineering Podcast

JUNE 22, 2020

This was an interesting conversation about a different approach to business intelligence and the importance of expanded access to data. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering?

Business Intelligence 100

Business Intelligence Data Engineering Data Engineer SQL 100

Data Engineering Podcast

JULY 12, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. Can you start by giving your definition of the modern data stack?

Designing 100

Designing Data Warehouse Data Governance Consulting 100

Data Engineering Podcast

JUNE 12, 2022

In this episode she shares her thoughts and insights on how to be intentional about establishing your own data team. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

DECEMBER 2, 2017

Continuous delivery lets you get new features in front of your users as fast as possible without introducing bugs or breaking production and GoCD is the open source platform made by the people at Thoughtworks who wrote the book about it. The platform that you have built provides hosting for a large variety of data sizes and types.

Data Pipeline 100

Data Pipeline Data Engineering Data Engineer Architecture 100