Data Engineering Podcast

JULY 10, 2022

They also discuss how they have established a guild system for training and supporting data professionals in the organization. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

The Modern Data Company

JANUARY 31, 2023

We also had the opportunity to talk about DataOS and the new paradigm it presents to the world of data. Our founders — and, as a result, the entire company — think about data differently than everyone else. This post highlights exactly how our founders taught us to think differently about data and why it matters.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

FEBRUARY 24, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

IT 100

IT Data Lake Data Pipeline Media 100

The Modern Data Company

FEBRUARY 15, 2023

Maintaining a clear strategy for gathering, storing, and labeling data is challenging. Limited resources: Data management has always been resource intensive, but not all organizations can maintain a full data team. Without suitable resources for company-wide data management, it’s easier to fall behind.

Government 98

Government Data Governance Raw Data Data 98

Data Engineering Podcast

MARCH 16, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building Data Warehouse NoSQL Data Lake 100

Data Engineering Podcast

DECEMBER 21, 2020

If you are starting down the path of implementing a data governance strategy then this episode will provide a great overview of what is involved. If you hand a book to a new data engineer, what wisdom would you add to it? What is data governance? If you hand a book to a new data engineer, what wisdom would you add to it?

Data Governance 130

Data Governance Government Data Lake Banking 130

Databand.ai

AUGUST 30, 2023

Besides these categories, specialized solutions tailored specifically for particular domains or use cases also exist, such as extract, transform and load (ETL) tools for managing data pipelines, data integration tools for combining information from disparate sources or systems and more.

Data Cleanse 52

Data Cleanse Data Pipeline Datasets Data Validation 52

Data Engineering Podcast

MARCH 2, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Data Engineering Podcast

JUNE 12, 2022

In this episode she shares her thoughts and insights on how to be intentional about establishing your own data team. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JANUARY 4, 2021

Summary As more organizations are gaining experience with data management and incorporating analytics into their decision making, their next move is to adopt machine learning. Monte Carlo’s end-to-end Data Observability Platform monitors and alerts for data issues across your data warehouses, data lakes, ETL, and business intelligence.

Python 100

Python Data Lake Machine Learning Computer Science 100

Data Engineering Podcast

JANUARY 11, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

PostgreSQL 100

PostgreSQL Python Data Lake Computer Science 100

Ascend.io

MAY 23, 2023

The data engineering world is full of tips and tricks on how to handle specific patterns that recur with every data pipeline. In general, the root of the orphaned data problem lies in the interim storage of data between the individual processing steps that make up a data pipeline.

Datasets 52

Datasets Data Pipeline Metadata Database 52

Monte Carlo

JUNE 16, 2023

Data integrity and quality may seem similar at first glance, and they are sometimes used interchangeably in everyday life, but they play unique roles in successful data management. Impact Now that you understand the purpose of data integrity and data quality, what is their impact on data management and decision-making?

Data Integration 52

Data Integration Hospitality Banking Healthcare 52

The Modern Data Company

JULY 31, 2023

Achieving such flexible loops requires interoperable code, declarative change management, and independent infra provisioning (via IaC), akin to Software 101. Read more about our partnership.

Government 98

Government Data Governance Architecture Data Pipeline 98

Data Engineering Podcast

DECEMBER 16, 2019

Even if you already have a metadata repository this is worth a listen to learn more about the value that visibility of your data can bring to your organization. with the subject "Data Engineering Podcast" to get a hands-on demo from one of their data experts. Can you start by describing what Marquez is?

Metadata 100

Metadata PostgreSQL Datasets Data Warehouse 100

The Modern Data Company

DECEMBER 29, 2022

If they can streamline their data pipelines to provide actionable insights, they can usher in the next generation of financial services and survive the next global disruption. If the finance industry can leverage an operational layer, fractured data ecosystems will become a challenge of the past.

Banking 52

Banking Pipeline-centric Building Finance 52

The Modern Data Company

JULY 31, 2023

Achieving such flexible loops requires interoperable code, declarative change management, and independent infra provisioning (via IaC), akin to Software 101. By omitting the complexity and debt of data pipelines resulting from multiple fragmented tools in the ‘MAD Ecosystem,’ these loops can be efficiently maintained.

Government 52

Government Data Governance Architecture Data Pipeline 52

The Modern Data Company

JULY 31, 2023

Achieving such flexible loops requires interoperable code, declarative change management, and independent infra provisioning (via IaC), akin to Software 101. By omitting the complexity and debt of data pipelines resulting from multiple fragmented tools in the ‘MAD Ecosystem,’ these loops can be efficiently maintained.

Government 52

Government Data Governance Architecture Data Pipeline 52

Cloudera

JANUARY 20, 2021

The six steps are: Data Collection – data ingestion and monitoring at the edge (whether the edge be industrial sensors or people in a brick and mortar retail store). Data Enrichment – data pipeline processing, aggregation & management to ready the data for further refinement.

Manufacturing 107

Manufacturing Data Warehouse Kafka Retail 107

Striim

FEBRUARY 10, 2023

Macy’s migrated its on-premise inventory and order data to Google Cloud storage to reach its objectives. The company decided to move to the cloud based on the benefits of cost efficiency, flexibility, and improved data management. Request a demo today.

Data Analytics 52

Data Analytics Telecommunication Insurance Google Cloud 52

Data Engineering Podcast

JUNE 1, 2021

Summary Every part of the business relies on data, yet only a small team has the context and expertise to build and maintain workflows and data pipelines to transform, clean, and integrate it. RudderStack’s smart customer data pipeline is warehouse-first. What are the main goals that you have for Shipyard?

Data Pipeline 100

Data Pipeline Data Warehouse Data Workflow Data 100

Data Engineering Podcast

DECEMBER 28, 2022

In this episode he shares his thoughts on the strategic and tactical elements of moving your work as a data professional from being task-oriented to being product-oriented and the long term improvements in your productivity that it provides. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Data Engineering Podcast

NOVEMBER 27, 2022

He also discusses the improvements that have been incorporated into FeatureBase to simplify integration with the rest of your data stack, and the SQL interface that was added to make working with the product easier. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

NOVEMBER 20, 2022

In this episode CEO and founder Salma Bakouk shares her views on the causes and impacts of "data entropy" and how you can tame it before it leads to failures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. What is the motivating goal for the company and product?

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

Cloudera

APRIL 15, 2019

Cloudera Flow Management (CFM) is a no-code data ingestion and management solution powered by Apache NiFi. With a slick user interface, 300+ processors and the NiFi Registry, CFM delivers highly scalable data management and DevOps capabilities to the enterprise. Cloudera Flow Management.

Management 17

Management Data Ingestion Data Collection Government 17

Monte Carlo

FEBRUARY 6, 2023

Governance What data is being housed and the requirements of your business and industry will determine the level and types of governance policies you’ll need to implement. For example, healthcare providers are generally subject to HIPAA regulations pertaining to data management and usage.

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

Data Engineering Podcast

JUNE 26, 2022

In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

MAY 1, 2022

He describes how the platform is architected, the challenges related to selling cloud technologies into enterprise organizations, and how you can adopt Matillion for your own workflows to reduce the maintenance burden of data integration workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Data Engineering Podcast

APRIL 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Missing data?

Data Warehouse 100

Data Warehouse Data Lake Data Pipeline BI 100

Data Engineering Podcast

APRIL 17, 2022

In this episode Prineha Narang, co-founder and CTO of Aliro, explains how these systems work, the capabilities that they can offer, and how you can start preparing for a post-quantum future for your data systems. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks.

Data Warehouse 100

Data Warehouse SQL Data Engineering Data Engineer 100

Data Engineering Podcast

MAY 18, 2021

Summary Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Data Engineering Podcast

APRIL 3, 2022

This allows everyone in the business to participate in data analysis in a sustainable manner. If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100