Data Engineering Podcast

NOVEMBER 27, 2022

He also discusses the improvements that have been incorporated into FeatureBase to simplify integration with the rest of your data stack, and the SQL interface that was added to make working with the product easier. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

DECEMBER 4, 2022

In this episode Arjun Narayan explains how the technical barriers to adopting real-time data in your analytics and applications have become surmountable by organizations of all sizes. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. or any other destination you choose.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

DECEMBER 11, 2022

Summary One of the most critical aspects of software projects is managing its data. Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. or any other destination you choose.

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

OCTOBER 30, 2022

In this episode he shares his experiences working with organizations to adopt analytics engineering patterns and the ways that Optimus and dbt were combined to let data analysts deliver insights without the roadblocks of complex pipeline management. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode she shares the story behind the project, the details of how it is implemented, and how you can use it for your own data projects. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Podcast

NOVEMBER 20, 2022

In this episode CEO and founder Salma Bakouk shares her views on the causes and impacts of "data entropy" and how you can tame it before it leads to failures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. What is the motivating goal for the company and product?

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

Data Engineering Podcast

APRIL 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Missing data?

Data Warehouse 100

Data Warehouse Data Lake Data Pipeline BI 100

Data Engineering Podcast

APRIL 3, 2022

Summary The flexibility of software oriented data workflows is useful for fulfilling complex requirements, but for simple and repetitious use cases it adds significant complexity. Coalesce is a platform designed to reduce repetitive work for common workflows by adopting a visual pipeline builder to support your data warehouse transformations.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

APRIL 17, 2022

In this episode Prineha Narang, co-founder and CTO of Aliro, explains how these systems work, the capabilities that they can offer, and how you can start preparing for a post-quantum future for your data systems. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks.

Data Warehouse 100

Data Warehouse SQL Data Engineering Data Engineer 100

Data Engineering Podcast

JUNE 26, 2022

In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

JULY 10, 2022

They also discuss how they have established a guild system for training and supporting data professionals in the organization. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

MAY 18, 2021

Summary Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

MARCH 16, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building Data Warehouse NoSQL Data Lake 100

Data Engineering Podcast

OCTOBER 2, 2021

If you need to deal with massive data, at high velocities, in milliseconds, then Aerospike is definitely worth learning about. If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Building 100

Building BI Data Architecture Architecture 100

Data Engineering Podcast

MAY 25, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Database 100

Database Building Data Warehouse Machine Learning 100

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

MARCH 27, 2022

He explains how they redesigned the core algorithms and storage management features to deliver ten times faster throughput, how the lower latencies work to reduce the burden on platform engineers, and how they are working toward an open source offering so that you can try it yourself with no friction. What is your target market and customer?

Relational Database 100

Relational Database Database SQL Data Warehouse 100

Data Engineering Podcast

APRIL 10, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Integration 100

Data Integration SQL Data Engineering Data Engineer 100

Data Engineering Podcast

FEBRUARY 24, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

IT 100

IT Data Lake Data Pipeline Media 100

Data Engineering Podcast

DECEMBER 21, 2020

If you are starting down the path of implementing a data governance strategy then this episode will provide a great overview of what is involved. If you hand a book to a new data engineer, what wisdom would you add to it? What is data governance? If you hand a book to a new data engineer, what wisdom would you add to it?

Data Governance 130

Data Governance Government Data Lake Banking 130

Snowflake

APRIL 24, 2023

However, this has resulted in dated systems that cause workflow inefficiencies, and data and technology silos that add to cost and complexity. Data management becomes increasingly manual, creating elongated data pipelines, delayed analytics, and greater potential for error.

Management 52

Management Portfolio Data Ingestion Data Warehouse 52

Cloudera

DECEMBER 20, 2023

Those models are trained or augmented with data from a data management platform. The data management platform, models, and end applications are powered by cloud infrastructure and/or specialized hardware. More information can be found in our blog post here.

Amazon Web Services 71

Amazon Web Services AWS Machine Learning Datasets 71

Data Engineering Podcast

JUNE 12, 2022

In this episode she shares her thoughts and insights on how to be intentional about establishing your own data team. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

MARCH 2, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Kafka 100

Kafka Process PostgreSQL MySQL 100

Data Engineering Podcast

JANUARY 4, 2021

Summary As more organizations are gaining experience with data management and incorporating analytics into their decision making, their next move is to adopt machine learning. Monte Carlo’s end-to-end Data Observability Platform monitors and alerts for data issues across your data warehouses, data lakes, ETL, and business intelligence.

Python 100

Python Data Lake Machine Learning Computer Science 100

Data Engineering Podcast

JANUARY 11, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

PostgreSQL 100

PostgreSQL Python Data Lake Computer Science 100

Data Engineering Podcast

DECEMBER 16, 2019

Even if you already have a metadata repository this is worth a listen to learn more about the value that visibility of your data can bring to your organization. You work hard to make sure that your data is clean, reliable, and reproducible throughout the ingestion pipeline, but what happens when it gets to the data warehouse?

Metadata 100

Metadata PostgreSQL Datasets Data Warehouse 100

Monte Carlo

FEBRUARY 20, 2024

Choosing one tool over another isn’t just about the features it offers today; it’s a bet on the future of how data will flow within organizations. Consideration What to Look For Integration Capabilities Support for a diverse array of data sources and destinations, ensuring compatibility with your data ecosystem.

Data Ingestion 52

Data Ingestion Google Cloud Kafka AWS 52

Cloudera

JANUARY 20, 2021

Most of what is written though has to do with the enabling technology platforms (cloud or edge or point solutions like data warehouses) or use cases that are driving these benefits (predictive analytics applied to preventive maintenance, financial institution’s fraud detection, or predictive health monitoring as examples) not the underlying data.

Manufacturing 107

Manufacturing Data Warehouse Kafka Retail 107

Striim

FEBRUARY 10, 2023

The company decided to move to the cloud based on the benefits of cost efficiency, flexibility, and improved data management. To facilitate the data integration process, it used Striim, which allowed it to: Import historical and real-time on-premise data from its Oracle and DB2 mainframe databases.

Data Analytics 52

Data Analytics Telecommunication Insurance Google Cloud 52

AltexSoft

MARCH 30, 2023

What is Databricks Databricks is an analytics platform with a unified set of tools for data engineering, data management , data science, and machine learning. It combines the best elements of a data warehouse, a centralized repository for structured data, and a data lake used to host large amounts of raw data.

Scala 64

Scala Data Lake BI Google Cloud 64

AltexSoft

OCTOBER 8, 2021

The bad news is, integrating data can become a tedious task, especially when done manually. Luckily, there are various data integration tools that support automation and provide a unified data view for more efficient data management. Data integration process. So, why is data integration such a big deal?

Data Integration 52

Data Integration Hadoop Data Warehouse Data Lake 52

Monte Carlo

APRIL 20, 2023

Solutions with automated data lineage capabilities constantly update these graphs and illustrate them as nodes and edges, or in other words, the objects through which the data travels and the relationship between them. This is one of the most frequent data lineage use cases leveraged by Vox. Data lineage can help!

Data Warehouse 52

Data Warehouse BI Data Government 52

AltexSoft

SEPTEMBER 10, 2021

What is data integration and why is it important? Data integration is the process of taking data from multiple disparate internal and external sources and putting it in a single location (e.g., data warehouse ) to achieve a unified view of collected data. Key types of data integration.

Data Integration 52

Data Integration Data Consolidation Data Warehouse BI 52