Data Engineering Podcast

MAY 14, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. What do you have planned for the future of your data platform? The Machine Learning Podcast helps you go from idea to production with machine learning.

Data Lake 147

Data Lake Machine Learning Data Warehouse AWS 147

Data Engineering Podcast

MAY 7, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Can you describe what Segment Unify is and the story behind it? What are the net-new capabilities that it brings to the Segment product suite?

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Data Engineering Podcast

APRIL 16, 2023

As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Who are the target users?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Knowledge Hut

MAY 3, 2024

Spark Streaming Vs Kafka Stream Now that we have understood high level what these tools mean, it’s obvious to have curiosity around differences between both the tools. ETL Transformation In this case, Kafka doesn't offer only ETL services. The demand for stream processing is increasing every day in today’s era.

Kafka 98

Kafka Scala Java Amazon Web Services 98

Data Engineering Podcast

JUNE 11, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Can you describe what Datafold is and what's new since we last spoke? How does the tooling used contribute to/help address those roadblocks?

Project 130

Project Building Data Lake Machine Learning 130

Data Engineering Podcast

AUGUST 13, 2023

In this episode Ariel Pohoryles explains what they are and how they work together to increase your chances of success. Can you start by defining what you mean by a "modern" data pipeline? What are the technical components of a pipeline infrastructure that are necessary to support a "modern" workflow?

Data Pipeline 130

Data Pipeline BI SQL Business Intelligence 130

Data Engineering Podcast

JUNE 4, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Can you describe what Agile Data Engine is and the story behind it? What are some of the types of experiments that are enabled by reduced operational overhead?

Data Engineering 130

Data Engineering Data Engineer Engineering Data Lake 130

Christophe Blefari

MARCH 2, 2024

I'll give a talk—in French—about how to put in production machine learning at a small scale. I'll give a talk—in French—about how to put in production machine learning at a small scale. This is a 1h podcast, I hope you will enjoy listening to it. Mistral perdant. It has to be put in context.

Data 162

Data Python Database BI 162

Data Engineering Podcast

FEBRUARY 5, 2023

What do you have planned for the future of the podcast? Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? The Machine Learning Podcast helps you go from idea to production with machine learning. Closing Announcements Thank you for listening!

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

Data Engineering Podcast

JANUARY 21, 2024

Can you describe what the role of the CDP is in the context of a businesses data ecosystem? What are the core technical challenges associated with building and maintaining a CDP? What are the core technical challenges associated with building and maintaining a CDP? Data projects are notoriously complex.

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

JULY 9, 2023

Summary For business analytics the way that you model the data in your warehouse has a lasting impact on what types of questions can be answered quickly and easily. Can you describe what entity-centric modeling (ECM) is and the story behind it? dbt vs. informatica vs. ETL scripts, etc.) When is ECM the wrong choice?

Database-centric 130

Database-centric Machine Learning SQL Data Engineering 130

Data Engineering Podcast

MARCH 10, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Can you describe what NetSpring is and the story behind it? What are the most interesting, innovative, or unexpected ways that you have seen NetSpring used?

Data Warehouse 130

Data Warehouse Data Lake Machine Learning Data Science 130

Christophe Blefari

SEPTEMBER 28, 2023

What are the current trends and why are people fighting around the concept of the modern data stack. Let's go through the current state of data to understand what you should do next. With a handy warehouse at the center and multiple SaaS tools revolving around to answer useful—sometimes not—use-cases.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Data Engineering Podcast

DECEMBER 17, 2023

Summary The "modern data stack" promised a scalable, composable data platform that gave everyone the flexibility to use the best tools for every job. The team at 5X understand the pain involved and the barriers to productivity and set out to solve it by pre-integrating the best tools from each layer of the stack.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

DECEMBER 28, 2022

Summary With all of the messaging about treating data as a product it is becoming difficult to know what that even means. No more shipping and praying, you can now know exactly what will change in your database! Can you describe what your definition of a "data product" is? or any other destination you choose.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

APRIL 2, 2023

Can you describe what the MAD landscape report is and the story behind it? Can you describe what the MAD landscape report is and the story behind it? At a high level, what is your goal in the compilation and maintenance of your landscape document? What are your guidelines for what to include in the landscape?

Hadoop 130

Hadoop Machine Learning Python Architecture 130

Knowledge Hut

MAY 3, 2024

Introduction Before getting into the fundamentals of Apache Spark, let’s understand What really is ‘Apache Spark’ is? It provides one execution model for all tasks and hence very easy for developers to learn and they can work with multiple APIs easily. Following is the authentic one-liner definition.

Scala 98

Scala Hadoop Healthcare Big Data 98

Data Engineering Podcast

DECEMBER 25, 2022

Summary Encryption and security are critical elements in data analytics and machine learning applications. No more shipping and praying, you can now know exactly what will change in your database! Can you describe what you are building at Opaque Systems and the story behind it? or any other destination you choose.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Data Engineering Podcast

NOVEMBER 13, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. With the rise in cloud platforms and self-serve data technologies the barrier of entry is dropping.

Building 130

Building Metadata MongoDB MySQL 130

Towards Data Science

NOVEMBER 4, 2023

Platform Specific Tools and Advanced Techniques Photo by Christopher Burns on Unsplash The modern data ecosystem keeps evolving and new data tools emerge now and then. What is it? What is required from the technological point of view to enable operational excellence? Are your data pipelines efficient?

Data Engineering 86

Data Engineering Data Engineer Engineering BI 86

Data Engineering Podcast

NOVEMBER 6, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. And don’t forget to thank them for their continued support of this show!

Building 130

Building IT Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 23, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Can you describe what AgileData is and the story behind it?

Project 130

Project Metadata MongoDB MySQL 130

Data Engineering Podcast

OCTOBER 30, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Can you describe what Sisense is and the story behind it?

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Data Engineering Podcast

JUNE 19, 2022

Summary Metadata is the lifeblood of your data platform, providing information about what is happening in your systems. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool.

Metadata 130

Metadata MongoDB Scala MySQL 130

Knowledge Hut

MAY 2, 2024

Apache Spark vs MapReduce After getting off hangover about how Apache Spark and MapReduce work, we need to understand how these two technologies compare with each other, and what are their pros and cons, so as to get a clear understanding of which technology fits our use case. It is surprising to know how much data is generated every minute.

Scala 94

Scala Hadoop Datasets Java 94

Data Engineering Podcast

DECEMBER 26, 2021

In this episode Zhamak re-joins the show to discuss the real world benefits that have been seen, the lessons that she has learned while working with her clients and the community, and her vision for the future of the data mesh. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own.

BI 130

BI Data Warehouse Data Engineering Data Engineer 130

Data Engineering Podcast

DECEMBER 21, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. Can you describe what Cube is and the story behind it? What are the main use cases and platform architectures that you are focused on?

Data Analytics 130

Data Analytics BI Computer Science SQL 130

Data Engineering Podcast

DECEMBER 12, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. Summary Spark is a powerful and battle tested framework for building highly scalable data pipelines.

Building 130

Building Data Pipeline BI Data Warehouse 130

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. What is Hadoop? Who are the data engineers?

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Christophe Blefari

APRIL 8, 2023

The only normalisation I did was back at the engineering school while learning SQL with Normal Forms. Actually what I cared was physical storage, data formats, logical partitioning or indexing. A data engineer should still be a software engineer working with data, empowering others with tooling and apps.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Christophe Blefari

APRIL 8, 2023

The only normalisation I did was back at the engineering school while learning SQL with Normal Forms. Actually what I cared was physical storage, data formats, logical partitioning or indexing. A data engineer should still be a software engineer working with data, empowering others with tooling and apps.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Snowflake

SEPTEMBER 13, 2023

Organizations can now leverage Salesforce data directly in Snowflake via zero-ETL data sharing to accelerate decision-making and help streamline business processes. This provides a reliable, secure and highly scalable zero-ETL data-sharing pattern. This provides a reliable, secure and highly scalable zero-ETL data-sharing pattern.

Finance 105

Finance AWS Accessible Accessibility 105

Data Engineering Podcast

JANUARY 7, 2022

Summary Reverse ETL is a product category that evolved from the landscape of customer data platforms with a number of companies offering their own implementation of it. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm.

ETL System 100

ETL System Data Pipeline Data Warehouse Architecture 100

Monte Carlo

MARCH 28, 2024

So what exactly IS a DataOps engineer? We’ll also be covering what they do day to day, what sort of salaries they command, and how individuals might go about pivoting to become a DataOps engineer. What does a DataOps engineer do? What is the average salary for a DataOps engineer? It depends!

Pipeline-centric 75

Pipeline-centric Engineering BI Google Cloud 75

Monte Carlo

JANUARY 23, 2024

One of the first data engineers at Facebook and Airbnb, he wrote and open sourced the wildly popular orchestrator, Apache Airflow , followed shortly thereafter by Apache Superset , a data exploration tool that’s taking the data viz landscape by storm. What is a data engineer today? What do data engineers do? But maybe that’s OK.”

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

ProjectPro

FEBRUARY 8, 2023

Do ETL and data integration activities seem complex to you? In 2023, more than 5140 businesses worldwide have started using AWS Glue as a big data tool. The popular data integration tool, AWS Glue, enables data analytics users to quickly acquire, analyze, migrate, and integrate data from multiple sources. billion by 2026?

AWS 98

AWS Scala Metadata Data Lake 98

Data Engineering Podcast

NOVEMBER 4, 2021

A natural outgrowth of that capability is the more recent growth of reverse ETL systems that use those analytics to feed back into the operational systems used to engage with the customer. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? No more scripts, just SQL.

Data Warehouse 100

Data Warehouse Business Intelligence ETL System Data Lake 100