Data Engineering Podcast

APRIL 14, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Knowledge Hut

MARCH 7, 2024

In the digital era, the demand for cloud computing has increased like never before. Increased security, scalability, reduced costs, and better collaboration are a few benefits of cloud computing. That is why the need for cloud computing companies has increased a lot. It is one of the safest platforms for cloud service.

Cloud Computing 98

Cloud Computing Amazon Web Services Cloud Google Cloud 98

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

MARCH 31, 2024

Summary Working with data is a complicated process, with numerous chances for something to go wrong. Identifying and accounting for those errors is a critical piece of building trust in the organization that your data is accurate and up to date. Dagster offers a new approach to building and running data platforms and data pipelines.

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

FEBRUARY 11, 2024

Summary Sharing data is a simple concept, but complicated to implement well. There are also numerous technical considerations to be made, particularly if the producer and consumer of the data aren't using the same platforms. Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Knowledge Hut

JANUARY 29, 2024

It is impossible to escape from technology in today’s modern world. Some techniques add to the development of technology in the business sectors, including Data Science and Cloud Computing, essential aspects of the technology industry. The term cloud is referred to as a metaphor for the internet.

Cloud Computing 98

Cloud Computing Data Science Cloud Amazon Web Services 98

Data Engineering Podcast

MARCH 3, 2024

Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin. Your first 30 days are free!

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Data lakes are notoriously complex. Go to dataengineeringpodcast.com/dagster today to get started.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MARCH 10, 2024

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. The link that bridges the gap between data lake and warehouse capabilities is the catalog. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Data Engineering Podcast

MARCH 17, 2024

Summary A significant portion of data workflows involve storing and processing information in database engines. In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data. Data lakes are notoriously complex.

Database 147

Database Data Lake High Quality Data Data Workflow 147

Knowledge Hut

APRIL 26, 2024

In the digital era, the demand for cloud computing has increased like never before. Increased security, scalability, reduced costs, and better collaboration are a few benefits of cloud computing. That is why the need for cloud computing companies has increased a lot. It is one of the safest platforms for cloud service.

Cloud Computing 52

Cloud Computing Amazon Web Services Cloud Google Cloud 52

Data Engineering Podcast

OCTOBER 29, 2023

Summary Databases are the core of most applications, whether transactional or analytical. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. With Materialize, you can!

Database 189

Database BI SQL Machine Learning 189

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. This offers a single location for managing visibility and error handling so that data platform engineers can manage complexity. With Materialize, you can!

BI 208

BI SQL Data Machine Learning 208

Knowledge Hut

APRIL 23, 2024

Cloud computing is an important way of storage that has taken over the internet and is now considered to be one biggest acquisition with a transaction volume of over $10 billion. Several cloud computing services examples have taken over the internet and have been extremely useful for people.

Cloud Computing 52

Cloud Computing Cloud Amazon Web Services AWS 52

Data Engineering Podcast

OCTOBER 22, 2023

Summary The primary application of data has moved beyond analytics. With the broader audience comes the need to present data in a more approachable format. This has led to the broad adoption of data products being the delivery mechanism for information. With Materialize, you can!

BI 162

BI SQL Machine Learning Programming Language 162

Data Engineering Podcast

OCTOBER 15, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Process 182

Process Building SQL BI 182

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor.

Data Governance 130

Data Governance Government Cloud Building 130

Data Engineering Podcast

SEPTEMBER 17, 2023

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. With Materialize, you can! Go to dataengineeringpodcast.com/materialize today to get 2 weeks free!

Building 189

Building BI SQL Python 189

Data Engineering Podcast

NOVEMBER 5, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

Data Engineering Podcast

NOVEMBER 12, 2023

Tabnine is one of the main platforms offering an AI powered assistant for software engineers. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. You shouldn't have to throw away the database to build with fast-changing data.

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

The Modern Data Company

FEBRUARY 28, 2023

Is Your Head Too High up in the Cloud? There is no doubt that the cloud is here to stay and that it will be a part of every company’s future data and analytics strategy. Charging for every CPU cycle and every piece of disk storage used is central to cloud models. On the surface, this sounds great.

Cloud 98

Cloud Hadoop Technology Coding 98

Data Engineering Podcast

AUGUST 27, 2023

Summary Data persistence is one of the most challenging aspects of computer systems. In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products.

Database 130

Database Building PostgreSQL BI 130

Data Engineering Podcast

DECEMBER 3, 2023

Summary The first step of data pipelines is to move the data to a place where you can process and prepare it for its eventual purpose. Data transfer systems are a critical component of data enablement, and building them to support large volumes of information is a complex endeavor. Data lakes are notoriously complex.

Systems 130

Systems Designing Data Lake SQL 130

Data Engineering Podcast

MAY 22, 2022

Summary Machine learning has become a meaningful target for data applications, bringing with it an increase in the complexity of orchestrating the entire data flow. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform.

Machine Learning 130

Machine Learning Data Engineering Data Engineer Cloud 130

Data Engineering Podcast

SEPTEMBER 3, 2023

Summary Cloud data warehouses and the introduction of the ELT paradigm has led to the creation of multiple options for flexible data integration, with a roughly equal distribution of commercial and open source options. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products.

Data Integration 162

Data Integration BI SQL Python 162

Data Engineering Podcast

MAY 29, 2022

Summary The latest generation of data warehouse platforms have brought unprecedented operational simplicity and effectively infinite scale. In order to ensure that you can explore and analyze your data without spending money on inefficient queries Mingsheng Hong and Zheng Shao created Bluesky Data.

Cloud 100

Cloud Data Warehouse Data Lake Data 100

Data Engineering Weekly

MAY 5, 2024

In an ideal world, data should be thought of as “borrowed” (possibly unpermitted) and thus can be “returned,” and unlearning should enable such revocation. Uber wrote an in-depth article about the evolution of its centralized ML platform, Michelangelo.

Data Engineering 67

Data Engineering Data Engineer Engineering Google Cloud 67

Data Engineering Podcast

OCTOBER 8, 2023

In this episode he shares his journey of data collection and analysis and the challenges of automating an intentionally manual industry. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. With Materialize, you can!

Insurance 162

Insurance BI SQL Machine Learning 162

Data Engineering Podcast

SEPTEMBER 24, 2023

In this episode Louis Brandy discusses the applications for vector search capabilities both in and outside of AI, as well as the challenges of maintaining real-time indexes of vector data. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

BI 147

BI SQL Machine Learning Python 147

Data Engineering Podcast

MAY 1, 2022

Summary The predominant pattern for data integration in the cloud has become extract, load, and then transform or ELT. Matillion was an early innovator of that approach and in this episode CTO Ed Thompson explains how they have evolved the platform to keep pace with the rapidly changing ecosystem.

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Data Engineering Podcast

OCTOBER 1, 2023

Summary Artificial intelligence applications require substantial high quality data, which is provided through ETL pipelines. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. With Materialize, you can! With Materialize, you can!

Building 162

Building BI SQL Machine Learning 162

Data Engineering Podcast

FEBRUARY 27, 2022

Summary There are a wealth of options for managing structured and textual data, but unstructured binary data assets are not as well supported across the ecosystem. As organizations start to adopt cloud technologies they need a way to manage the distribution, discovery, and collaboration of data across their operating environments.

Unstructured Data 100

Unstructured Data Cloud Management Metadata 100

Cloudera

APRIL 26, 2023

Data is core to decision making today and organizations often turn to the cloud to build modern data apps for faster access to valuable insights. With cloud operating models, decision making can be accelerated, leading to competitive advantages and increased revenue. What is cloud native exactly?

Cloud 64

Cloud Building Utilities Architecture 64

Data Engineering Podcast

AUGUST 20, 2023

In this episode Greg Werner explains how they are using generative AI as an assistive tool for creating educational material, as well as building a data driven experience for learners. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Education 130

Education BI SQL Database 130

Data Engineering Podcast

JUNE 11, 2021

Summary Google pioneered an impressive number of the architectural underpinnings of the broader big data ecosystem. Now they offer the technologies that they run internally to external users of their cloud platform. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads?

Google Cloud 100

Google Cloud Cloud Big Data Ecosystem BI 100

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. The idea is to create a living reference about Data Engineering.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Cloudera

FEBRUARY 2, 2024

Forrester Research recently released the Forester Wave for Streaming Platforms, Q4 2023. This score was stronger than anyone outside of one-cloud vendors Microsoft and Google, including a stronger current offering than Confluent. We believe our hybrid strategy will pay off as well.

Kafka 99

Kafka Architecture Data Architecture Cloud 99