Data Engineering Podcast

APRIL 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

JANUARY 21, 2024

Summary Databases and analytics architectures have gone through several generational shifts. A substantial amount of the data that is being managed in these systems is related to customers and their interactions with an organization.

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

NOVEMBER 5, 2023

Summary Databases are the core of most applications, but they are often treated as inscrutable black boxes. When an application is slow, there is a good probability that the database needs some attention. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

Data Engineering Podcast

NOVEMBER 26, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. With Materialize, you can!

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Data Engineering Podcast

MARCH 31, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Datafold has recently launched data replication testing, providing ongoing validation for source-to-target replication.

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

DECEMBER 31, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data.

Designing 130

Designing Data Lake High Quality Data SQL 130

Data Engineering Podcast

JANUARY 7, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

Data Process 147

Data Process Process Data Lake High Quality Data 147

Data Engineering Podcast

DECEMBER 17, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. With Materialize, you can!

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

DECEMBER 3, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. With Materialize, you can!

Systems 130

Systems Designing Data Lake SQL 130

Data Engineering Podcast

NOVEMBER 19, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack You shouldn't have to throw away the database to build with fast-changing data. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. With Materialize, you can!

Project 147

Project Data Lake High Quality Data SQL 147

Data Engineering Podcast

DECEMBER 10, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You shouldn't have to throw away the database to build with fast-changing data. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

NOVEMBER 12, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Data lakes are notoriously complex. You shouldn't have to throw away the database to build with fast-changing data.

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

Data Engineering Podcast

OCTOBER 1, 2023

Summary Artificial intelligence applications require substantial high quality data, which is provided through ETL pipelines. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. With Materialize, you can!

Building 162

Building BI SQL Machine Learning 162

Monte Carlo

APRIL 1, 2024

Implement Routine Data Audits Build a data cleaning cadence into your data teams’ schedule. Routine data quality checks will not only help to reduce the risk of discrepancies in your data, but it will also help to fortify a culture of high-quality data throughout your organization.

High Quality Data 52

High Quality Data Datasets Data Data Pipeline 52

Monte Carlo

FEBRUARY 27, 2024

Organizations need to connect LLMs with their proprietary data and business context to actually create value for their customers and employees. They need robust data pipelines, high-quality data, well-guarded privacy, and cost-effective scalability. Data engineers. Who can deliver?

Data Engineering 59

Data Engineering Data Engineer Engineering High Quality Data 59

Monte Carlo

MARCH 7, 2024

Why prompt engineering isn’t all that and a bag of SQL queries Understand vector databases Create AI differentiation with RAG Find and solve real business problems High-quality data always lives up to the hype What is prompt engineering? Table of Contents Why is prompt engineering important?

Engineering 59

Engineering High Quality Data Database Architecture 59

Databand.ai

MAY 30, 2023

Data quality refers to the degree of accuracy, consistency, completeness, reliability, and relevance of the data collected, stored, and used within an organization or a specific context. High-quality data is essential for making well-informed decisions, performing accurate analyses, and developing effective strategies.

Data Cleanse 85

Data Cleanse Datasets Data Governance Data Validation 85

Monte Carlo

JANUARY 3, 2024

But RAG development comes with a learning curve, even for your most talented data engineers. They need to know prompt engineering , vector databases and embedding vectors , data modeling, data orchestration , data pipelines and all for RAG. away from your data infrastructure being GenAI ready.

Technology 113

Technology Database Data Governance Data Engineering 113

Striim

APRIL 30, 2024

Striim’s Solution Kramp adopted Striim for its powerful, mature real-time data integration, seamlessly connecting diverse databases like Oracle, Microsoft, and Postgres, to ensure continuous, high-quality data replication essential for forecasting and order management.

Google Cloud 52

Google Cloud High Quality Data Business Intelligence Data Warehouse 52

Databand.ai

AUGUST 30, 2023

Data Consistency vs Data Integrity: Similarities and Differences Joseph Arnold August 30, 2023 What Is Data Consistency? Data consistency refers to the state of data in which all copies or instances are the same across all systems and databases.

Data Integration 52

Data Integration Data Cleanse Data Validation High Quality Data 52

Monte Carlo

DECEMBER 22, 2023

She also explores the key considerations for organizations looking to implement these use cases, including vector databases, fine-tuning models, and unstructured or streaming data processing. Organizing Generative AI: 5 Lessons Learned From Data Science Teams So, your business leadership is all in on generative AI?

High Quality Data 52

High Quality Data Data Science Data Engineering Data Engineer 52

Monte Carlo

DECEMBER 22, 2023

She also explores the key considerations for organizations looking to implement these use cases, including vector databases, fine-tuning models, and unstructured or streaming data processing. Organizing Generative AI: 5 Lessons Learned From Data Science Teams So, your business leadership is all in on generative AI?

High Quality Data 52

High Quality Data Data Science Data Engineering Data Engineer 52

Ripple Engineering

JANUARY 29, 2024

For Ripple's product capabilities, the Payments team of Ripple, for example, ingests millions of transactional records into databases and performs analytics to generate invoices, reports, and other related payment operations.    A lack of a centralized system makes building a single source of high-quality data difficult.

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

Precisely

MARCH 12, 2024

In this blog post, we’ll explain what data enrichment is, why you need it, how it works, and how B2B companies can use enriched data to drive results. What is data enrichment? Third-party data could come from various sources, including social media platforms, mobile devices, and IoT devices.

Insurance 52

Insurance Telecommunication Retail High Quality Data 52

Monte Carlo

JANUARY 10, 2024

Extrinsic data, meanwhile, is more about the context — it’s how your data interacts with the world outside and how it fits into the larger picture of your project or organization. Consider a database that holds customer details. usability) would be about extrinsic data quality.

Data Cleanse 52

Data Cleanse Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

JUNE 29, 2020

This was a great conversation about the complexities of working in a niche domain of data analysis and how to build a pipeline of high quality data from collection to analysis. The team at Audio Analytic are working to impart a sense of hearing to our myriad devices with their sound recognition technology.

Data Collection 100

Data Collection Management High Quality Data Metadata 100

Monte Carlo

JANUARY 5, 2024

Ingestion layer The ingestion layer in data lakehouse architecture extracts data from various sources, including transactional and relational databases, APIs, real-time data streams, CRM applications, NoSQL databases, and more, and brings them into the data lake.

Architecture 52

Architecture Data Lake Metadata Unstructured Data 52

Monte Carlo

JANUARY 5, 2024

Ingestion layer The ingestion layer in data lakehouse architecture extracts data from various sources, including transactional and relational databases, APIs, real-time data streams, CRM applications, NoSQL databases, and more, and brings them into the data lake.

Architecture 52

Architecture Data Lake Metadata Unstructured Data 52

Precisely

MARCH 14, 2023

Read Quality data you can depend on – today, tomorrow, and beyond For many years Precisely customers have ensured the accuracy of data across their organizations by leveraging our leading data solutions including Trillium Quality, Spectrum Quality, and Data360 DQ+. What does all this mean for your business?

Data Integration 52

Data Integration High Quality Data BI Data 52

Precisely

SEPTEMBER 25, 2023

Data validation performs a check against existing values in a database to ensure that they fall within valid parameters. Data enrichment is the process of enhancing your data by appending relevant context from additional sources – improving its overall value, accuracy, and usability.

Data Validation 100

Data Validation Process Raw Data Data Cleanse 100

Monte Carlo

JANUARY 23, 2024

RAG involves integrating a real-time database into the LLM’s response generation process, while fine-tuning trains models on targeted datasets to improve domain-specific responses. That implies working with new patterns like vector databases.” Those who don’t embrace it will be left behind. With the right prompt (this is key!)

Pipeline-centric 52

Pipeline-centric Database-centric Metadata Unstructured Data 52

Data Engineering Weekly

MARCH 3, 2024

Here is the agenda, 1) Data Application Lifecycle Management - Harish Kumar( Paypal) Hear from the team in PayPal on how they build the data product lifecycle management (DPLM) systems. 3) DataOPS at AstraZeneca The AstraZeneca team talks about data ops best practices internally established and what worked and what didn’t work!!!

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Knowledge Hut

APRIL 23, 2024

Each data point provides a specific value or attribute that contributes to the overall understanding and analysis of the data. On the other hand, data sources pertain to the origins or locations from which the data is collected. Data sources provide the context and environment from which the data points are collected.

Big Data 52

Big Data Media Datasets Unstructured Data 52

Workfall

JULY 4, 2023

With DBT, they weave powerful SQL spells to create data models that capture the essence of their organization’s information. DBT’s superpowers include seamlessly connecting with databases and data warehouses, performing amazing transformations, and effortlessly managing dependencies to ensure high-quality data.

Data Warehouse 69

Data Warehouse SQL PostgreSQL Database 69

Monte Carlo

OCTOBER 5, 2023

Ensure data quality Even if there are no errors during the ETL process, you still have to make sure the data meets quality standards. High-quality data is crucial for accurate analysis and informed decision-making.

Data Cleanse 52

Data Cleanse ETL Tools Datasets High Quality Data 52

Towards Data Science

MARCH 29, 2023

Great reads on modeling, processes, and leadership Photo by Emil Widlund on Unsplash At the very start of my journey in data, I thought I was going to be a data scientist, and my first foray into data was centered on studying statistics and linear algebra, not software engineering or database management.

Data Warehouse 68

Data Warehouse BI Healthcare Database 68

Precisely

DECEMBER 19, 2022

The PropTech industry has been booming – and data holds the key to continuous transformation and competitive edge. High quality data and analytics helps PropTech companies gain deeper context on properties and locations, build richer models with accurate information, and more.

Data Governance 52

Data Governance Retail Government High Quality Data 52