Data Engineering Podcast

MARCH 24, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

FEBRUARY 4, 2024

Summary Stream processing systems have long been built with a code-first design, adding SQL as a layer on top of the existing framework. In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Starburst : ![Starburst

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Engineering Podcast

FEBRUARY 18, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

JANUARY 28, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. SIEM) A query engine is useless without data to analyze. What are the data acquisition paths/sources that you are designed to work with?-

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

MARCH 31, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you describe what Elementary is and how it is designed to enhance the development and maintenance work in dbt projects? How is Elementary designed/implemented?

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

DECEMBER 24, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

FEBRUARY 25, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MARCH 10, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

The Modern Data Company

MAY 22, 2024

A Modern Data Strategy for Enterprises – 2nd Edition Stuck with unusable data? Take a data products approach and create secure, high-quality data packages designed for specific business needs. Imagine data like department-specific apps – easily accessible and relevant.

High Quality Data 52

High Quality Data Data Designing Accessible 52

Data Engineering Podcast

MARCH 3, 2024

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

DECEMBER 17, 2023

Summary The "modern data stack" promised a scalable, composable data platform that gave everyone the flexibility to use the best tools for every job. Data lakes are notoriously complex. What are the points of friction that teams run into when trying to build their data platform? Data lakes are notoriously complex.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

NOVEMBER 26, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Data Engineering Podcast

DECEMBER 10, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Data Engineering Podcast

NOVEMBER 19, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Project 147

Project Data Lake High Quality Data SQL 147

Data Engineering Podcast

NOVEMBER 5, 2023

Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

Data Engineering Podcast

NOVEMBER 12, 2023

If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Data lakes are notoriously complex. Can you describe the structure and implementation of Tabnine?

Software Engineer 147

Software Engineer Software Engineering Engineering Data Lake 147

Snowflake

APRIL 11, 2024

This comprehensive functionality streamlines your workflow and enables high-quality data analyses. Seamless AI within Snowflake Copilot is a complete solution designed to seamlessly integrate with your existing Snowflake workflow.

SQL 137

SQL AWS Data Analysis High Quality Data 137

U-Next

OCTOBER 27, 2022

Many marketing teams struggle to deal with massive data sets and produce insights because they lack personnel with data science and AI skills. Thus, it’s high time that aspiring individuals acquire the right Data Science skills and enrich the workforce. . Maintain High-Quality Data.

Designing 40

Designing Machine Learning Media Digital Media 40

Snowflake

OCTOBER 26, 2023

Novel compounds designed with desired properties, accelerating drug discovery. Realistic synthetic data created at scale, expediting research in rare under-addressed disease areas. A patient interaction turned into clinician notes in seconds, increasing patient engagement and clinical efficiency.

Healthcare 85

Healthcare Unstructured Data High Quality Data Medical 85

Data Engineering Weekly

MAY 19, 2024

Data value depends on the users and the use cases Data quality is multi-dimensional, and high-quality data costs more. An Empirical Evaluation of Columnar Storage Formats paper compares ORC and Parquet formats regarding efficiency, design choices, and what is lacking in that system.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

The Modern Data Company

JULY 18, 2023

Data Products Data products encompass several key aspects that contribute to their effectiveness and value in addressing data challenges and delivering actionable insights. These aspects ensure that data products are well-designed, user-centric, and aligned with business goals.

Data Governance 69

Data Governance High Quality Data Government Data 69

Precisely

MARCH 14, 2023

Read Quality data you can depend on – today, tomorrow, and beyond For many years Precisely customers have ensured the accuracy of data across their organizations by leveraging our leading data solutions including Trillium Quality, Spectrum Quality, and Data360 DQ+. What does all this mean for your business?

Data Integration 52

Data Integration High Quality Data BI Data 52

Data Engineering Weekly

APRIL 14, 2024

Discover how a universal semantic layer is transforming modern business intelligence, making data more accessible and reliable for organizations striving for informed business decisions. The highlight of the design is the unique blending of batch and real-time events with the common feature definition.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

Data Engineering Weekly

MARCH 3, 2024

Here is the agenda, 1) Data Application Lifecycle Management - Harish Kumar( Paypal) Hear from the team in PayPal on how they build the data product lifecycle management (DPLM) systems. 3) DataOPS at AstraZeneca The AstraZeneca team talks about data ops best practices internally established and what worked and what didn’t work!!!

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Data Engineering Podcast

FEBRUARY 20, 2022

Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. With the Oxylabs scraper APIs you can extract data from even javascript heavy websites. Who are the target users for Fugue and how does that influence the feature priorities and API design?

Python 100

Python Data Process IT Building 100

Monte Carlo

FEBRUARY 27, 2024

Organizations need to connect LLMs with their proprietary data and business context to actually create value for their customers and employees. They need robust data pipelines, high-quality data, well-guarded privacy, and cost-effective scalability. Data engineers. Who can deliver?

Data Engineering 59

Data Engineering Data Engineer Engineering High Quality Data 59

Databand.ai

MAY 30, 2023

Data quality refers to the degree of accuracy, consistency, completeness, reliability, and relevance of the data collected, stored, and used within an organization or a specific context. High-quality data is essential for making well-informed decisions, performing accurate analyses, and developing effective strategies.

Data Cleanse 85

Data Cleanse Datasets Data Governance Data Validation 85

The Modern Data Company

JULY 18, 2023

Data Products Data products encompass several key aspects that contribute to their effectiveness and value in addressing data challenges and delivering actionable insights. These aspects ensure that data products are well-designed, user-centric, and aligned with business goals.

Data Governance 52

Data Governance High Quality Data Government Data 52

The Modern Data Company

JULY 18, 2023

Data Products Data products encompass several key aspects that contribute to their effectiveness and value in addressing data challenges and delivering actionable insights. These aspects ensure that data products are well-designed, user-centric, and aligned with business goals.

Data Governance 52

Data Governance High Quality Data Government Data 52

Snowflake

MAY 14, 2024

Long established in the adtech domain, these principles are now gaining momentum in the life sciences and healthtech data provider ecosystem, where collaborations and access to high-quality data are pivotal in the pursuit of targeted therapies.

Pharmaceutical 77

Pharmaceutical IT Hospitality Data 77

Data Engineering Weekly

APRIL 2, 2023

There are some interesting threads on Twitter, but the highlight for me is the design of the Tweet search system. The cluster split approach to store real-time, protected, and archive tweets is an excellent reference model for designing enterprise search engines.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Data Engineering Podcast

JUNE 29, 2020

This was a great conversation about the complexities of working in a niche domain of data analysis and how to build a pipeline of high quality data from collection to analysis. Can you describe how your overall data management system is architected?

Data Collection 100

Data Collection Management High Quality Data Metadata 100

Picnic Engineering

DECEMBER 12, 2023

The rich context provided by our Snowflake-powered Data Warehouse enhances their performance, allowing us to create a robust feature set for training. Training these models on our historical demand and assessing their performance is manageable as a one-time project since concepts like data drift are still not a big concern.

Machine Learning 52

Machine Learning Deep Learning Food High Quality Data 52

Ripple Engineering

JANUARY 29, 2024

  A lack of a centralized system makes building a single source of high-quality data difficult. The key aspect of any business-centric team in delivering products and features is to make critical decisions on ensuring low latency, high throughput, cost-effective storage, and highly efficient infrastructure.

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

Databand.ai

AUGUST 30, 2023

Benefits of a Data Quality Strategy Implementing a robust data quality strategy offers numerous benefits that directly impact your business’s bottom line and overall success. Additionally, high-quality data reduces costly errors stemming from inaccurate information.

Building 52

Building Data Cleanse Data Governance Datasets 52

Knowledge Hut

APRIL 25, 2024

Big data provides the foundation for machine learning by supplying the large and diverse datasets that machine learning algorithms require to train and improve their models. Pattern Recognition Machine learning algorithms are specifically designed to recognize and learn patterns from data.

Machine Learning 52

Machine Learning Big Data Unstructured Data Data Mining 52

Airbnb Tech

NOVEMBER 28, 2023

Starting in 2020, the Midas process, along with the work to re-architect our most critical data models, has brought a dramatic increase in data quality and timeliness to Airbnb’s most critical data. While this made sense for our most critical data, pursuing such rigorous standards at scale presented challenges.

Data Warehouse 98

Data Warehouse Metadata Data Certification 98