Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. Atlan is the metadata hub for your data ecosystem.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

DECEMBER 4, 2022

Summary Business intelligence is the foremost application of data in organizations of all sizes. Zing Data is building a mobile native platform for business intelligence. In this episode Sabin Thomas shares his experiences building the platform and the interesting ways that it is being used.

Business Intelligence 100

Business Intelligence Metadata BI MongoDB 100

Data Engineering Podcast

OCTOBER 22, 2021

Summary The market for business intelligence has been going through an evolutionary shift in recent years. Lightdash has fully embraced that shift by building an entire open source business intelligence framework that is powered by dbt models. The data you’re looking for is already in your data warehouse and BI tools.

Engineering 100

Engineering Business Intelligence BI Data Warehouse 100

Data Engineering Podcast

OCTOBER 2, 2022

This requires a significant amount of data to track shipments and operational characteristics of materials and goods. Roambee is a platform that collects, integrates, and analyzes all of that information to provide companies with the critical insights that businesses need to stay running, especially in a time of such constant change.

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 5, 2022

Summary The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

FEBRUARY 15, 2021

Summary Every business aims to be data driven, but not all of them succeed in that effort. In order to be able to truly derive insights from the data that an organization collects, there are certain foundational capabilities that they need to have capacity for.

Building 100

Building Data Warehouse BI Data Pipeline 100

Data Engineering Podcast

APRIL 26, 2021

Summary The reason for collecting, cleaning, and organizing data is to make it usable by the organization. One of the most common and widely used methods of access is through a business intelligence dashboard. Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Business Intelligence 100

Business Intelligence Data Warehouse Hadoop Data Pipeline 100

Data Engineering Podcast

JANUARY 25, 2021

Summary Businesses often need to be able to ingest data from their customers in order to power the services that they provide. In order to reduce the friction involved in supporting new data transformations David Molot and Hassan Syyid built the Hotlue platform.

Data Integration 100

Data Integration IT Python BI 100

Data Engineering Podcast

JANUARY 18, 2021

Summary The data warehouse has become the central component of the modern data stack. Building on this pattern, the team at Hightouch have created a platform that synchronizes information about your customers out to third party systems for use by marketing and sales teams.

Data Warehouse 100

Data Warehouse BI Data Data Pipeline 100

Data Engineering Podcast

APRIL 12, 2021

Summary "Business as usual" is changing, with more companies investing in data as a first class concern. As a result, the data team is growing and introducing more specialized roles. He also talks about how his experience working with these teams informs his work at Databand.

Data Warehouse 100

Data Warehouse Data Pipeline BI Data 100

Data Engineering Podcast

MARCH 22, 2021

Summary The world of business is becoming increasingly dependent on information that is accurate up to the minute. For analytical systems, the only way to provide this reliably is by implementing change data capture (CDC). Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Warehouse 100

Data Warehouse Metadata Data Lake Hadoop 100

Data Engineering Podcast

FEBRUARY 8, 2021

Summary With all of the tools and services available for building a data platform it can be difficult to separate the signal from the noise. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify.

Data Warehouse 100

Data Warehouse Building BI SQL 100

Data Engineering Podcast

DECEMBER 4, 2021

Summary Hiring data professionals is challenging for a multitude of reasons, and as with every interview process there is a potential for bias to creep in. Tim Freestone founded Alooba to provide a more stable reference point for evaluating candidates to ensure that you can make more informed comparisons based on their actual knowledge.

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Data Engineering Podcast

MARCH 8, 2021

Summary A majority of the time spent in data engineering is copying data between systems to make the information available for different purposes. This introduces challenges such as keeping information synchronized, managing schema evolution, building transformations to match the expectations of the destination systems.

IT 100

IT Data Warehouse MongoDB Kafka 100

Ascend.io

MARCH 2, 2023

Background Audacy is a media company that owns 235 radio stations and a portfolio of podcast channels across 48 media markets. “Snowflake greatly reduced our DBA overhead and simplified our infrastructure,” said Terry in a recent interview with the Ascend team. ” Ascend is simply way more than just an ELT tool.

Data Pipeline 52

Data Pipeline Media Portfolio Data Ingestion 52

Data Engineering Weekly

APRIL 16, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make collecting data from every application, website, and SaaS platform easy, then activating it in your warehouse and business tools. Sign up free to test out the tool today. 📊 ⏱️Got 5 minutes?

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

ThoughtSpot

JANUARY 10, 2023

As I reflect back on season three of The Data Chief, I once again am so grateful for our guests who share their insights and inspire us all. In fact, The Data Chief podcast was downloaded in over 190 countries and was among Spotify’s top 5% of most shared content globally. Listeners from all over the world tuned in.

Recruitment 52

Recruitment Food Data Data Science 52

Data Engineering Weekly

NOVEMBER 27, 2022

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. Sign up free to test out the tool today.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Data Engineering Podcast

FEBRUARY 25, 2018

Summary One of the sources of data that often gets overlooked is the systems that we use to run our businesses. This data is not used to directly provide value to customers or understand the functioning of the business, but it is still a critical component of a successful system.

Kafka 100

Kafka AWS Data Pipeline Data 100

Monte Carlo

MAY 31, 2022

This article is sourced based on the interview between Lior Solomon, (now the former) VP of Engineering, Data, at Vimeo with the co-founders of Firebolt on their Data Engineering Show podcast which took place August 18, 2021. Vimeo is a leading video hosting, sharing, and services platform provider.

BI 52

BI Data Warehouse Unstructured Data Data Schemas 52

U-Next

MARCH 13, 2023

The Portal aims to feature veterans who will help them seamlessly transition into the digital space and start discussions on crucial topics, that will only make workforces across businesses future-proof. Is it the job of the entrepreneur to think, like, a product manager to think, like, a product owner, or to think like a business owner?

Business Analyst 97

Business Analyst Management Education Media 97

Rockset

AUGUST 11, 2022

I had the pleasure of recently hosting a data engineering expert discussion on a topic that I know many of you are wrestling with – when to deploy batch or streaming data in your organization’s data stack. Andreas also produces the well-known ‘Plumbers of Data Science’ blog and podcast , as well as a YouTube channel.

Bytes 52

Bytes Consulting Kafka MongoDB 52

Workfall

FEBRUARY 28, 2023

10 Ways to Increase the Effectiveness of Your Work as a ML Engineer Using ChatGPT 6 Mind-Blowing ChatGPT Extensions To Use Top 6 ChatGPT Use Cases Considerations Regarding Ethics, Laws, and Regulations Weakness of ChatGPT Benefits of ChatGPT for Businesses ChatGPT’s Role in the Future ChatGPT Pricing Conclusion What Is ChatGPT?

Datasets 62

Datasets Coding Algorithm Technology 62

Data Engineering Podcast

MARCH 3, 2024

Summary Artificial intelligence technologies promise to revolutionize business and produce new sources of value. Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization. Data lakes are notoriously complex.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

DECEMBER 17, 2023

Summary The "modern data stack" promised a scalable, composable data platform that gave everyone the flexibility to use the best tools for every job. The team at 5X understand the pain involved and the barriers to productivity and set out to solve it by pre-integrating the best tools from each layer of the stack.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Monte Carlo

JULY 7, 2022

At the time, he was one of a very few people using the term “data contract.” Data contracts have since became one of the most discussed topics in data engineering. For posterity, we have preserved Barr’s forward that examines what was then a very nascent trend, but we have also added an updated data contract FAQ as an addendum.

Data Warehouse 52

Data Warehouse Software Engineer Software Engineering Data 52

ProjectPro

JULY 10, 2021

Time series analysis and forecasting is a dark horse in the domain of Data Science. Time series is among the most applied Data Science techniques in various industrial and business operations, such as financial analysis , production planning, supply chain management, and many more. What is Time Series Analysis ?

Project 40

Project Banking Datasets Python 40

AltexSoft

JANUARY 18, 2023

For example, it can provide the same definition in the Snoop Dogg or Shakespeare styles. Models then use the patterns they learn from this training data to predict the next word in a sentence or generate new text that is grammatically correct and semantically coherent. In 2020, a remarkable AI took Silicon Valley by storm.

Datasets 64

Datasets Architecture Deep Learning SQL 64

Data Engineering Podcast

OCTOBER 23, 2022

Summary Agile methodologies have been adopted by a majority of teams for building software applications. Applying those same practices to data can prove challenging due to the number of systems that need to be included to implement a complete feature. Atlan is the metadata hub for your data ecosystem.

Project 130

Project Metadata MongoDB MySQL 130

Data Engineering Podcast

DECEMBER 28, 2022

Summary With all of the messaging about treating data as a product it is becoming difficult to know what that even means. Vishal Singh is the head of products at Starburst which means that he has to spend all of his time thinking and talking about the details of product thinking and its application to data.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

JUNE 4, 2023

Summary A significant portion of the time spent by data engineering teams is on managing the workflows and operations of their pipelines. DataOps has arisen as a parallel set of practices to that of DevOps teams as a means of reducing wasted effort. Can you describe what Agile Data Engine is and the story behind it?

Data Engineering 130

Data Engineering Data Engineer Engineering Data Lake 130

Data Engineering Podcast

DECEMBER 29, 2022

Summary Making effective use of data requires proper context around the information that is being used. Rehgan Avon co-founded AlignAI to help address this challenge through a more purposeful platform designed to collect and distribute the knowledge of how and why data is used in a business. Missing data?

Management 130

Management Metadata Business Intelligence Data Lake 130

Data Engineering Podcast

DECEMBER 25, 2022

Summary Encryption and security are critical elements in data analytics and machine learning applications. We have well developed protocols and practices around data that is at rest and in motion, but security around data in use is still severely lacking.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

ProjectPro

FEBRUARY 11, 2021

That’s very savvy because machine learning engineers are the Swiss Army Knife of the data world. Having that designation means you can build end-to-end machine learning solutions , which is a highly marketable skill set considering the fact that it has been the fastest-growing job title in the world since 2019.

Machine Learning 40

Machine Learning Engineering Programming Language Portfolio 40

Data Engineering Podcast

DECEMBER 25, 2022

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. Atlan is the metadata hub for your data ecosystem. Missing data? Stale dashboards?

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

DECEMBER 18, 2022

Summary The data ecosystem has seen a constant flurry of activity for the past several years, and it shows no signs of slowing down. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100