Data Engineering Podcast

JUNE 19, 2022

Summary Data analysis is a valuable exercise that is often out of reach of non-technical users as a result of the complexity of data systems. Atlan is the metadata hub for your data ecosystem. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

JUNE 12, 2022

Summary Building a well rounded and effective data team is an iterative process, and the first hire can set the stage for future success or failure. Trupti Natu has been the first data hire multiple times and gone through the process of building teams across the different stages of growth.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JULY 10, 2022

Summary Building a data platform is a journey, not a destination. In this episode Inbar Yogev and Lior Winner share the journey that they and their teams at Riskified have been on for their data platform. They also discuss how they have established a guild system for training and supporting data professionals in the organization.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

AUGUST 21, 2022

Summary The position of Chief Data Officer (CDO) is relatively new in the business world and has not been universally adopted. In this episode Tracy Daniels, CDO of Truist, shares her journey into the position, her responsibilities, and her relationship to the data professionals in her organization.

Metadata 100

Metadata MongoDB MySQL Data Lake 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Summary Regardless of how data is being used, it is critical that the information is trusted. The practice of data reliability engineering has gained momentum recently to address that question. Atlan is the metadata hub for your data ecosystem. And don’t forget to thank them for their continued support of this show!

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 9, 2022

Summary The core of any data platform is the centralized storage and processing layer. For many that is a data warehouse, but in order to support a diverse and constantly changing set of uses and technologies the data lakehouse is a paradigm that offers a useful balance of scale and cost, with performance and ease of use.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

AUGUST 28, 2022

For data engineers, this is a monumental undertaking. Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. The only thing worse than having bad data is not knowing that you have it.

Data Engineering 100

Data Engineering Data Engineer MongoDB Metadata 100

Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. Atlan is the metadata hub for your data ecosystem.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

AUGUST 6, 2022

Summary Data mesh is a frequent topic of conversation in the data community, with many debates about how and when to employ this architectural pattern. The team at AgileLab have first-hand experience helping large enterprise organizations evaluate and implement their own data mesh strategies.

Metadata 100

Metadata MongoDB MySQL Business Intelligence 100

Data Engineering Podcast

JUNE 5, 2022

Summary The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

SEPTEMBER 4, 2022

Summary Data observability is a product category that has seen massive growth and adoption in recent years. Monte Carlo is in the vanguard of companies who have been enabling data teams to observe and understand their complex data systems. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

MAY 22, 2022

Summary Machine learning has become a meaningful target for data applications, bringing with it an increase in the complexity of orchestrating the entire data flow. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform.

Machine Learning 130

Machine Learning Data Engineering Data Engineer Cloud 130

Data Engineering Podcast

OCTOBER 16, 2022

Pathway is a streaming database engine that embeds artificial intelligence into the storage, with functionality designed to support the spatiotemporal data that is crucial for shipping and logistics. Atlan is the metadata hub for your data ecosystem.

Database 100

Database Metadata MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

This requires a significant amount of data to track shipments and operational characteristics of materials and goods. Roambee is a platform that collects, integrates, and analyzes all of that information to provide companies with the critical insights that businesses need to stay running, especially in a time of such constant change.

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data. Missing data?

Systems 100

Systems Building Metadata Data Warehouse 100

Knowledge Hut

MAY 9, 2024

Data Science, with its interdisciplinary approach, combines statistics, computer science, and domain knowledge and has opened up a world of exciting and lucrative career opportunities for professionals with the right skills and expertise. The market is flooding with the highest paying data science jobs. What is Data Science?

Data Science 52

Data Science Data Mining Data Architect Programming Language 52

Data Engineering Podcast

NOVEMBER 20, 2021

Summary The technology for scaling storage and processing of data has gone through massive evolution over the past decade, leaving us with the ability to work with massive datasets at the cost of massive complexity. Struggling with broken pipelines? Missing data? Start trusting your data with Monte Carlo today!

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

AltexSoft

FEBRUARY 11, 2023

Data is now one of the most valuable assets for any kind of business. The 11th annual survey of Chief Data Officers (CDOs) and Chief Data and Analytics Officers reveals 82 percent of organizations are planning to increase their investments in data modernization in 2023. What is a data architect?

Data Architect 87

Data Architect Certification Generalist Big Data 87

Silectis

JANUARY 3, 2021

If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. Big Data Large volumes of structured or unstructured data.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data. What is a data lake?

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. DataOps helps the data mesh deliver greater business agility by enabling decentralized domains to work in concert. . But first, let’s define the data mesh design pattern.

Pharmaceutical 98

Pharmaceutical Data Lake Data Architecture Scala 98

Ascend.io

AUGUST 31, 2023

In the dynamic world of data, many professionals are still fixated on traditional patterns of data warehousing and ETL, even while their organizations are migrating to the cloud and adopting cloud-native data services. This unchanging schema forms the foundation for all queries and business intelligence.

Data Lake 52

Data Lake ETL Tools Data Warehouse Data Pipeline 52

Monte Carlo

JANUARY 16, 2024

They run on data powered by Fox. Factor in the advertising strategies, media production, partner programming, audience analytics…and you’re looking at an ocean of data that would fill even the deepest trench (we’d like a television show about that too, please!). So how does Fox’s data strategy support these complex data workflows?

Government 52

Government High Quality Data Data Governance Entertainment 52

Monte Carlo

APRIL 27, 2023

In the battle against data downtime , data teams must be equipped with full visibility into their data pipelines from ingestion and orchestration to transformation and visualization. Each interconnected layer can introduce and be impacted by poor data quality. There will always be data quality issues.

BI 52

BI Business Intelligence Metadata Data Warehouse 52

Monte Carlo

JUNE 28, 2022

To help organizations realize the full potential of their data lake and lakehouse investments, Monte Carlo, the data observability leader, is proud to announce integrations with Delta Lake and Databricks’ Unity Catalog for full data observability coverage. billion in 2020 to 17.60 billion in 2020 to 17.60

Data Lake 52

Data Lake Metadata AWS Data Warehouse 52

Monte Carlo

MARCH 3, 2021

When it comes to managing your company’s data, security is high on your list of priorities. Our SOC 2 designation means that Monte Carlo has designed a set of internal controls, systems, policies, and procedures that meet industry best practices for protecting our customer data. What does this mean for you? What’s next?

Metadata 52

Metadata Data Lake Business Intelligence Certification 52

Monte Carlo

AUGUST 17, 2021

It would be an understatement to say your company is bullish on data. And your entire data engineering team is all in on this “data as code” movement. The flip side of this data-driven coin: your stakeholders (CEO and CTO included) ping you nearly every other hour to ask you: “is my data up-to-date?”

Insurance 40

Insurance Retail Data Pipeline Portfolio 40

Data Engineering Podcast

NOVEMBER 20, 2022

Summary The majority of blog posts and presentations about data engineering and analytics assume that the consumers of those efforts are internal business users accessing an environment controlled by the business. Atlan is the metadata hub for your data ecosystem.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. Atlan is the metadata hub for your data ecosystem. Struggling with broken pipelines? Missing data?

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

DECEMBER 29, 2022

Summary Making effective use of data requires proper context around the information that is being used. Rehgan Avon co-founded AlignAI to help address this challenge through a more purposeful platform designed to collect and distribute the knowledge of how and why data is used in a business. Struggling with broken pipelines?

Management 130

Management Metadata Business Intelligence Data Lake 130

Data Engineering Podcast

DECEMBER 18, 2022

Summary One of the reasons that data work is so challenging is because no single person or team owns the entire process. This introduces friction in the process of collecting, processing, and using data. In order to reduce the potential for broken pipelines some teams have started to adopt the idea of data contracts.

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Monte Carlo

JUNE 14, 2023

In this post, we will help you quickly level up your overall knowledge of data pipeline architecture by reviewing: Table of Contents What is data pipeline architecture? Why is data pipeline architecture important? What is data pipeline architecture? Why is data pipeline architecture important?

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52

Data Engineering Podcast

OCTOBER 23, 2022

Applying those same practices to data can prove challenging due to the number of systems that need to be included to implement a complete feature. Atlan is the metadata hub for your data ecosystem. Data engineers don’t enjoy writing, maintaining, and modifying ETL pipelines all day, every day.

Project 130

Project Metadata MongoDB MySQL 130

Data Engineering Podcast

NOVEMBER 27, 2022

Summary The data ecosystem has been growing rapidly, with new communities joining and bringing their preferred programming languages to the mix. This has led to inefficiencies in how data is stored, accessed, and shared across process and system boundaries. Atlan is the metadata hub for your data ecosystem. Missing data?

Data Process 100

Data Process Process Metadata Business Intelligence 100

Cloudera

FEBRUARY 8, 2021

You can read part 1, here: Digital Transformation is a Data Journey From Edge to Insight. The first blog introduced a mock connected vehicle manufacturing company, The Electric Car Company (ECC), to illustrate the manufacturing data path through the data lifecycle. 1 The enterprise data lifecycle.

Data Pipeline 117

Data Pipeline Building Manufacturing Data Warehouse 117

Data Engineering Podcast

JUNE 26, 2022

Summary The proliferation of sensors and GPS devices has dramatically increased the number of applications for spatial data, and the need for scalable geospatial analytics. Atlan is the metadata hub for your data ecosystem. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130