Data Management and Metadata - Data Engineering Digest

Metadata Management and Data Governance with Cloudera SDX

Cloudera

JANUARY 26, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata

Metadata Data Governance Government Management

Metadata Management & Data Governance with Cloudera SDX

Cloudera

MARCH 4, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata

Metadata Data Governance Government Management

Eliminate Friction In Your Data Platform Through Unified Metadata Using OpenMetadata

Data Engineering Podcast

NOVEMBER 10, 2021

Summary A significant source of friction and wasted effort in building and integrating data management systems is the fragmentation of metadata across various tools. What are the common challenges faced by engineers and data practitioners in organizing the metadata for their systems?

Metadata

Metadata Data Warehouse Data Lake BI

Webinars

How to Optimize the Developer Experience for Monumental Impact

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Understanding User Needs and Satisfying Them

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Leading the Development of Profitable and Sustainable Products

MORE WEBINARS

Metadata Management And Integration At LinkedIn With DataHub

Data Engineering Podcast

AUGUST 24, 2020

Summary In order to scale the use of data across an organization there are a number of challenges related to discovery, governance, and integration that need to be solved. The key to those solutions is a robust and flexible metadata management system. If you hand a book to a new data engineer, what wisdom would you add to it?

Metadata

Metadata Management Kafka Data Engineering

Making The Total Cost Of Ownership For External Data Manageable With Crux

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data.

Data Management

Data Management Management Metadata MongoDB

Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data

Data Engineering Podcast

OCTOBER 15, 2021

Summary The binding element of all data work is the metadata graph that is generated by all of the workflows that produce the assets used by teams across the organization. The DataHub project was created as a way to bring order to the scale of LinkedIn’s data needs. How is the governance of DataHub being managed?

Metadata

Metadata BI Data Warehouse Government

Octopai: Metadata Management for Better Business Intelligence with Amnon Drori - Episode 28

Data Engineering Podcast

APRIL 22, 2018

Summary The information about how data is acquired and processed is often as important as the data itself. For this reason metadata management systems are built to track the journey of your business data to aid in analysis, presentation, and compliance. What is involved in deploying your metadata collection agents?

Business Intelligence

Business Intelligence Metadata Management Data Governance

Collecting And Retaining Contextual Metadata For Powerful And Effective Data Discovery

Data Engineering Podcast

AUGUST 13, 2022

In this episode Shinji Kim discusses the challenges of data discovery and how to collect and preserve additional context about each piece of information so that you can find what you need when you don’t even know what you’re looking for yet. What do you have planned for the future of SelectStar?

Metadata

Metadata MongoDB Scala MySQL

Databook: Turning Big Data into Knowledge with Metadata at Uber

Uber Engineering

AUGUST 3, 2018

Data powers Uber’s global marketplace, enabling more reliable and seamless user experiences across our products for riders, … The post Databook: Turning Big Data into Knowledge with Metadata at Uber appeared first on Uber Engineering Blog.

Metadata

Metadata Big Data Transportation Data

Why Your Master Data Management Needs Data Governance

Precisely

SEPTEMBER 5, 2023

With cloud computing, the capacity to extract value from data is greater than ever. As this realization grows, businesses are shifting their investments from hardware to technologies that optimize data assets. Master Data Management systems (MDM) play an important role in harmonizing data assets across large and midsize enterprises.

Data Governance

Data Governance Government Data Management Management

The Data Integration Solution Checklist: Top 10 Considerations

Precisely

MAY 13, 2024

Integrated data catalog for metadata support As you build out your IT ecosystem, it’s important to leverage tools that have the capabilities to support forward-looking use cases. A notable capability that achieves this is the data catalog. If so, how do you combine that metadata with other data across the enterprise? #4.

Data Integration

Data Integration Metadata Amazon Web Services Data Governance

Mastering the Art of ETL on AWS for Data Management

ProjectPro

FEBRUARY 16, 2023

With so much riding on the efficiency of ETL processes for data engineering teams, it is essential to take a deep dive into the complex world of ETL on AWS to take your data management to the next level. This is particularly useful for companies that need to process data in near-real-time.

AWS

AWS Data Management ETL Tools Management

Build Data Products Without A Data Team Using AgileData

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Building

Building Metadata MongoDB MySQL

The Data Discovery Team

Jesse Anderson

NOVEMBER 14, 2023

First of all, in data science, data discovery means finding patterns in data using database query languages to test hypotheses. This kind of data discovery can be subdivided into several steps, as e.g. suggested by Piethein Strengholt in Data Management at Scale.

Metadata

Metadata Data Science Big Data Data

Build Better Data Products By Creating Data, Not Consuming It

Data Engineering Podcast

NOVEMBER 6, 2022

He also describes the considerations involved in bringing behavioral data into your systems, and the ways that he and the rest of the Snowplow team are working to make that an easy addition to your platforms. Atlan is the metadata hub for your data ecosystem. What are some of the unique characteristics of that information?

Building

Building IT Metadata MongoDB

Expanding The Reach of Business Intelligence Through Ubiquitous Embedded Analytics With Sisense

Data Engineering Podcast

OCTOBER 30, 2022

In this episode Amir Orad discusses the Sisense platform and how it facilitates the embedding of analytics and data insights in every aspect of organizational and end-user experiences. Atlan is the metadata hub for your data ecosystem. What is your view on the role of business intelligence in a data driven organization?

Business Intelligence

Business Intelligence Metadata MongoDB MySQL

The View Below The Waterline Of Apache Iceberg And How It Fits In Your Data Lakehouse

Data Engineering Podcast

FEBRUARY 19, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Hey there podcast listener, are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze.

IT

IT Data Lake Metadata Data Warehouse

A Look Back at the Gartner Data and Analytics Summit

Cloudera

APRIL 18, 2024

In those discussions, it was clear that everyone understood the need to treat data estates more cohesively as a whole—that means bringing more attention to security, data governance, and metadata management, the latter of which has become increasingly popular.

Metadata

Metadata Government Data Governance Data

Combining The Simplicity Of Spreadsheets With The Power Of Modern Data Infrastructure At Canvas

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata

Metadata Unstructured Data MongoDB MySQL

FedRAMP In Process Designation, A Milestone in Cybersecurity Commitment

Cloudera

APRIL 2, 2024

I am pleased to announce that Cloudera has achieved FedRAMP “In Process”, a significant milestone that underscores our commitment to providing the public sector with secure and reliable data management solutions across on-prem, hybrid and multi-cloud environments.

Designing

Designing Process Government Metadata

Solving The Persistent Challenges of Data Modeling

The Modern Data Company

APRIL 15, 2024

Understands how different pieces of data (books) relate to each other, helping users find related information. Keeps the library orderly, adjusting as new data arrives. Tracks key performance indicators to understand the use and value of the data. Manages information about data access and monitors data quality.

Government

Government Metadata Data Data Lake

What "Data Lineage Done Right" Looks Like And How They're Doing It At Manta

Data Engineering Podcast

JULY 31, 2022

In this episode Ernie Ostic shares the approach that he and his team at Manta are taking to build a complete view of data lineage across the various data systems in your organization and the useful applications of that information in the work of every data stakeholder. Atlan is the metadata hub for your data ecosystem.

IT

IT Metadata MongoDB MySQL

Understanding The Role Of The Chief Data Officer

Data Engineering Podcast

AUGUST 21, 2022

In this episode Tracy Daniels, CDO of Truist, shares her journey into the position, her responsibilities, and her relationship to the data professionals in her organization. Atlan is the metadata hub for your data ecosystem. RudderStack helps you build a customer data platform on your warehouse or data lake.

Metadata

Metadata MongoDB MySQL Data Lake

Business Intelligence In The Palm Of Your Hand With Zing Data

Data Engineering Podcast

DECEMBER 4, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Business Intelligence

Business Intelligence Metadata BI MongoDB

Build Confidence In Your Data Platform With Schema Compatibility Reports That Span Systems And Domains Using Schemata

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Systems

Systems Metadata Building MongoDB

An Exploration Of What Data Automation Can Provide To Data Engineers And Ascend's Journey To Make It A Reality

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Engineering

Data Engineering Data Engineer MongoDB Metadata

Speeding Up The Time To Insight For Supply Chains And Logistics With The Pathway Database That Thinks

Data Engineering Podcast

OCTOBER 16, 2022

In this episode Adrian Kosowski explains how the Pathway product got started, how its design simplifies the creation of data products that support supply chain operations, and how developers can help to build an ecosystem of applications that allow businesses to accelerate their time to insight. Closing Announcements Thank you for listening!

Database

Database Metadata MongoDB MySQL

Gain Visibility And Insight Into Your Supply Chains Through Operational Analytics Powered By Roambee

Data Engineering Podcast

OCTOBER 2, 2022

In this episode Roambee CEO, Sanjay Sharma, shares the types of questions that companies are asking about their logistics, the technical work that they do to provide ways to answer those questions, and how they approach the challenge of data quality in its many forms. Atlan is the metadata hub for your data ecosystem.

Metadata

Metadata Electronics MongoDB MySQL

Build A Common Understanding Of Your Data Reliability Rules With Soda Core and Soda Checks Language

Data Engineering Podcast

SEPTEMBER 25, 2022

In this episode Tom Baeyens explains their reasons for creating a new syntax for expressing and validating checks for data assets and processes, as well as how to incorporate it into your own projects. Atlan is the metadata hub for your data ecosystem. What are the ways that reliability is measured for data assets?

Building

Building Metadata MongoDB MySQL

Useful Lessons And Repeatable Patterns Learned From Data Mesh Implementations At AgileLab

Data Engineering Podcast

AUGUST 6, 2022

In this episode Paolo Platter shares the lessons they have learned in that process, the Data Mesh Boost platform that they have built to reduce some of the boilerplate required to make it successful, and some of the considerations to make when deciding if a data mesh is the right choice for you.

Metadata

Metadata MongoDB MySQL Business Intelligence

Making The Open Data Lakehouse Affordable Without The Overhead At Iomete

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata

Metadata AWS MongoDB MySQL

Charting the Path of Riskified's Data Platform Journey

Data Engineering Podcast

JULY 10, 2022

They also discuss how they have established a guild system for training and supporting data professionals in the organization. Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities.

Metadata

Metadata MongoDB MySQL Data Pipeline

A Reflection On Data Observability As It Reaches Broader Adoption

Data Engineering Podcast

SEPTEMBER 4, 2022

In this episode founders Barr Moses and Lior Gavish rejoin the show to reflect on the evolution and adoption of data observability technologies and the capabilities that are being introduced as the broader ecosystem adopts the practices. Atlan is the metadata hub for your data ecosystem.

IT

IT Metadata MongoDB MySQL

Hire And Scale Your Data Team With Intention

Data Engineering Podcast

JUNE 12, 2022

In this episode she shares her thoughts and insights on how to be intentional about establishing your own data team. Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities.

Metadata

Metadata Unstructured Data Business Intelligence MongoDB

Building A Data Mesh Platform At PayPal

Data Engineering Podcast

FEBRUARY 26, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze. We feel your pain.

Building

Building Metadata Machine Learning Data Integration

The View From The Lakehouse Of Architectural Patterns For Your Data Platform

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. Atlan is the metadata hub for your data ecosystem. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Architecture

Architecture Metadata MongoDB MySQL

Building A Data Governance Bridge Between Cloud And Datacenters For The Enterprise At Privacera

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. What do you have planned for the future of Privacera?

Data Governance

Data Governance Government Cloud Building

Building A System Of Record For Your Organization's Data Ecosystem At Metaphor

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data.

Systems

Systems Building Metadata Data Warehouse

Exploring The Nuances Of Building An Intential Data Culture

Data Engineering Podcast

MARCH 5, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Hey there podcast listener, are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze.

Building

Building Database Design Metadata Machine Learning

Simplify Data Security For Sensitive Information With The Skyflow Data Privacy Vault

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. Atlan is the metadata hub for your data ecosystem.

Data Security

Data Security Metadata MongoDB Scala

5 Layers of Data Lakehouse Architecture Explained

Monte Carlo

JANUARY 5, 2024

This architecture format consists of several key layers that are essential to helping an organization run fast analytics on structured and unstructured data. Table of Contents What is data lakehouse architecture? The 5 key layers of data lakehouse architecture 1. Metadata layer 4. Ingestion layer 2. Storage layer 3.

Architecture

Architecture Data Lake Metadata Unstructured Data

Data Lakehouse Architecture Explained: 5 Layers

Monte Carlo

JANUARY 5, 2024

This architecture format consists of several key layers that are essential to helping an organization run fast analytics on structured and unstructured data. Table of Contents What is data lakehouse architecture? The 5 key layers of data lakehouse architecture 1. Metadata layer 4. Ingestion layer 2. Storage layer 3.

Architecture

Architecture Data Lake Metadata Unstructured Data

What Is MLOps?

Edureka

MAY 6, 2024

MLOPS basically contains eight components which are Data management, version control, automation, Experiment tracking, ci/cd, monitoring or retraining, provisioning, and Governance. Now, let’s check them one by one: Data Management: Data management focuses on keeping data organized, clean, and accessible.

Machine Learning

Machine Learning Metadata Programming Language Healthcare

Three Takeaways from Gartner’s 2019 Magic Quadrant for Data Management Solutions for Analytics

Cloudera

FEBRUARY 11, 2019

The January 2019 “Magic Quadrant for Data Management Solutions for Analytics” provides valuable insights into the status, direction, and players in the DMSA market. All this while the platform serves as the core foundation providing metadata and governance capabilities across these workloads.

Data Management

Data Management Management Metadata Government

Metadata Management and Data Governance with Cloudera SDX

Metadata Management & Data Governance with Cloudera SDX

Webinars

Trending Sources

Eliminate Friction In Your Data Platform Through Unified Metadata Using OpenMetadata

Webinars

Metadata Management And Integration At LinkedIn With DataHub

Making The Total Cost Of Ownership For External Data Manageable With Crux

Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data

Octopai: Metadata Management for Better Business Intelligence with Amnon Drori - Episode 28

Collecting And Retaining Contextual Metadata For Powerful And Effective Data Discovery

Databook: Turning Big Data into Knowledge with Metadata at Uber

Why Your Master Data Management Needs Data Governance

The Data Integration Solution Checklist: Top 10 Considerations

Mastering the Art of ETL on AWS for Data Management

Build Data Products Without A Data Team Using AgileData

The Data Discovery Team

Build Better Data Products By Creating Data, Not Consuming It

Expanding The Reach of Business Intelligence Through Ubiquitous Embedded Analytics With Sisense

The View Below The Waterline Of Apache Iceberg And How It Fits In Your Data Lakehouse

A Look Back at the Gartner Data and Analytics Summit

Combining The Simplicity Of Spreadsheets With The Power Of Modern Data Infrastructure At Canvas

FedRAMP In Process Designation, A Milestone in Cybersecurity Commitment

Solving The Persistent Challenges of Data Modeling

What "Data Lineage Done Right" Looks Like And How They're Doing It At Manta

Understanding The Role Of The Chief Data Officer

Business Intelligence In The Palm Of Your Hand With Zing Data

Build Confidence In Your Data Platform With Schema Compatibility Reports That Span Systems And Domains Using Schemata

An Exploration Of What Data Automation Can Provide To Data Engineers And Ascend's Journey To Make It A Reality

Speeding Up The Time To Insight For Supply Chains And Logistics With The Pathway Database That Thinks

Gain Visibility And Insight Into Your Supply Chains Through Operational Analytics Powered By Roambee

Build A Common Understanding Of Your Data Reliability Rules With Soda Core and Soda Checks Language

Useful Lessons And Repeatable Patterns Learned From Data Mesh Implementations At AgileLab

Making The Open Data Lakehouse Affordable Without The Overhead At Iomete

Charting the Path of Riskified's Data Platform Journey

A Reflection On Data Observability As It Reaches Broader Adoption

Hire And Scale Your Data Team With Intention

Building A Data Mesh Platform At PayPal

The View From The Lakehouse Of Architectural Patterns For Your Data Platform

Building A Data Governance Bridge Between Cloud And Datacenters For The Enterprise At Privacera

Building A System Of Record For Your Organization's Data Ecosystem At Metaphor

Exploring The Nuances Of Building An Intential Data Culture

Simplify Data Security For Sensitive Information With The Skyflow Data Privacy Vault

5 Layers of Data Lakehouse Architecture Explained

Data Lakehouse Architecture Explained: 5 Layers

What Is MLOps?

Three Takeaways from Gartner’s 2019 Magic Quadrant for Data Management Solutions for Analytics

Stay Connected