article thumbnail

Toward a Data Mesh (part 2) : Architecture & Technologies

François Nguyen

Just an illustration – not the truth and you certainly can do it with other technologies. There are certainly many other ways to do the same with another technologies. More than the technology chosen, it is more interesting to talk about the spirit behind these choices. How do we build data products ?

article thumbnail

Eliminate Friction In Your Data Platform Through Unified Metadata Using OpenMetadata

Data Engineering Podcast

Summary A significant source of friction and wasted effort in building and integrating data management systems is the fragmentation of metadata across various tools. After experiencing the impacts of fragmented metadata and previous attempts at building a solution Suresh Srinivas and Sriharsha Chintalapani created the OpenMetadata project.

Metadata 100
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Metadata Management And Integration At LinkedIn With DataHub

Data Engineering Podcast

The key to those solutions is a robust and flexible metadata management system. LinkedIn has gone through several iterations on the most maintainable and scalable approach to metadata, leading them to their current work on DataHub. What were you using at LinkedIn for metadata management prior to the introduction of DataHub?

Metadata 100
article thumbnail

Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data

Data Engineering Podcast

Summary The binding element of all data work is the metadata graph that is generated by all of the workflows that produce the assets used by teams across the organization. What are some examples of automated actions that can be triggered from metadata changes? What are the available events that can be used to trigger actions?

Metadata 100
article thumbnail

New Approaches to Visualizing Snowflake Query Statistics with Snowflake Technology Partners

Snowflake

Along with our Snowflake technology partners (with similar examples of innovation early last year ), we will demonstrate how they use the new get_query_operator_stats function within their respective products and further empower all of our customers for ongoing success with Snowflake.

article thumbnail

Octopai: Metadata Management for Better Business Intelligence with Amnon Drori - Episode 28

Data Engineering Podcast

For this reason metadata management systems are built to track the journey of your business data to aid in analysis, presentation, and compliance. What are some of the types of information that you classify and collect as metadata? What are some of the challenges that are typically faced by metadata management systems?

article thumbnail

Collecting And Retaining Contextual Metadata For Powerful And Effective Data Discovery

Data Engineering Podcast

Contact Info LinkedIn @shinjikim on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Contact Info LinkedIn @shinjikim on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Metadata 100