Precisely

APRIL 11, 2023

It’s a common scenario across industries: a new strategic initiative is launching, and you need trustworthy, relevant data to analyze and support the best outcomes for the business. What existing data does the organization have to support my initiative? What existing data does the organization have to support my initiative?

Accessible 52

Accessible Accessibility Metadata Data Integration 52

Cloudera

SEPTEMBER 15, 2022

Apache Ozone is a distributed, scalable, and high-performance object store , available with Cloudera Data Platform (CDP), that can scale to billions of objects of varying sizes. Structured data (such as name, date, ID, and so on) will be stored in regular SQL databases like Hive or Impala databases.

Systems 87

Systems Hadoop Metadata Telecommunication 87

Knowledge Hut

APRIL 25, 2024

Intrusion detection systems (IDS) are designed to identify suspicious and malicious activity through network traffic. So, let's get to know the meaning of an intrusion detection system and how it works. What Is a n Intrusion Detection System? Why Are Intrusion Detection Systems Important? and how it works.

Systems 52

Systems Recruitment Machine Learning Certification 52

Precisely

MAY 13, 2024

Key Takeaways: Data integration is vital for real-time data delivery across diverse cloud models and applications, and for leveraging technologies like generative AI. As enterprise technology landscapes grow more complex, the role of data integration is more critical than ever before.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Confluent

MAY 7, 2019

Although it is the simplest way to subscribe to and access events from Kafka, behind the scenes, Kafka consumers handle tricky distributed systems challenges like data consistency, failover and load balancing. Data processing requirements. We therefore need a way of splitting up the data ingestion work.

Kafka 111

Kafka Accessible Accessibility Metadata 111

LinkedIn Engineering

DECEMBER 6, 2022

With a reasonably sizable footprint of servers in data centers, LinkedIn is responsible for ensuring that these hosts are always on an operating system (OS) version deemed the ���latest and greatest��� for all intents and purposes. The existing solution had challenges with its maintainability, scalability, and high availability.

Systems 55

Systems Metadata Datasets Designing 55

Netflix Tech

SEPTEMBER 29, 2022

Timestone: Netflix’s High-Throughput, Low-Latency Priority Queueing System with Built-in Support for Non-Parallelizable Workloads by Kostas Christidis Introduction Timestone is a high-throughput, low-latency priority queueing system we built in-house to support the needs of Cosmos , our media encoding platform. Over the past 2.5

Systems 85

Systems Metadata Media Kafka 85

The Modern Data Company

JANUARY 22, 2024

The Modern Data Company has been given an honorable mention in Gartner’s 2023 Magic Quadrant for Data Integration. In response, The Modern Data Company emerged, driven by a clear mission: to revolutionize data management and address challenges posed by a diverse and rapidly evolving data environment.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98

Precisely

MAY 23, 2024

Key takeaways: Quickly adapt to market changes by easily adding new data sources and targets, ensuring your IT landscape evolves at the pace of your business. Gain a competitive edge with real-time data integration, crucial for time-sensitive decisions and actions in fraud detection and customer interactions.

Data Integration 62

Data Integration Metadata Telecommunication Data Governance 62

Data Engineering Podcast

DECEMBER 4, 2022

Summary Business intelligence is the foremost application of data in organizations of all sizes. The typical conception of how it is accessed is through a web or desktop application running on a powerful laptop. Zing Data is building a mobile native platform for business intelligence.

Business Intelligence 100

Business Intelligence Metadata BI MongoDB 100

Monte Carlo

MAY 20, 2024

Much like a confused tourist trying to differentiate between an alligator and a crocodile from a safe distance, you might squint at a data catalog vs. data dictionary and wonder, “Aren’t they just two boring lists of data stuff?” It is primarily concerned with the accuracy and proper usage of data at a micro level.

Datasets 52

Datasets Government Data Business Analyst 52

Data Engineering Weekly

MARCH 24, 2024

link] Kai Waehner: The Data Streaming Landscape 2024 This is a comprehensive overview of the state of the data streaming landscape in 2024. The author goes beyond comparing the tools to various offerings from streaming vendors in stream processing and Kafka protocol-supported systems.

Data Engineering 62

Data Engineering Data Engineer Engineering Metadata 62

Knowledge Hut

MARCH 7, 2024

The year 2024 saw some enthralling changes in volume and variety of data across businesses worldwide. The surge in data generation is only going to continue. Foresighted enterprises are the ones who will be able to leverage this data for maximum profitability through data processing and handling techniques.

Big Data 98

Big Data Bytes Data Governance Raw Data 98

The Modern Data Company

JANUARY 22, 2024

The Modern Data Company has been given an honorable mention in Gartner’s 2023 Magic Quadrant for Data Integration. In response, The Modern Data Company emerged, driven by a clear mission: to revolutionize data management and address challenges posed by a diverse and rapidly evolving data environment.

Data Integration 87

Data Integration Metadata Government Unstructured Data 87

Christophe Blefari

FEBRUARY 3, 2023

Delivering the data news ( credits ) Hey you, it's already February. The article shows our Netflix art creators are using past data to create new artworks. ebay, Variable Hub a data access layer for risk decisioning — Looks like a feature store but for risk topics. Every week same analysis for me.

BI 130

BI Google Cloud SQL Machine Learning 130

Monte Carlo

MAY 14, 2024

As organizations seek greater value from their data, data architectures are evolving to meet the demand — and table formats are no exception. But while the modern data stack , and how it’s structured, may be evolving, the need for reliable data is not — and that also has some real implications for your data platform.

Data Lake 52

Data Lake Metadata Hadoop Data Governance 52

Engineering at Meta

MARCH 18, 2024

Systems and application logs play a key role in operations, observability, and debugging workflows at Meta. We designed the system to support service-level guarantees on log freshness, completeness, durability, query latency, and query result completeness. Each log line can have zero or more metadata key-value pairs attached to it.

Engineering 91

Engineering Metadata Architecture Designing 91

Knowledge Hut

MAY 26, 2024

Kubernetes is a system for managing and orchestrating containerized applications across a cluster of nodes. The system is designed to simplify life cycle management, allowing developers to focus on their application code rather than infrastructure maintenance. It was designed by Google to manage and schedule containers at scale.

Metadata 52

Metadata Certification Accessible Accessibility 52

Cloudera

DECEMBER 16, 2022

We are pleased to announce that Cloudera has been named a Leader in the 2022 Gartner ® Magic Quadrant for Cloud Database Management Systems. We’re proud to be recognized for the data management and data analytics innovations we have delivered in the new Cloudera Data Platform (CDP). This is unique.

Database 90

Database Cloud Systems Management 90

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with. Atlan is the metadata hub for your data ecosystem.

IT 100

IT Metadata MongoDB MySQL 100

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.

Data Lake 91

Data Lake Data Warehouse Cloud Unstructured Data 91

Cloudera

SEPTEMBER 28, 2021

Cloudera Data platform ( CDP ) provides a Shared Data Experience ( SDX ) for centralized data access control and audit in the Enterprise Data Cloud. The Ranger Authorization Service (RAZ) is a new service added to help provide fine-grained access control (FGAC) for cloud storage.

Accessible 62

Accessible Accessibility Cloud Cloud Storage 62

Engineering at Meta

OCTOBER 31, 2023

Meta’s Systematic Code and Asset Removal Framework (SCARF) has a subsystem for identifying and removing unused data types. SCARF scans production data systems to identify tables or assets that are unused and safely removes them. Each represents a class of data — not individual records.

Data 93

Data Metadata Coding Relational Database 93

LinkedIn Engineering

JULY 19, 2023

Co-Authors: Sumedh Sakdeo , Lei Sun , Sushant Raikar , Stanislav Pak , and Abhishek Nath Introduction At LinkedIn, we build and operate an open source data lakehouse deployment to power Analytics and Machine Learning workloads. Unfortunately, there is currently no system in open source that unifies them through a single control plane.

Big Data 78

Big Data Data Management Management Metadata 78

Precisely

OCTOBER 16, 2023

Big data analytics, IoT, AI, and machine learning are revolutionizing the way businesses create value and competitive advantage. For organizations that continue to rely on mainframe systems to process their most mission-critical workloads, the digital revolution demands new approaches to integration.

Cloud 70

Cloud Cloud Computing Metadata Data Governance 70

Data Engineering Podcast

NOVEMBER 27, 2022

Summary The data ecosystem has been growing rapidly, with new communities joining and bringing their preferred programming languages to the mix. This has led to inefficiencies in how data is stored, accessed, and shared across process and system boundaries. Atlan is the metadata hub for your data ecosystem.

Data Process 100

Data Process Process Metadata Business Intelligence 100

Data Engineering Podcast

MARCH 18, 2023

Summary As with all aspects of technology, security is a critical element of data applications, and the different controls can be at cross purposes with productivity. In this episode Yoav Cohen from Satori shares his experiences as a practitioner in the space of data security and how to align with the needs of engineers and business users.

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Data Engineering Podcast

AUGUST 6, 2022

Summary Data mesh is a frequent topic of conversation in the data community, with many debates about how and when to employ this architectural pattern. The team at AgileLab have first-hand experience helping large enterprise organizations evaluate and implement their own data mesh strategies.

Metadata 100

Metadata MongoDB MySQL Business Intelligence 100

Data Engineering Podcast

AUGUST 28, 2022

For data engineers, this is a monumental undertaking. Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. The only thing worse than having bad data is not knowing that you have it.

Data Engineering 100

Data Engineering Data Engineer MongoDB Metadata 100

Striim

FEBRUARY 5, 2024

Organizations are grappling with the increasing complexity and diversity of their data sources. Traditional approaches often fall short in addressing the challenges posed by disparate data silos, and there arises a need for a more cohesive and integrated solution. What is Data Fabric?

Pharmaceutical 52

Pharmaceutical Data Cleanse Metadata Medical 52

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor.

Data Governance 130

Data Governance Government Cloud Building 130

Monte Carlo

FEBRUARY 26, 2024

A data catalog is a constantly updated inventory of the universe of data assets within an organization. It uses metadata to create a picture of the data, as well as the relationships between data assets of diverse sources, and the processing that takes place as data moves through systems.

Metadata 52

Metadata Government Data Data Governance 52

Snowflake

NOVEMBER 29, 2023

Data cloud technology can accelerate FAIRification of the world’s biomedical patient data. In other instances, the concern is primarily the risk of potential patient re-identification that comes with longitudinal data enrichment.

Metadata 84

Metadata Healthcare Medical Data Storage 84

Monte Carlo

JANUARY 5, 2024

You know what they always say: data lakehouse architecture is like an onion. …ok, Data lakehouse architecture combines the benefits of data warehouses and data lakes, bringing together the structure and performance of a data warehouse with the flexibility of a data lake. Metadata layer 4. API layer 5.

Architecture 52

Architecture Data Lake Metadata Unstructured Data 52

Data Engineering Weekly

DECEMBER 29, 2022

Data catalogs are the most expensive data integration systems you never intended to build. Data Catalog as a passive web portal to display metadata requires significant rethinking to adopt modern data workflow, not just adding “modern” in its prefix. How happy are you with your data catalogs?

Metadata 62

Metadata Data Warehouse ETL Tools Data Workflow 62

Cloudera

OCTOBER 7, 2021

In this blog, I will demonstrate the value of Cloudera DataFlow (CDF) , the edge-to-cloud streaming data platform available on the Cloudera Data Platform (CDP) , as a Data integration and Democratization fabric. Introduction to the Data Mesh Architecture and its Required Capabilities. Components of a Data Mesh.

Architecture 124

Architecture Metadata Government Kafka 124

Netflix Tech

DECEMBER 3, 2022

by Jasmine Omeke , Obi-Ike Nwoke , Olek Gorajek Intro This post is for all data practitioners, who are interested in learning about bootstrapping, standardization and automation of batch data pipelines at Netflix. You may remember Dataflow from the post we wrote last year titled Data pipeline asset management with Dataflow.

Data Pipeline 143

Data Pipeline Scala Metadata Food 143