Monte Carlo

MARCH 13, 2024

Let’s dig in and explore the landscape of the top so-called “data quality tools” — what they are, what they’re not, and whether they’re the right first step towards more reliable data. Governance helps companies set important standards and achieve higher levels of data security, data accessibility, and data quality.

Data Governance 52

Data Governance Government Metadata Data 52

Databand.ai

JULY 19, 2023

This capability is useful for businesses, as it provides a clear and comprehensive view of their data’s history and transformations. Data lineage tools are not a new concept. In this article: Why Are Data Lineage Tools Important? It provides context for data, making it easier to understand and manage.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

Databand.ai

AUGUST 30, 2023

A DataOps architecture is the structural foundation that supports the implementation of DataOps principles within an organization. It encompasses the systems, tools, and processes that enable businesses to manage their data more efficiently and effectively. As a result, they can be slow, inefficient, and prone to errors.

Architecture 52

Architecture Data Ingestion Data Governance Data Cleanse 52

AltexSoft

FEBRUARY 11, 2023

This specialist works closely with people on both business and IT sides of a company to understand the current needs of the stakeholders and help them unlock the full potential of data. To get a better understanding of a data architect’s role, let’s clear up what data architecture is.

Data Architect 87

Data Architect Certification Generalist Big Data 87

Cloudera

MAY 2, 2022

from China to the UK , new data governance and protection rules are coming in on an almost daily basis. But increasingly at Cloudera, our clients are looking for a hybrid cloud architecture in order to manage compliance requirements.

Cloud 100

Cloud Government Healthcare Metadata 100

The Modern Data Company

MARCH 1, 2024

For today’s Chief Data Officers (CDOs) and data teams, the struggle is real. We’re drowning in data yet thirsting for actionable insights. We need a new approach, a paradigm shift that delivers data with the agility and efficiency of a speedboat – enter Data Products.

Database-centric 40

Database-centric Data Lake Government Retail 40

Cloudera

MARCH 23, 2022

In fact, we recently announced the integration with our cloud ecosystem bringing the benefits of Iceberg to enterprises as they make their journey to the public cloud, and as they adopt more converged architectures like the Lakehouse. 1: Multi-function analytics . 3: Open Performance.

Metadata 89

Metadata Data Architecture BI Machine Learning 89

Cloudera

OCTOBER 7, 2021

Those decentralization efforts appeared under different monikers through time, e.g., data marts versus data warehousing implementations (a popular architectural debate in the era of structured data) then enterprise-wide data lakes versus smaller, typically BU-Specific, “data ponds”.

Architecture 123

Architecture Metadata Government Kafka 123

Snowflake

DECEMBER 12, 2023

To give customers flexibility for how they fit Snowflake into their data architecture, Iceberg Tables can be configured to use either Snowflake or an external service such as AWS Glue as the table’s catalog to track metadata, with an easy, one-line SQL command to convert the table’s catalog to Snowflake in a metadata-only operation.

Metadata 90

Metadata Python AWS Government 90

Monte Carlo

MAY 25, 2023

That’s why we’re excited to announce that uniting diverse data team personas to collectively ensure data quality just got easier, thanks to a new integration between Monte Carlo’s data observability platform and Atlan’s active metadata platform.

Metadata 52

Metadata Data Governance Government Business Intelligence 52

Snowflake

JANUARY 22, 2024

Fidelity has two main enterprise data architecture guiding principles for its data scientists and data engineers: For data storage, Snowflake is the platform for storing all of the company’s structured and semi-structured analytical data in its Enterprise Data Lake and Data Labs.

Engineering 63

Engineering Data Lake Unstructured Data Metadata 63

Data Engineering Podcast

SEPTEMBER 23, 2018

Can you walk through the stages of an ideal lifecycle for data within the context of an organizations uses for it? What are some of the common mistakes that are made when designing a data architecture and how do they lead to failure?

Data Lake 100

Data Lake Data Warehouse Data Architecture Architecture 100

Monte Carlo

DECEMBER 8, 2023

Enter: data provenance and data lineage. Data lineage is a visual tool that tracks the movement and transformations of data through various systems, processes, and applications. Data provenance is the record of metadata from data’s original sources, providing the historical context and authenticity of data.

Metadata 52

Metadata Data Data Warehouse Government 52

Monte Carlo

DECEMBER 8, 2023

Enter: data provenance and data lineage. Data lineage is a visual tool that tracks the movement and transformations of data through various systems, processes, and applications. Data provenance is the record of metadata from data’s original sources, providing the historical context and authenticity of data.

Metadata 52

Metadata Data Data Warehouse Government 52

Monte Carlo

JANUARY 23, 2023

But while most every company would consider themselves a “data-first” organization, not every data architecture is treated to the same level of democratization and scalability. In this post we’ll look at the dizzyingly buzzy data mesh and how it stacks up to the more traditional aggregated architectural approach of a data lake.

Data Lake 52

Data Lake Unstructured Data Business Intelligence Architecture 52

Cloudera

SEPTEMBER 15, 2022

The consumption of the data should be supported through an elastic delivery layer that aligns with demand, but also provides the flexibility to present the data in a physical format that aligns with the analytic application, ranging from the more traditional data warehouse view to a graph view in support of relationship analysis.

Data Lake 80

Data Lake Analytics Application Cloud Storage Architecture 80

AltexSoft

NOVEMBER 10, 2021

The pun being obvious, there’s more to that than just a new term: Data lakehouses combine the best features of both data lakes and data warehouses and this post will explain this all. What is a data lakehouse? Schema enforcement and data governance. Metadata layer. Metadata layer.

Architecture 52

Architecture Data Lake Data Warehouse Metadata 52

Monte Carlo

APRIL 1, 2021

Data Catalogs Can Drown in a Data Lake Although exceptionally flexible and scalable, data lakes lack the organization necessary to facilitate proper metadata management and data governance. Data discovery tools and platforms can help. Image courtesy of Adrian on Unsplash.

Data Lake 52

Data Lake Unstructured Data Data Warehouse Metadata 52

AltexSoft

AUGUST 29, 2023

Instead of relying on traditional hierarchical structures and predefined schemas, as in the case of data warehouses, a data lake utilizes a flat architecture. This structure is made efficient by data engineering practices that include object storage. Watch our video explaining how data engineering works.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

DataKitchen

JANUARY 23, 2023

Data lineage can also be used for compliance, auditing, and data governance purposes. DataOps Observability Five on data lineage: Data lineage traces data’s origin, history, and movement through various processing, storage, and analysis stages. What is missing in data lineage?

Data Governance 52

Data Governance Government Data Pipeline Data 52

AltexSoft

MAY 12, 2023

Identify the data sources, types of analytics to be performed, and the expected outcomes to guide your efforts. Build data architecture. To effectively leverage unstructured data, allocate resources toward creating a comprehensive data architecture that supports the storage, management, and analysis of various data types.

Unstructured Data 59

Unstructured Data NoSQL Hadoop Data Lake 59

Ascend.io

MARCH 14, 2023

We’re increasingly coming to realize that the rigid, monolithic architectures we’re currently using just don’t make the cut to store, organize, access, and use the ever-growing amounts of data. Thus, the data mesh follows the seams of organizational units. So, what’s the solution?

Government 52

Government Architecture Data Lake Data 52

Ascend.io

MARCH 14, 2023

We’re increasingly coming to realize that the rigid, monolithic architectures we’re currently using just don’t make the cut to store, organize, access, and use the ever-growing amounts of data. Thus, the data mesh follows the seams of organizational units. So, what’s the solution?

Government 52

Government Architecture Data Lake Data 52

Monte Carlo

JULY 30, 2020

In the age of self-service business intelligence , nearly every company considers themselves a data-first company, but not every company is treating their data architecture with the level of democratization and scalability it deserves. Your company, for one, views data as a driver of innovation.

IT 45

IT Data Lake Business Intelligence Data Architecture 45

Cloudera

AUGUST 21, 2020

The governance aspect is perhaps even more important and businesses need to be able to understand where the data comes from. Data lineage, personally identifiable information or PPI and metadata all fall under a broad data governance banner which is critically important in terms of what needs to be protected and mapped out.

Banking 60

Banking Kafka Cloud Storage Government 60

Monte Carlo

APRIL 20, 2023

I can surface ownership metadata and alert the relevant owners to make sure the appropriate changes are made so these breakages never happen. Data System Modernization And Team Reorganization The only constant in data engineering is change.

Data Warehouse 52

Data Warehouse BI Government Data 52

AltexSoft

MARCH 30, 2023

Source: The Data Team’s Guide to the Databricks Lakehouse Platform Integrating with Apache Spark and other analytics engines, Delta Lake supports both batch and stream data processing. Besides that, it’s fully compatible with various data ingestion and ETL tools. Databricks focuses on data engineering and data science.

Scala 64

Scala Data Lake BI Google Cloud 64

Monte Carlo

JUNE 14, 2023

Data observability platforms reduce your data downtime up to 80% and make your data engineers 30% more time efficient by replacing static, cumbersome data testing with machine learning models that can help detect, resolve, and prevent data issues. Now Go Build Some Data Pipelines!

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52

ProjectPro

JANUARY 31, 2022

Several organizations can quickly transform, integrate, and analyze their data with Snowflake's Data Cloud. They can also design and run data apps and securely share, gather, and commercialize real-time data. The query processing layer is separated from the disk storage layer in the Snowflake data architecture.

Architecture 52

Architecture IT Data Warehouse Amazon Web Services 52

Monte Carlo

MAY 17, 2023

Many times this is by freeing them from having to manually implement and maintain hundreds of data tests as was the case with Contentsquare and Gitlab. “We We had too many manual data checks by operations and data analysts,” said Otávio Bastos, former global data governance lead, Contentsquare. “It

Data 52

Data Data Pipeline Data Engineering Data Engineer 52

Monte Carlo

MAY 17, 2023

Many times this is by freeing them from having to manually implement and maintain hundreds of data tests as was the case with Contentsquare and Gitlab. “We We had too many manual data checks by operations and data analysts,” said Otávio Bastos, former global data governance lead, Contentsquare. “It

Data Pipeline 52

Data Pipeline Data Data Engineering Data Engineer 52

ProjectPro

MAY 1, 2018

Hortonworks announced the launch of a data governance plug-in named Studio for its DataPlane platform at the recent Hadoop Summit.DataPlane platform that was announced last year provides data architecture as a service with data governance baked in Apache Atlas.Scott Gnau, chief technology officer at Hortonworks said that the supplier is at stage 3.0

Hadoop 40

Hadoop Banking Healthcare Food 40

phData: Data Engineering

JULY 18, 2023

Better Transparency: There’s more clarity about where data is coming from, where it’s going, why it’s being transformed, and how it’s being used. Improved Data Governance: This level of transparency can also enhance data governance and control mechanisms in the new data system.

Data Warehouse 52

Data Warehouse Pipeline-centric Government Data 52