Snowflake

SEPTEMBER 14, 2023

More than 50% of data leaders recently surveyed by BCG said the complexity of their data architecture is a significant pain point in their enterprise. Your technology stack should accommodate growth—in data volumes as well as in your business. It should foster collaboration across functions.

Data Architecture 97

Data Architecture Architecture Hadoop Data Warehouse 97

Cloudera

AUGUST 8, 2022

In June 2022, Cloudera announced the general availability of Apache Iceberg in the Cloudera Data Platform (CDP). Iceberg is a 100% open-table format, developed through the Apache Software Foundation , which helps users avoid vendor lock-in and implement an open lakehouse. .

Data Warehouse 91

Data Warehouse BI Machine Learning SQL 91

Monte Carlo

DECEMBER 20, 2023

“The data and AI space moves fast. Wondering what’s next for the future of data engineering and GenAI? Each year, we chat with one of the data industry’s pioneering leaders about their predictions for the modern data stack – and share a few of our own. Data teams will look like software teams 3.

Amazon Web Services 52

Amazon Web Services Metadata AWS Data 52

Monte Carlo

DECEMBER 20, 2023

“The data and AI space moves fast. Wondering what’s next for the future of data engineering and GenAI? Each year, we chat with one of the data industry’s pioneering leaders about their predictions for the modern data stack – and share a few of our own. Data teams will look like software teams 3.

Amazon Web Services 52

Amazon Web Services Metadata AWS Data 52

Monte Carlo

APRIL 24, 2023

Data lakes are useful, flexible data storage repositories that enable many types of data to be stored in its rawest state. Traditionally, after being stored in a data lake, raw data was then often moved to various destinations like a data warehouse for further processing, analysis, and consumption.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

Cloudera

NOVEMBER 1, 2022

A quick trip in the congressional time machine to revisit 2017’s Modernizing Government Technology Act surfaces some of the most salient points regarding agencies’ challenges: The federal government spends nearly 75% of its annual information technology funding on operating and maintaining existing legacy information technology systems.

Cloud 86

Cloud Government Cloud Computing Data Architecture 86

Monte Carlo

MARCH 21, 2024

I maintain a sandbox environment for Monte Carlo’s solution engineers to demo our data observability solution. I take a lot of pride in creating a realistic environment that mirrors the architecture our customers use to deliver data every day. I use Mockaroo to simulate source data. How easily? What does an alert look like?

Data Warehouse 52

Data Warehouse Technology Database Machine Learning 52

Cloudera

JANUARY 26, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 73

Metadata Data Governance Government Management 73

Monte Carlo

AUGUST 31, 2023

Data is a priority for your CEO, as it often is for digital-first companies, and she is fluent in the latest and greatest business intelligence tools. But what happens when the data is wrong? What about a frantic email from your CTO about “duplicate data” in a business intelligence dashboard? The root of data downtime?

Data Lake 52

Data Lake Data Pipeline Unstructured Data Data Warehouse 52

Cloudera

JULY 14, 2023

Introduction For more than a decade now, the Hive table format has been a ubiquitous presence in the big data ecosystem, managing petabytes of data with remarkable efficiency and scale. Some of the common issues include constrained schema evolution, static partitioning of data, and long planning time because of S3 directory listings.

Metadata 70

Metadata Data Warehouse Big Data Ecosystem Java 70

Cloudera

MARCH 4, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 52

Metadata Data Governance Government Management 52

Cloudera

JANUARY 5, 2023

In the past year, businesses who doubled down on digital transformation during the pandemic saw their efforts coming to fruition in the form of cost savings and more streamlined data management. 1- Treating data as a strategic business asset . 2- Operationalizing adaptive AI systems for quicker business decision-making.

Banking 52

Banking Machine Learning Insurance Data Architecture 52

Cloudera

OCTOBER 30, 2023

The legacy IT infrastructure to run the business operations — mainly data centers — has a deadline to shift to cloud-based services. The public cloud is increasingly becoming the preferred platform to host data analytics – related projects, such as business intelligence, machine learning (ML), and AI applications. Why FinOps?

Cloud 77

Cloud Finance Cloud Computing Government 77

Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Data lakes are notoriously complex. Join in with the event for the global data community, Data Council Austin.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data. What is a data lake?

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Monte Carlo

JULY 27, 2023

It’s not enough for data teams to be magicians transforming raw data into business value, they need to be responsible (and cost conscious) stewards as well. Unfortunately, data optimization is far from an exact science. Like much of the B2B SaaS universe, costs are based on platform usage. What are the benefits?

Data 52

Data Architecture Data Warehouse Data Engineering 52

ProjectPro

FEBRUARY 21, 2023

With so many data engineering certifications available , choosing the right one can be a daunting task. There are over 133K data engineer job openings in the US, but how will you stand out in such a crowded job market? The answer is- by earning professional data engineering certifications! Cloudera or Databricks?

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Monte Carlo

AUGUST 17, 2022

Over the years, I’ve helped companies of all sizes build and maintain data systems—from my days as a data engineer at Facebook to my current role as an end-to-end data solutions consultant. As a YouTuber and blogger , I’ve connected with data engineers from all over the world. The stakes also get higher.

Systems 52

Systems Building Data Data Engineering 52

Knowledge Hut

SEPTEMBER 21, 2023

Data is the new fuel, and data engineers are the architects who refine and harness its power. As an aspiring data engineer, this is the world I entered a world where I have the privilege of harnessing the power of data to leverage its full potential. Who is an Azure Data Engineer and What Do They Do?

Data Engineering 52

Data Engineering Data Engineer Engineering Cloud Computing 52

Ascend.io

AUGUST 31, 2023

In the dynamic world of data, many professionals are still fixated on traditional patterns of data warehousing and ETL, even while their organizations are migrating to the cloud and adopting cloud-native data services. This unchanging schema forms the foundation for all queries and business intelligence.

Data Lake 52

Data Lake ETL Tools Data Warehouse Data Pipeline 52

Data Engineering Podcast

APRIL 21, 2024

When Andrew Lee started work on Shortwave he was focused on making email more productive. In this episode he shares the technical challenges that he and his team have overcome in integrating AI into their product, as well as the benefits and features that it provides to their customers. Data lakes are notoriously complex.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin.

Database 162

Database Technology Data Lake High Quality Data 162

Cloudera

SEPTEMBER 23, 2022

In a recent blog, Cloudera Chief Technology Officer Ram Venkatesh described the evolution of a data lakehouse, as well as the benefits of using an open data lakehouse, especially the open Cloudera Data Platform (CDP). Modern data lakehouses are typically deployed in the cloud.

Data Governance 106

Data Governance Government Amazon Web Services Cloud Computing 106

Cloudera

MARCH 28, 2024

At a time when AI is exploding in popularity and finding its way into nearly every facet of business operations, data has arguably never been more valuable. As organizations continue to navigate this AI-driven world, we set out to understand the strategies and emerging data architectures that are defining the future.

Data Architecture 91

Data Architecture Architecture Data Lake Data 91

Cloudera

FEBRUARY 8, 2024

It brings the reliability and simplicity of SQL tables to big data while enabling engines like Hive, Impala, Spark, Trino, Flink, and Presto to work with the same tables at the same time. Apache Iceberg forms the core foundation for Cloudera’s Open Data Lakehouse with the Cloudera Data Platform (CDP).

Metadata 89

Metadata Data Warehouse BI AWS 89

Monte Carlo

SEPTEMBER 1, 2023

One of the stars taking center stage at the Databricks’ Data + AI Summit this year was their latest data quality feature, Lakehouse Monitoring. As we’ve been saying for years: to achieve stakeholder trust, data teams need a comprehensive understanding of the health and performance of data within their systems!

BI 52

BI Data Pipeline Data Data Lake 52

Monte Carlo

NOVEMBER 14, 2023

Does it feel colder in here or is it all this Apache Iceberg talk? How data observability can increase data trust in Iceberg tables What is Apache Iceberg? It is designed to be easily queryable with SQL even for large analytic tables (we’re talking petabytes of data). Is your data lake a good fit for Iceberg?

Data Lake 64

Data Lake Metadata Data Warehouse SQL 64

Towards Data Science

FEBRUARY 15, 2023

Concepts, theory, and functionalities of this modern data storage framework Photo by Nick Fewings on Unsplash Introduction I think it’s now perfectly clear to everybody the value data can have. To use a hyped example, models like ChatGPT could only be built on a huge mountain of data, produced and collected over years.

Data Lake 80

Data Lake Data Warehouse Hadoop Data Architecture 80

Cloudera

DECEMBER 15, 2022

Since we announced the general availability of Apache Iceberg in Cloudera Data Platform (CDP), Cloudera customers, such as Teranet , have built open lakehouses to future-proof their data platforms for all their analytical workloads. Cloudera partners are also benefiting from Apache Iceberg in CDP.

Cloud 78

Cloud Metadata Google Cloud Data Warehouse 78

Rockset

OCTOBER 26, 2023

JetBlue is the data leader in the airline industry using data to offer industry-leading customer experiences and disruptive low fares to popular destinations around the world. The complexity of the data and situation can be hard to quickly comprehend and take action on without the assistance of machine learning.

Machine Learning 64

Machine Learning Data Science Architecture Data Warehouse 64

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.

Data Lake 91

Data Lake Data Warehouse Cloud Unstructured Data 91

Monte Carlo

AUGUST 25, 2023

Modern companies are ingesting, storing, transforming, and leveraging more data to drive more decision-making than ever before. Data teams need to balance the need for robust, powerful data platforms with increasing scrutiny on costs. But, the options for data storage are evolving quickly. Let’s dive in.

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Monte Carlo

JULY 25, 2023

What is data curation? Data curation is the process of transforming and enriching larger amounts of raw data into smaller, more widely accessible subsets of data that provide additional value to the organization or the intended use case. Medallion architecture is a type of data curation.

Raw Data 52

Raw Data Data Warehouse Data Architecture 52

Cloudera

JULY 11, 2022

The need for data fabric. As Cloudera CMO David Moxey outlined in his blog , we live in a hybrid data world. Data is growing and continues to accelerate its growth. Driving insight and value from it all is as much of an opportunity as it is a challenge. Cloudera data fabric and analyst acclaim.

Data Architecture 106

Data Architecture Architecture Data Government 106

Monte Carlo

FEBRUARY 23, 2024

Over the past several years, data observability tooling has emerged as a must-have for every industry, from retail to security. And this renewed focus on data quality is bringing much needed visibility into the health of technical systems. Table of Contents All eyes on GenAI 3 LLM use cases 1.

Systems 64

Systems Building Retail Data Lake 64

Cloudera

SEPTEMBER 7, 2022

We are now well into 2022 and the megatrends that drove the last decade in data — The Apache Software Foundation as a primary innovation vehicle for big data, the arrival of cloud computing, and the debut of cheap distributed storage — have now converged and offer clear patterns for competitive advantage for vendors and value for customers.

Big Data Ecosystem 110

Big Data Ecosystem Hadoop Big Data Architecture 110

Scott Logic

NOVEMBER 6, 2023

In this post we’ll take a whistle-stop tour around the topic of data, going from sources and types through to use cases and the technologies involved in processing data. Introduction Over the last decade we’ve heard more and more about data.

Data Lake 52

Data Lake Kafka BI Unstructured Data 52