Monte Carlo

FEBRUARY 21, 2023

Despite its prevalence, data can be messy, siloed, ungovernable, and inaccessible—especially to the non-technical employees who rely on it. Enter data fabric: a data management architecture designed to serve the needs of the business, not just those of data engineers. Table of Contents What is a data fabric?

Data Architecture 52

Data Architecture Architecture Metadata Unstructured Data 52

Cloudera

JUNE 27, 2023

During the COVID-19 pandemic, telcos made unprecedented use of data and data-driven automation to optimize their network operations, improve customer support, and identify opportunities to expand into new markets. There are savings to be made in modernizing the data stack itself, in addition to applying data across the enterprise.

Data Architecture 63

Data Architecture Architecture Data Governance Government 63

Cloudera

SEPTEMBER 29, 2022

Data teams have the impossible task of delivering everything (data and workloads) everywhere (on premise and in all clouds) all at once (with little to no latency). Each of these trends claim to be complete models for their data architectures to solve the “everything everywhere all at once” problem. Data mesh defined.

Architecture 76

Architecture Data Architecture Metadata Data Warehouse 76

Cloudera

AUGUST 9, 2022

With all of the buzz around cloud computing, many companies have overlooked the importance of hybrid data. The truth is, the future of data architecture is all about hybrid. As a leader in hybrid data, Cloudera is positioned to help organizations take on the challenge of managing and analyzing data wherever it resides.

Data Architecture 92

Data Architecture Architecture Cloud Computing Cloud 92

KDnuggets

FEBRUARY 10, 2022

Going forward, data professionals have found a new way to address the scalability of sources through data mesh.

Data Architecture 107

Data Architecture Architecture IT Data 107

Snowflake

JULY 27, 2023

In this guest blog post, HomeToGo’s director of data, Stephan Claus, explains why the company migrated to Snowflake to meet its data needs. This article is based on Stephan’s presentation during the Snowflake Data World Tour 2022. Over the course of this journey, HomeToGo’s data needs have evolved considerably.

Data Architecture 52

Data Architecture Architecture Building Structured Data 52

DataKitchen

JULY 22, 2021

Learn about four data architectures patterns for agility - DataOps, Data Fabric, Data Mesh & Functional Data Engineering - & an example combining all four. The post DataOps: The Foundation for Your Agile Data Architecture first appeared on DataKitchen.

Data Architecture 52

Data Architecture Architecture Data Data Engineering 52

KDnuggets

FEBRUARY 16, 2022

How to Learn Math for Machine Learning; Data Mesh & Its Distributed Data Architecture; 5 Ways to Apply AI to Small Data Sets; Top 5 Free Machine Learning Courses; Junior Data Scientist: The Next Level.

Machine Learning 95

Machine Learning Data Architecture Architecture IT 95

databricks

APRIL 26, 2023

One of Lakehouse's outstanding achievements is the ability to combine workloads for modern use cases, such as traditional BI, machine learning & AI.

Data Architecture 106

Data Architecture Architecture BI Machine Learning 106

KDnuggets

APRIL 5, 2023

Learn about Apache Kafka architecture and its implementation using a real-world use case of a taxi booking app.

Kafka 102

Kafka Architecture Data Architecture Building 102

Data Engineering Podcast

SEPTEMBER 7, 2020

For analytical use cases you often want to combine data across multiple sources and storage locations. This frequently requires cumbersome and time-consuming data integration. If you hand a book to a new data engineer, what wisdom would you add to it? And don’t forget to thank them for their continued support of this show!

Architecture 100

Architecture Data Architecture SQL Engineering 100

DataKitchen

AUGUST 3, 2021

The post Building a Modern Data Architecture for the 2020s first appeared on DataKitchen.

Data Architecture 59

Data Architecture Architecture Building Data 59

Confluent

OCTOBER 30, 2020

A fundamental challenge with today’s “data explosion” is finding the best answer to the question, “So where do I put my data?” while avoiding the longer-term problem of data warehouses, […].

Data Architecture 117

Data Architecture Architecture Kafka Building 117

DataKitchen

JULY 16, 2021

The post How to build a successful cloud data architecture first appeared on DataKitchen.

Data Architecture 52

Data Architecture Architecture Cloud Building 52

KDnuggets

OCTOBER 30, 2023

A comparative overview of data warehouses, data lakes, and data marts to help you make informed decisions on data storage solutions for your data architecture.

Data Lake 121

Data Lake Data Warehouse Data Storage Data 121

Towards Data Science

JANUARY 16, 2024

My personal take on justifying the existence of Data Mesh A senior stakeholder at one my projects mentioned that they wanted to decentralise their data platform architecture and democratise data across the organisation. When I heard the words ‘decentralised data architecture’, I was left utterly confused at first!

Data Warehouse 67

Data Warehouse Data Architecture Relational Database NoSQL 67

Simon Späti

APRIL 3, 2023

Amidst the excitement and hype surrounding artificial intelligence, the significance of data engineering and its critical foundation—data modeling—can often be overlooked.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Architecture 130

Simon Späti

APRIL 3, 2023

Amidst the excitement and hype surrounding artificial intelligence, the significance of data engineering and its critical foundation—data modeling—can often be overlooked.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Architecture 130

Simon Späti

MAY 26, 2023

Welcome to the third and final installment of our series “Data Modeling: The Unsung Hero of Data Engineering.” In this third part, we’ll delve into data architecture patterns and their influence on data modeling.

Architecture 130

Architecture Data Engineering Data Engineer Engineering 130

Simon Späti

MAY 26, 2023

Welcome to the third and final installment of our series “Data Modeling: The Unsung Hero of Data Engineering.” In this third part, we’ll delve into data architecture patterns and their influence on data modeling.

Architecture 130

Architecture Data Engineering Data Engineer Engineering 130

Scott Logic

APRIL 18, 2024

In this episode, Oliver Cronk, Andrew Carr and David Hope talk about the ever-changing world of data, with conversations moving from data warehouse to data lake, and data mesh to data fabric.

Data Lake 59

Data Lake Data Warehouse Data Architecture Architecture 59

Snowflake

OCTOBER 23, 2023

Every day, we witness approximately 20 million Snowpark queries² driving a spectrum of data engineering and data science tasks, with Python leading the way. We are looking forward to working with the incoming Ponder team and the Modin community to bring additional Python capabilities to the Snowflake Data Cloud.

Python 141

Python Cloud Data Science Data Architecture 141

Start Data Engineering

AUGUST 29, 2021

Understanding your data engineering task 2.1. Data infrastructure overview 2.2. Delivering your data engineering task 3.1. You are given a quick overview of the business and data architecture and are assigned your very first data engineering task. Introduction 2. What exactly 2.3. Why exactly 2.4.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Architecture 130

The Modern Data Company

MAY 2, 2023

The Modern Data Company Brief The Modern Data Company is radically simplifying data architecture with its paradigm-shifting data operating system, DataOS. Find out how DataOS can put you on the fastest path from data to decisions. Find out how DataOS can put you on the fastest path from data to decisions.

Data Architecture 98

Data Architecture Government Data Architecture 98

Towards Data Science

OCTOBER 25, 2023

Not too long ago, almost all data architectures and data team structures followed a centralized approach. As a data or analytics engineer, you knew where to find all the transformation logic and models because they were all in the same codebase. There was only one data team, two at most.

Data Pipeline 82

Data Pipeline SQL Data Architecture Data 82

The Modern Data Company

FEBRUARY 26, 2024

Data, the lifeblood of its operations, was becoming increasingly complex and unwieldy. With each new product launch and market expansion, the data architecture that once supported its growth now threatened to be its Achilles’ heel.

Architecture 52

Architecture Data Architecture Data Technology 52

Rockset

JULY 21, 2020

Lessons from scaling facebook's online data infrastructure There are 3 growth numbers that stand out when I look back at the hyper-growth years of facebook from 2007 until 2015, when I was managing facebook's online data infrastructure team: user growth, team growth and infrastructure growth.

Data Architecture 40

Data Architecture Architecture Building Programming Language 40

The Modern Data Company

FEBRUARY 3, 2023

Get to the Future Faster – Modernize Your Manufacturing Data Architecture Without Ripping and Replacing Implementing customer lifetime value as a mission-critical KPI has many challenges. Companies need consistent, high-quality data and a straightforward way to measure CLV.

Manufacturing 98

Manufacturing High Quality Data Data Architecture Architecture 98

Snowflake

DECEMBER 18, 2023

With the race to incorporate AI, and now gen AI, into their business technology toolboxes, data leaders find themselves faced with new opportunities—and challenges. The effort and impact of these data leaders shouldn’t go unnoticed or unappreciated, and so Snowflake is thrilled to announce our “AI-Focused Data Leaders to Watch.”

Transportation 80

Transportation Insurance Data Architecture Technology 80

Cloudera

JUNE 7, 2022

We live in a hybrid data world. In the past decade, the amount of structured data created, captured, copied, and consumed globally has grown from less than 1 ZB in 2011 to nearly 14 ZB in 2020. Impressive, but dwarfed by the amount of unstructured data, cloud data, and machine data – another 50 ZB.

IT 112

IT Unstructured Data Data Architecture Government 112

Snowflake

DECEMBER 18, 2023

With the race to incorporate AI, and now gen AI, into their business technology toolboxes, data leaders find themselves faced with new opportunities—and challenges. The effort and impact of these data leaders shouldn’t go unnoticed or unappreciated, and so Snowflake is thrilled to announce our “AI-Focused Data Leaders to Watch.”

Transportation 77

Transportation Insurance Data Architecture Technology 77

Towards Data Science

DECEMBER 15, 2023

For a data scientist, there’s no such thing as too much data. Photo by Trnava University on Unsplash Data Science vs Security/IT: A Battle for the Ages Acquiring and keeping data is the focus of a huge amount of our mental energy as data scientists. If you ask a data scientist “Can we solve this problem?”

Machine Learning 93

Machine Learning Data Science Data Security Data Storage 93

Data Engineering Podcast

JULY 3, 2022

Summary The ecosystem for data tools has been going through rapid and constant evolution over the past several years. These technological shifts have brought about corresponding changes in data and platform architectures for managing data and analytical workflows. Atlan is the metadata hub for your data ecosystem.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Snowflake

JANUARY 19, 2023

The Chief Data Officer is arguably one of the most important roles at a company, particularly those that aspire to be data-driven. CDO appointments and the elevation of data leaders have accelerated in recent years, and the role has morphed as perceptions of data have evolved. Data leaders deliver.

Data 108

Data Data Governance Data Architecture Government 108

Team Data Science

JANUARY 14, 2021

Job change within Data Science is definitely possible Well, it is possible to switch from one profession to another if only you can learn the fundamental and core things you must know before jumping into it. in data science in their research. Some professionals within the sector can think of switching from one discipline to the other.

Data Science 130

Data Science Computer Science Algorithm Data Engineering 130

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. DataOps helps the data mesh deliver greater business agility by enabling decentralized domains to work in concert. . But first, let’s define the data mesh design pattern.

Pharmaceutical 98

Pharmaceutical Data Lake Data Architecture Scala 98

AltexSoft

FEBRUARY 11, 2023

Data is now one of the most valuable assets for any kind of business. The 11th annual survey of Chief Data Officers (CDOs) and Chief Data and Analytics Officers reveals 82 percent of organizations are planning to increase their investments in data modernization in 2023. What is a data architect?

Data Architect 87

Data Architect Certification Generalist Big Data 87