The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. Potential downsides of data lakes include governance and integration challenges.

Data Management 52

Data Management Data Lake Management Data Governance 52

The Modern Data Company

MAY 8, 2023

Data lakes, data warehouses, data hubs, data lakehouses, and data operating systems are data management and storage solutions designed to meet different needs in data analytics, integration, and processing. Potential downsides of data lakes include governance and integration challenges.

Data Management 52

Data Management Data Lake Management Data Governance 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Data Lake Management Data Warehouse 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Data Lake Management Data Warehouse 52

The Modern Data Company

MAY 10, 2023

In our previous post, The Pros and Cons of Leading Data Management and Storage Solutions , we untangled the differences among data lakes, data warehouses, data lakehouses, data hubs, and data operating systems. What factors are most important when building a data management ecosystem?

Data Management 52

Data Management Data Lake Management Data Warehouse 52

Data Engineering Podcast

FEBRUARY 18, 2024

Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). To start, can you share your definition of what constitutes a "Data Lakehouse"? Closing Announcements Thank you for listening!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Monte Carlo

FEBRUARY 6, 2023

So, you’re planning a cloud data warehouse migration. But be warned, a warehouse migration isn’t for the faint of heart. As you probably already know if you’re reading this, a data warehouse migration is the process of moving data from one warehouse to another. A worthy quest to be sure.

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

Monte Carlo

AUGUST 25, 2023

Different vendors offering data warehouses, data lakes, and now data lakehouses all offer their own distinct advantages and disadvantages for data teams to consider. So let’s get to the bottom of the big question: what kind of data storage layer will provide the strongest foundation for your data platform?

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP).

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

NOVEMBER 16, 2020

In this episode CEO and co-founder of Isima Darshan Rawal explains how the biOS platform is architected to enable ease of use, the challenges that were involved in building an entirely new system from scratch, and how it can integrate with the rest of your data platform to allow for incremental adoption. What is the story behind the name?

Data Management 100

Data Management Management BI Business Intelligence 100

Cloudera

MAY 28, 2024

Modern data architectures deliver key functionality in terms of flexibility and scalability of data management. This form of architecture can handle data in all forms—structured, semi-structured, unstructured—blending capabilities from data warehouses and data lakes into data lakehouses.

Data Architecture 75

Data Architecture Architecture Data Lake Data Warehouse 75

Data Engineering Podcast

FEBRUARY 4, 2024

In this episode Yingjun Wu explains how it is architected to power analytical workflows on continuous data flows, and the challenges of making it responsive and scalable. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Data Science Blog: Data Engineering

MAY 20, 2024

Enter AnalyticsCreator AnalyticsCreator, a powerful tool for data management, brings a new level of efficiency and reliability to the CI/CD process. It offers full BI-Stack Automation, from source to data warehouse through to frontend. It supports a holistic data model, allowing for rapid prototyping of various models.

Data Pipeline 52

Data Pipeline BI Data Lake Data Warehouse 52

Jesse Anderson

NOVEMBER 15, 2022

My guest this week is Kulani Likotsi , the Head of Data Management and Data Governance at one of the four biggest banks in Africa. She’s had a rising career journey going from an analyst, to a Business Intelligence developer, to the data warehouse team, to the data governance team.

Data Governance 130

Data Governance Government Banking Business Intelligence 130

Knowledge Hut

MARCH 7, 2024

Two, it creates a commonality of data definitions, concepts, metadata and the like. The traditional data management and data warehouses, and the sequence of data transformation, extraction and migration- all arise a situation in which there are risks for data to become unsynchronized.

Big Data 98

Big Data Bytes Data Governance Raw Data 98

Knowledge Hut

MARCH 28, 2024

Data Engineer Career: Overview Currently, with the enormous growth in the volume, variety, and veracity of data generated and the will of large firms to store and analyze their data, data management is a critical aspect of data science. That’s where data engineers are on the go.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

AltexSoft

MARCH 14, 2023

As the volume and complexity of data continue to grow, organizations seek faster, more efficient, and cost-effective ways to manage and analyze data. In recent years, cloud-based data warehouses have revolutionized data processing with their advanced massively parallel processing (MPP) capabilities and SQL support.

IT 59

IT Data Warehouse Data Governance Data Lake 59

Databand.ai

JULY 19, 2023

A Beginner’s Guide [SQ] Niv Sluzki July 19, 2023 ELT is a data processing method that involves extracting data from its source, loading it into a database or data warehouse, and then later transforming it into a format that suits business needs. The data is loaded as-is, without any transformation.

Data Cleanse 52

Data Cleanse Raw Data Data Storage Data Warehouse 52

Monte Carlo

MAY 14, 2024

This development was crucial for enabling both batch and streaming data workflows in dynamic environments, ensuring consistency and durability in big data processing. The new kids on the table format block Within the last five or six years, as data management needs have grown in complexity and scale, newer table formats have emerged.

Data Lake 52

Data Lake Metadata Hadoop Data Governance 52

Databand.ai

AUGUST 30, 2023

DataOps , short for data operations, is an emerging discipline that focuses on improving the collaboration, integration, and automation of data processes across an organization. These tools help organizations implement DataOps practices by providing a unified platform for data teams to collaborate, share, and manage their data assets.

Data Cleanse 52

Data Cleanse Data Pipeline Data Ingestion Data Validation 52

Databand.ai

JULY 19, 2023

Data lineage tools are not a new concept. However, their importance has grown significantly in recent years due to the increasing complexity of data architectures and the growing need for data governance and compliance. In this article: Why Are Data Lineage Tools Important?

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

Data Engineering Podcast

JUNE 11, 2021

He shares some of the common patterns for building pipelines to power business intelligence dashboards, machine learning applications, and data warehouses. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? No more scripts, just SQL.

Google Cloud 100

Google Cloud Cloud Big Data Ecosystem BI 100

Data Engineering Podcast

JULY 12, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. RudderStack’s smart customer data pipeline is warehouse-first.

Designing 100

Designing Data Warehouse Data Governance Consulting 100

Data Engineering Podcast

APRIL 26, 2021

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Business Intelligence 100

Business Intelligence Data Warehouse Hadoop Data Pipeline 100

Data Engineering Podcast

JUNE 5, 2022

In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information. Data teams are increasingly under pressure to deliver.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

phData: Data Engineering

SEPTEMBER 19, 2023

With many data modeling methodologies and processes available, choosing the right approach can be daunting. This blog will guide you through the best data modeling methodologies and processes for your data lake, helping you make informed decisions and optimize your data management practices. What is a Data Lake?

Data Lake 52

Data Lake Process Metadata Data Warehouse 52

Snowflake

NOVEMBER 14, 2023

Several outcomes factored into Snowflake’s performance rating: Among the top customer outcomes and benefits reported by respondents are improved operational efficiency (88%) and cost-effectiveness (38%), followed by improved data integration and decision-making capabilities. Manager, Aug. Director, Sep.

Healthcare 84

Healthcare Data Warehouse Data Governance Cloud 84

Data Engineering Podcast

OCTOBER 15, 2021

They also share their ambitions for the near future of adding data observability and data quality management features. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Metadata 100

Metadata BI Data Warehouse Government 100

Precisely

MAY 13, 2024

Wide support for enterprise-grade sources and targets Large organizations with complex IT landscapes must have the capability to easily connect to a wide variety of data sources. Whether it’s a cloud data warehouse or a mainframe, look for vendors who have a wide range of capabilities that can adapt to your changing needs.

Data Integration 52

Data Integration Metadata Amazon Web Services Data Governance 52

Data Engineering Podcast

APRIL 5, 2021

She explains how the design of the platform is informed by the needs of managing data projects for large and small teams across her previous roles, how it integrates with your existing systems, and how it can work to bring everyone onto the same page. RudderStack’s smart customer data pipeline is warehouse-first.

Data Warehouse 100

Data Warehouse Data Pipeline BI Metadata 100

Precisely

OCTOBER 16, 2023

The cloud is especially well-suited to large-scale storage and big data analytics, due in part to its capacity to handle intensive computing requirements at scale. BI platforms and data warehouses have been replaced by modern data lakes and cloud analytics solutions. Secure data exchange takes on much greater importance.

Cloud 70

Cloud Cloud Computing Metadata Data Governance 70

Towards Data Science

JUNE 8, 2023

This post will focus on the most common team ownership models including: data engineering, data reliability engineering, analytics engineering, data quality analysts, and data governance teams. Why is data quality ownership important? The governance team treats every team output as a data product.

Data Governance 80

Data Governance Government Generalist Data Engineering 80

Data Engineering Podcast

MARCH 15, 2021

This is a valuable look at how to manage a large and growing data platform with that supports a variety of teams with varied and evolving needs. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Management 100

Management Data Warehouse PostgreSQL Kafka 100

Christophe Blefari

JUNE 16, 2023

Why data consumers do not trust your reporting — It is a good illustration of the data journey manifesto. Stakeholders often notice data issues before the data team does. Data warehouses are mutable, this is one of the many root causes proposed by Lucas. Data Documentation 101: Why?

Programming Language 130

Programming Language SQL PostgreSQL Data 130

Monte Carlo

JUNE 6, 2023

This post will focus on the most common team ownership models including: data engineering, data reliability engineering, analytics engineering, data quality analysts, and data governance teams. Table of Contents Why is important to answer who is responsible for data quality?

Data Governance 98

Data Governance Government Data Data Engineering 98

Towards Data Science

JULY 24, 2023

A data engineering manager at a Fortune 500 company expressed the pain of on-prem limitations to me by saying: “Our analysts were unable to run the queries they wanted to run when they wanted to run them. Why are these things related, and more importantly, why should data leaders care? Take data governance for example.

Metadata 97

Metadata Data Warehouse Data Lake Data Science 97