Monte Carlo

MARCH 12, 2024

Google Cloud’s Dataplex is a data fabric tool that enables organizations to discover, manage, monitor, and govern their data across all of their data systems, including their data lakes, data warehouses, data lakehouses, and data marts. Courtesy of Google Cloud.

Google Cloud 52

Google Cloud Metadata SQL Data Lake 52

DareData

JULY 5, 2023

Learn how we build data lake infrastructures and help organizations all around the world achieving their data goals. In today's data-driven world, organizations are faced with the challenge of managing and processing large volumes of data efficiently.

Data Lake 52

Data Lake Building Raw Data ETL Tools 52

dbt Developer Hub

AUGUST 2, 2022

If we wanted to run the test only for the models in the mart folder, we could write ^models/mart. If we are using dbt Cloud, let’s jump to step 4 , where we will set up pre-commit-dbt as part of the CI process, otherwise, with dbt Core we can go to step 2.

Python 52

Python SQL Data Warehouse Project 52

Big Data Tools

AUGUST 3, 2021

August is a good time to start new things – some people are on vacation and have more spare time to read than usual, while others are back and looking for a quick refresher on what’s new in data engineering. Without further ado, let’s get started with this inaugural issue of Data Engineering Annotated! Marie Kondo would be proud!

Data Engineering 52

Data Engineering Data Engineer Engineering Big Data Tools 52

Big Data Tools

AUGUST 3, 2021

August is a good time to start new things – some people are on vacation and have more spare time to read than usual, while others are back and looking for a quick refresher on what’s new in data engineering. Without further ado, let’s get started with this inaugural issue of Data Engineering Annotated! Marie Kondo would be proud!

Data Engineering 52

Data Engineering Data Engineer Engineering Big Data Tools 52

Data Engineering Weekly

FEBRUARY 19, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. The focus is to bring in data in-specific to their media assets and build a feature store.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Cloudera

OCTOBER 28, 2020

By using data to listen to their customers better. The most innovative companies use data and analytics to offer appropriate products and services. To start, they look to traditional financial services data, combining and correlating account activity, borrowing history, core banking, investments, and call center data.

Unstructured Data 70

Unstructured Data Banking Machine Learning Media 70

Cloudera

OCTOBER 19, 2021

Airflow has been adopted by many Cloudera Data Platform (CDP) customers in the public cloud as the next generation orchestration service to setup and operationalize complex data pipelines. Until now, the setup of such pipelines still required knowledge of Airflow and the associated python configurations.

Coding 115

Coding Data Engineering Data Engineer Engineering 115

Cloudera

DECEMBER 21, 2020

In this blog we will take you through a persona-based data adventure, with short demos attached, to show you the A-Z data worker workflow expedited and made easier through self-service, seamless integration, and cloud-native technologies. In our data adventure we assume the following: . The Data Scientist.

Banking 97

Banking Data Lake Data Data Warehouse 97

ProjectPro

FEBRUARY 8, 2023

ETL developers play a vital role in designing, implementing, and maintaining the processes that help organizations extract valuable business insights from data. The purpose of ETL is to provide a centralized, consistent view of the data used for reporting and analysis. What is an ETL Developer?

ETL Tools 52

ETL Tools Data Cleanse Data Warehouse Big Data 52

AltexSoft

AUGUST 4, 2021

Now let’s think of sweets as the data required for your company’s daily operations. Instead of combing through the vast amounts of all organizational data stored in a data warehouse, you can use a data mart — a repository that makes specific pieces of data available quickly to any given business unit.

Data Lake 64

Data Lake Data Warehouse ETL Tools Telecommunication 64

Knowledge Hut

OCTOBER 30, 2023

For any organization to grow, it requires business intelligence reports and data to offer insights to aid in decision-making. This data and reports are generated and developed by Power BI developers. The work of a Power BI developer is to take data in its raw form, derive meaning, and make sense of it.

BI 52

BI Business Intelligence Data Cleanse Business Analyst 52

Monte Carlo

DECEMBER 20, 2023

Monitoring for bad data is like fishing. And, even in the cloud, you can’t be everywhere. So what is the optimal data observability architecture or data quality coverage strategy? Each company organizes their data and business slightly differently so the answer will vary, but there are some guiding principles to follow.

Architecture 40

Architecture Data Datasets Machine Learning 40

phData: Data Engineering

JULY 12, 2022

Within data engineering , one of the most frequent tasks is modeling data into data marts and data products. Traditionally, if a company wanted to model some data, there were two types of resources available that could accomplish this: data engineers or data analysts. What Is Data Modeling?

Data Engineering 52

Data Engineering Data Engineer SQL Data 52

Data Engineering Weekly

DECEMBER 21, 2022

The Rise of Data Modeling Data modeling has been one of the hot topics in Data LinkedIn. Hadoop put forward the schema-on-read strategy that leads to the disruption of data modeling techniques as we know until then. Let’s reference what the data world looked like before the Hadoop era. Why is it so?

Data Engineering 57

Data Engineering Data Engineer Engineering Hadoop 57

Monte Carlo

MARCH 16, 2023

Fintech consumer lending marketplace Best Egg has built its products—including personal loans, debt consolidation, credit cards, and a financial health platform—on a foundation of data. Employee number three at Best Egg was Randy Addis, now Head of Data and Business Intelligence.

Software Engineer 52

Software Engineer Software Engineering Data Governance Government 52

DataKitchen

NOVEMBER 4, 2021

Data organizations often have a mix of centralized and decentralized activity. DataOps concerns itself with the complex flow of data across teams, data centers and organizational boundaries. It expands beyond tools and data architecture and views the data organization from the perspective of its processes and workflows.

Process 98

Process Data Process Pharmaceutical Data Lake 98

Snowflake

APRIL 5, 2023

Two different data modeling approaches—dimensional data modeling and Data Vault—each have their own pros and cons. Modernizing a data warehouse with Snowflake Data Cloud is a smart investment that can provide significant benefits to businesses of all sizes, today more than ever as data models become ever more complex.

Data Warehouse 52

Data Warehouse Healthcare Unstructured Data Metadata 52

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data. What is a data lake?

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

dbt Developer Hub

MAY 16, 2022

But this means the name has a life outside of dbt and supports the many end users who will potentially never know about dbt and where this data came from, but still access the database objects in the database or business intelligence (BI) tool. With these goals in mind, they must form a hypothesis and prove their point with data.

BI 52

BI Data Warehouse Database SQL 52

Monte Carlo

MARCH 16, 2023

Fintech consumer lending marketplace Best Egg has built its products—including personal loans, debt consolidation, credit cards, and a financial health platform—on a foundation of data. Employee number three at Best Egg was Randy Addis, now Head of Data and Business Intelligence.

Software Engineer 40

Software Engineer Software Engineering Data Governance Government 40

Cloudera

OCTOBER 26, 2017

Become more agile with business intelligence and data analytics. Clouds (source: Pexels ). Many of us are all too familiar with the traditional way enterprises operate when it comes to on-premises data warehousing and data marts: the enterprise data warehouse (EDW) is often the center of the universe.

Cloud 42

Cloud BI Cloud Storage Business Intelligence 42

Monte Carlo

APRIL 16, 2022

The data warehouse is the foundation of the modern data stack, so it caught our attention when we saw Convoy head of data Chad Sanderson declare, “ the data warehouse is broken ” on LinkedIn. I’ll let you decide for yourself if the “immutable data warehouse” (or active vs passive ETL) is the right path for your data team.

Data Warehouse 52

Data Warehouse Architecture Data Government 52

AltexSoft

OCTOBER 18, 2022

Everyday the global healthcare system generates tons of medical data that — at least, theoretically — could be used for machine learning purposes. Regardless of industry, data is considered a valuable resource that helps companies outperform their rivals, and healthcare is not an exception. Medical data labeling. Let’s sum up.

Medical 52

Medical Datasets Machine Learning Hospitality 52

ProjectPro

AUGUST 11, 2021

“Data Lake vs Data Warehouse = Load First, Think Later vs Think First, Load Later” The terms data lake and data warehouse are frequently stumbled upon when it comes to storing large volumes of data. Data Warehouse Architecture What is a Data lake? Is Hadoop a data lake or data warehouse?

Data Lake 52

Data Lake Data Warehouse Cloud Hadoop 52

Cloudera

OCTOBER 26, 2020

Cloudera and Dell/EMC are continuing our long and successful partnership of developing shared storage solutions for analytic workloads running in hybrid cloud. . Since the inception of Cloudera Data Platform (CDP), Dell / EMC PowerScale and ECS have been highly requested solutions to be certified by Cloudera.

Certification 84

Certification Cloud Kafka Unstructured Data 84

Christophe Blefari

DECEMBER 19, 2022

Last week dbt Labs decided to change the pricing of their Cloud offering. of the Data News. In a nutshell, dbt Cloud pricing is per seat based, which means you pay for each dbt developer. Sometimes data teams forget it. I've already analysed this in week #22.50 You can do development on your codebase.

Management 130

Management Pipeline-centric Database-centric SQL 130

Cloudera

NOVEMBER 5, 2020

Many organizations struggle to meet growing and variable data warehouse demands. How do you control data privacy and protect against data breaches when the data is spread across so many different systems? How do you optimize your enterprise-wide infrastructure (mostly cloud) and application expenditures?

Cloud 70

Cloud Data Lake Data Warehouse Metadata 70

dbt Developer Hub

OCTOBER 11, 2022

The power of a semantic layer on top of a mature data modeling framework ​ As a longtime dbt Community member, I knew I had to get involved when I first saw the dbt Semantic Layer in the now infamous dbt should know about metrics Github Issue. Now, the mart is the springboard for the creation of our metric. What do we mean?

Designing 52

Designing Finance Datasets Coding 52

dbt Developer Hub

FEBRUARY 12, 2024

What’s in a data platform? We, as data professionals, have poured ourselves into raising happy healthy data products, and we should be proud of the insights they’ve driven. A lot of blood, sweat, and tears are shed in the service of clean data! Where’s this data coming from? Raising a dbt project is hard work.

Metadata 59

Metadata Raw Data BI Project 59

Edureka

FEBRUARY 9, 2023

Azure Data Engineering is a rapidly growing field that involves designing, building, and maintaining data processing systems using Microsoft Azure technologies. As a certified Azure Data Engineer, you have the skills and expertise to design, implement and manage complex data storage and processing solutions on the Azure cloud platform.

Data Engineering 52

Data Engineering Data Engineer Engineering Amazon Web Services 52

Snowflake

MARCH 30, 2023

A 2016 data science report from data enrichment platform CrowdFlower found that data scientists spend around 80% of their time in data preparation (collecting, cleaning, and organizing of data) before they can even begin to build machine learning (ML) models to deliver business value.

Engineering 82

Engineering Raw Data Data Science Scala 82

Knowledge Hut

JANUARY 24, 2024

Step into the realm of data visualization with a comprehensive exploration of Power BI and Tableau. In a world where data is important, deciding between power bi vs tableau can change your path in analyzing things. Are you ready to unlock my data potential and master advanced analytics! Utilizing Power BI is simple.

BI 98

BI Business Intelligence Non-relational Database Data Analysis 98

dbt Developer Hub

DECEMBER 5, 2021

With every new engagement, we find ourselves working in a project with a unique mix of data challenges. company, and we want to try out dbt to model our patient, doctor and claims data for a critical KPI dashboard. We’ve also included some sample raw data to add to your warehouse so you can run these projects yourself!

Project 52

Project Building Metadata BI 52

Cloudera

AUGUST 17, 2021

When we announced the GA of Cloudera Data Engineering back in September of last year, a key vision we had was to simplify the automation of data transformation pipelines at scale. With Airflow based pipelines in DE, customers can now specify their data pipeline using a simple python configuration file.

Data Pipeline 111

Data Pipeline Management BI Python 111

AltexSoft

JUNE 26, 2023

All successful companies do it: constantly collect data. While today’s world abounds with data, gathering valuable information presents a lot of organizational and technical challenges, which we are going to address in this article. What is data collection?

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

Monte Carlo

JANUARY 12, 2023

For Freshly, food isn’t the only thing that needs to be delivered fresh and fast; our data also needs to be reliable, timely, and most importantly, accurate. Our modern data platform architecture story at Freshly started almost four years ago. Today, we have a data team of about 20 professionals organized among three teams.

Architecture 52

Architecture Building Data Warehouse BI 52