Christophe Blefari

AUGUST 4, 2023

The modern data stack as a collection of tools which interacts altogether to serve data to consumers is still relevant. Personally I think that the modern data stack characterises by having a central data storage in which everything happens. I want something cloud agnostic—when possible.

Google Cloud 225

Google Cloud MongoDB NoSQL Data 225

Cloudera

NOVEMBER 1, 2022

In the private sector, excluding highly regulated industries like financial services, the migration to the public cloud was the answer to most IT modernization woes, especially those around data, analytics, and storage. It’s here where the private cloud delivers.

Cloud 87

Cloud Government Cloud Computing Data Architecture 87

Christophe Blefari

JANUARY 20, 2024

Data engineering inherits from years of data practices in US big companies. Hadoop initially led the way with Big Data and distributed computing on-premise to finally land on Modern Data Stack — in the cloud — with a data warehouse at the center. This is often linked to real-time.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Edureka

JUNE 1, 2023

This is where AWS Data Analytics comes into action, providing businesses with a robust, cloud-based data platform to manage, integrate, and analyze their data. In this blog, we’ll explore the world of Cloud Data Analytics and a real-life application of AWS Data Analytics. How can the Cloud Help?

AWS 52

AWS Data Analytics Cloud Amazon Web Services 52

Cloudera

JANUARY 26, 2022

Modern, real-time businesses require accelerated cycles of innovation that are expensive and difficult to maintain with legacy data platforms. Cloud technologies and respective service providers have evolved solutions to address these challenges. . But how did the hybrid cloud come to dominate the data sector?

Cloud 85

Cloud Cloud Computing Hadoop Data Warehouse 85

Christophe Blefari

SEPTEMBER 25, 2023

A guide to the Snowflake results cache — Cache is a critical piece to every data warehouse either for reusing data between runs or between stages in the same run. Tabular is the company providing a cloud platform on top of Apache Iceberg—developed by Iceberg founders. Tabular raised $26m Series B.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

Christophe Blefari

SEPTEMBER 25, 2023

A guide to the Snowflake results cache — Cache is a critical piece to every data warehouse either for reusing data between runs or between stages in the same run. Tabular is the company providing a cloud platform on top of Apache Iceberg—developed by Iceberg founders. Tabular raised $26m Series B.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

Knowledge Hut

MARCH 28, 2024

Azure Data Engineer Career Demands & Benefits Azure has become one of the most powerful platforms in the industry, where Microsoft offers a variety of data services and analytics tools. As a result, organizations are looking to capitalize on cloud-based data solutions.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Christophe Blefari

SEPTEMBER 28, 2023

That's why big data technologies got swooshed by the modern data stack when it arrived on the market—excepting Spark. We jumped from HDFS to Cloud Storage (S3, GCS) for storage and from Hadoop, Spark to Cloud warehouses (Redshift, BigQuery, Snowflake) for processing. Cloud-first.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Christophe Blefari

NOVEMBER 11, 2022

Ingredients of a Data Warehouse Going back to basics. Kovid wrote an article that tries to explain what are the ingredients of a data warehouse. A data warehouse is a piece of technology that acts on 3 ideas: the data modeling, the data storage and processing engine.

BI 130

BI Data Warehouse Data Database 130

Towards Data Science

MARCH 23, 2023

The difference between tables and views and how to use them Continue reading on Towards Data Science »

Data Warehouse 74

Data Warehouse Data Science Data Cloud Computing 74

Monte Carlo

APRIL 27, 2023

Empowered by the rise of the modern data stack , leading companies like Warner Music Group , Chime and Petsmart are now turning to the Composable CDP to unlock a modular customer data architecture that centers on the cloud data warehouse. Why not build one single store of customer data to activate from centrally?

Data Warehouse 72

Data Warehouse Data Collection Architecture Data Storage 72

Monte Carlo

MAY 28, 2024

Data Transformation : Clean, format, and convert extracted data to ensure consistency and usability for both batch and real-time processing. Data Loading : Load transformed data into the target system, such as a data warehouse or data lake. Used for identifying and cataloging data sources.

Data Ingestion 52

Data Ingestion Architecture Designing Hadoop 52

Cloudera

OCTOBER 25, 2022

Enabling data and analytics in the cloud allows you to have infinite scale and unlimited possibilities to gain faster insights and make better decisions with data. Cloud data lakehouses provide significant scaling, agility, and cost advantages compared to cloud data lakes and cloud data warehouses.

Cloud 52

Cloud AWS Data Lake Amazon Web Services 52

Christophe Blefari

FEBRUARY 24, 2023

The current data state is heavily dependent on infrastructure, wether it's cloud, on-premise or semi-related we need to understand where the data lands and where the code runs. At the moment the best way to develop in your data warehouse is still in the query editor of BigQuery / Snowflake / etc.,

Kafka 130

Kafka Data Lake Data Storage Data 130

Cloudera

AUGUST 31, 2021

Cloudera and Accenture demonstrate strength in their relationship with an accelerator called the Smart Data Transition Toolkit for migration of legacy data warehouses into Cloudera Data Platform. Accenture’s Smart Data Transition Toolkit . Are you looking for your data warehouse to support the hybrid multi-cloud?

Data Warehouse 82

Data Warehouse Database-centric Metadata Cloud 82

Cloudera

DECEMBER 12, 2018

After having rebuilt their data warehouse, I decided to take a little bit more of a pointed role, and I joined Oracle as a database performance engineer. I spent eight years in the real-world performance group where I specialized in high visibility and high impact data warehousing competes and benchmarks.

Data Warehouse 42

Data Warehouse Relational Database Hadoop BI 42

phData: Data Engineering

SEPTEMBER 19, 2023

This blog will guide you through the best data modeling methodologies and processes for your data lake, helping you make informed decisions and optimize your data management practices. What is a Data Lake? What are Data Modeling Methodologies, and Why Are They Important for a Data Lake?

Data Lake 52

Data Lake Process Metadata Data Warehouse 52

Hevo

AUGUST 17, 2023

Organizations rely on multiple databases and cloud-based solutions for their ever-expanding data storage needs. However, for in-depth analysis of organizational data, it is essential to perform seamless integrations between different platforms.

MySQL 52

MySQL Data Warehouse Data Storage Database 52

Knowledge Hut

NOVEMBER 17, 2023

Additionally, they convert data into formats that can be used and store it effectively and securely in the Azure cloud. The data engineers are responsible for creating conversational chatbots with the Azure Bot Service and automating metric calculations using the Azure Metrics Advisor.

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Christophe Blefari

JUNE 16, 2023

Why data consumers do not trust your reporting — It is a good illustration of the data journey manifesto. Stakeholders often notice data issues before the data team does. Data warehouses are mutable, this is one of the many root causes proposed by Lucas. Data Documentation 101: Why?

Programming Language 130

Programming Language SQL PostgreSQL Data 130

phData: Data Engineering

MARCH 11, 2024

For years, marketing teams across industries have turned to implementing traditional Customer Data Platforms (CDPs) as separate systems purpose-built to unlock growth with first-party data. The ELT platform offers 200+ pre-built connections to centralize data to any data platform. dbt has become the standard for modeling.

Data Warehouse 52

Data Warehouse Data Data Storage Cloud 52

Edureka

FEBRUARY 9, 2023

Azure Data Engineering is a rapidly growing field that involves designing, building, and maintaining data processing systems using Microsoft Azure technologies. As a certified Azure Data Engineer, you have the skills and expertise to design, implement and manage complex data storage and processing solutions on the Azure cloud platform.

Data Engineering 52

Data Engineering Data Engineer Engineering Amazon Web Services 52

Ascend.io

NOVEMBER 21, 2023

The emergence of cloud data warehouses, offering scalable and cost-effective data storage and processing capabilities, initiated a pivotal shift in data management methodologies. Extract The initial stage of the ELT process is the extraction of data from various source systems.

Raw Data 52

Raw Data Data Warehouse Data Cleanse Data Integration 52

Knowledge Hut

NOVEMBER 28, 2023

The contemporary world experiences a huge growth in cloud implementations, consequently leading to a rise in demand for data engineers and IT professionals who are well-equipped with a wide range of application and process expertise. This can be easier when you are using existing cloud services.

Data Engineering 52

Data Engineering Data Engineer Engineering Generalist 52

Knowledge Hut

NOVEMBER 19, 2023

One of the most important responsibilities for experts in big data is configuring the cloud to store data and provide high availability. As a result, data engineers working with big data today require a basic grasp of cloud computing platforms and tools.

Data Engineering 52

Data Engineering Data Engineer Engineering PostgreSQL 52

Databand.ai

JULY 19, 2023

A Beginner’s Guide [SQ] Niv Sluzki July 19, 2023 ELT is a data processing method that involves extracting data from its source, loading it into a database or data warehouse, and then later transforming it into a format that suits business needs. The data is loaded as-is, without any transformation.

Data Cleanse 52

Data Cleanse Raw Data Data Storage Data Warehouse 52

Towards Data Science

DECEMBER 1, 2023

This tool automates ELT (Extract, Load, Transform) process, integrating your data from the source system of Google Calendar to our Snowflake data warehouse. Storage —  Snowflake Snowflake, a cloud-based data warehouse tailored for analytical needs, will serve as our data storage solution.

Data Engineering 70

Data Engineering Data Engineer Project Engineering 70

Monte Carlo

FEBRUARY 20, 2024

Integrations : They offer a wide array of connectors for databases, SaaS applications, cloud storage solutions, and more, covering both popular and niche data sources. Batch vs Streaming : They focus on automated batch data integration but also support near real-time data replication for certain sources.

Data Ingestion 52

Data Ingestion Google Cloud Kafka AWS 52

Data Engineering Weekly

DECEMBER 2, 2023

Revisiting The Current State of Data Infrastructure Let’s revisit the current state of the data infrastructure before discussing the S3 Express. There are two critical properties of data warehouse access patterns. Data freshness matters a lot—the more recent the data, the more frequently it is accessed.

IT 85

IT BI AWS Kafka 85

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Monte Carlo

APRIL 24, 2023

Data lakes are useful, flexible data storage repositories that enable many types of data to be stored in its rawest state. Traditionally, after being stored in a data lake, raw data was then often moved to various destinations like a data warehouse for further processing, analysis, and consumption.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

Knowledge Hut

OCTOBER 6, 2023

Using Data Analytics to Learn abilities: The AWS Data Analytics certification is a great way to learn crucial data analysis abilities. It covers data gathering, cloud computing, data storage, processing, analysis, visualization, and data security.

AWS 52

AWS Certification Data Analytics IT 52

AltexSoft

MARCH 14, 2023

As the volume and complexity of data continue to grow, organizations seek faster, more efficient, and cost-effective ways to manage and analyze data. In recent years, cloud-based data warehouses have revolutionized data processing with their advanced massively parallel processing (MPP) capabilities and SQL support.

IT 59

IT Data Warehouse Data Governance Data Lake 59

DareData

JANUARY 30, 2023

In this post, we'll discuss some key data engineering concepts that data scientists should be familiar with, in order to be more effective in their roles. These concepts include concepts like data pipelines, data storage and retrieval, data orchestrators or infrastructure-as-code.

Data Engineering 52

Data Engineering Data Engineer NoSQL Engineering 52

Knowledge Hut

APRIL 25, 2023

Data engineers add meaning to the data for companies, be it by designing infrastructure or developing algorithms. The practice requires them to use a mix of various programming languages, data warehouses, and tools. While they go about it - enter big data data engineer tools. What are Data Engineering Tools?

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Monte Carlo

MARCH 28, 2023

As part of this movement, Fivetran and dbt fundamentally altered the data pipeline from ETL to ELT. Hightouch interrupted SaaS eating the world in an attempt to shift the center of gravity to the data warehouse. Other common light transformations done within the ingestion phase are data formatting and deduplication.

Data Warehouse 126

Data Warehouse Raw Data Data Pipeline Software Engineer 126