Data Lake, Data Workflow, Raw Data and Structured Data

Data Engineering Zoomcamp – Data Ingestion (Week 2)

Hepta Analytics

FEBRUARY 14, 2022

This week, we got to think about our data ingestion design. We looked at the following: How do we ingest – ETL vs ELT Where do we store the data – Data lake vs data warehouse Which tool to we use to ingest – cronjob vs workflow engine NOTE : This weeks task requires good internet speed and good compute.

Data Ingestion

Data Ingestion Data Engineering Data Engineer Engineering

Data Pipeline Architecture Explained: 6 Diagrams and Best Practices

Monte Carlo

JUNE 14, 2023

The modern data stack era , roughly 2017 to present data, saw the widespread adoption of cloud computing and modern data repositories that decoupled storage from compute such as data warehouses, data lakes, and data lakehouses. It is like a smart scheduler for your data workflows.

Data Pipeline

Data Pipeline Architecture Data Lake Data Warehouse

The Modern Data Stack: What It Is, How It Works, Use Cases, and Ways to Implement

AltexSoft

MARCH 14, 2023

Built around a cloud data warehouse, data lake, or data lakehouse. Modern data stack tools are designed to integrate seamlessly with cloud data warehouses such as Redshift, Bigquery, and Snowflake, as well as data lakes or even the child of the first two — a data lakehouse.

IT Data Warehouse Data Governance Data Lake

Data Engineering Digest

Data Engineering Zoomcamp – Data Ingestion (Week 2)

Data Pipeline Architecture Explained: 6 Diagrams and Best Practices

The Modern Data Stack: What It Is, How It Works, Use Cases, and Ways to Implement

Webinars

Stay Connected