Data Ingestion, Data Workflow, Raw Data and Structured Data

Data Engineering Zoomcamp – Data Ingestion (Week 2)

Hepta Analytics

FEBRUARY 14, 2022

DE Zoomcamp 2.2.1 – Introduction to Workflow Orchestration Following last weeks blog , we move to data ingestion. We already had a script that downloaded a csv file, processed the data and pushed the data to postgres database. This week, we got to think about our data ingestion design.

Data Ingestion

Data Ingestion Data Engineering Data Engineer Engineering

Data Pipeline Architecture Explained: 6 Diagrams and Best Practices

Monte Carlo

JUNE 14, 2023

Why is data pipeline architecture important? Databricks – Databricks, the Apache Spark-as-a-service platform, has pioneered the data lakehouse, giving users the options to leverage both structured and unstructured data and offers the low-cost storage features of a data lake.

Data Pipeline

Data Pipeline Architecture Data Lake Data Warehouse

The Modern Data Stack: What It Is, How It Works, Use Cases, and Ways to Implement

AltexSoft

MARCH 14, 2023

Data storage The tools mentioned in the previous section are instrumental in moving data to a centralized location for storage, usually, a cloud data warehouse, although data lakes are also a popular option. But this distinction has been blurred with the era of cloud data warehouses.

IT Data Warehouse Data Governance Data Lake

Data Engineering Digest

Data Engineering Zoomcamp – Data Ingestion (Week 2)

Data Pipeline Architecture Explained: 6 Diagrams and Best Practices

The Modern Data Stack: What It Is, How It Works, Use Cases, and Ways to Implement

Webinars

Stay Connected