Data Engineering Weekly

MARCH 3, 2024

Here is the agenda, 1) Data Application Lifecycle Management - Harish Kumar( Paypal) Hear from the team in PayPal on how they build the data product lifecycle management (DPLM) systems. 3) DataOPS at AstraZeneca The AstraZeneca team talks about data ops best practices internally established and what worked and what didn’t work!!!

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Knowledge Hut

DECEMBER 26, 2023

Data Engineering is typically a software engineering role that focuses deeply on data – namely, data workflows, data pipelines, and the ETL (Extract, Transform, Load) process. What is the role of a Data Engineer? Data scientists and data Analysts depend on data engineers to build these data pipelines.

Data Engineering 98

Data Engineering Data Engineer Engineering Pipeline-centric 98

Databand.ai

JULY 19, 2023

This capability is useful for businesses, as it provides a clear and comprehensive view of their data’s history and transformations. Data lineage tools are not a new concept. In this article: Why Are Data Lineage Tools Important? Atlan Atlan offers a modern approach to data governance.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

Snowflake

JUNE 28, 2023

Snowpark is our secure deployment and processing of non-SQL code, consisting of two layers: Familiar Client Side Libraries – Snowpark brings deeply integrated, DataFrame-style programming and OSS compatible APIs to the languages data practitioners like to use. Previously, tasks could be executed as quickly as 1-minute.

Python 52

Python Accessible Accessibility Pipeline-centric 52

Knowledge Hut

SEPTEMBER 26, 2023

Key Features of Azure Synapse Here are some of the key features of Azure Synapse: Cloud Data Service: Azure Synapse operates as a cloud-native service, residing within the Microsoft Azure cloud ecosystem. This cloud-centric approach ensures scalability, flexibility, and cost-efficiency for your data workloads.

Data Lake 52

Data Lake Database-centric Pipeline-centric Machine Learning 52