Data Engineering Weekly

MARCH 3, 2024

Here is the agenda, 1) Data Application Lifecycle Management - Harish Kumar( Paypal) Hear from the team in PayPal on how they build the data product lifecycle management (DPLM) systems. 3) DataOPS at AstraZeneca The AstraZeneca team talks about data ops best practices internally established and what worked and what didn’t work!!!

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Ripple Engineering

JANUARY 29, 2024

For Ripple's product capabilities, the Payments team of Ripple, for example, ingests millions of transactional records into databases and performs analytics to generate invoices, reports, and other related payment operations.    A lack of a centralized system makes building a single source of high-quality data difficult.

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

Ascend.io

JUNE 8, 2023

Treating data as a product is more than a concept; it’s a paradigm shift that can significantly elevate the value that business intelligence and data-centric decision-making have on the business. It is the stage where data truly becomes a product, delivering tangible value to its end users.

Pipeline-centric 52

Pipeline-centric Database-centric Data Ingestion Data Pipeline 52

Monte Carlo

JANUARY 23, 2024

RAG involves integrating a real-time database into the LLM’s response generation process, while fine-tuning trains models on targeted datasets to improve domain-specific responses. That implies working with new patterns like vector databases.” Those who don’t embrace it will be left behind. With the right prompt (this is key!)

Pipeline-centric 52

Pipeline-centric Database-centric Metadata Unstructured Data 52

Knowledge Hut

JANUARY 24, 2024

However, beneath the surface of these data-centric activities lies the core role of a data scientist – that of a problem solver. Beyond manipulating and analyzing data, data scientists are fundamentally problem solvers. This involves writing scripts, using data extraction tools, and ensuring data quality.

Database-centric 52

Database-centric Data Science Machine Learning Datasets 52

DataKitchen

NOVEMBER 4, 2021

It’s too hard to change our IT data product. Can we create high-quality data in an “answer-ready” format that can address many scenarios, all with minimal keyboarding? . “I I get cut off at the knees from a data perspective, and I am getting handed a sandwich of sorts and not a good one!”.

Process 98

Process Data Process Pharmaceutical Data Lake 98

Maxime Beauchemin

JANUARY 20, 2017

Data modeling is changing Typical data modeling techniques — like the star schema  — which defined our approach to data modeling for the analytics workloads typically associated with data warehouses, are less relevant than they once were. Those systems have been taught to normalize the data for storage on their own.

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130