2006, Database-centric and Metadata - Data Engineering Digest

The Good and the Bad of Apache Spark Big Data Processing

AltexSoft

JULY 18, 2023

Datasets: RDDs can contain any type of data and can be created from data stored in local filesystems, HDFS (Hadoop Distributed File System), databases, or data generated through transformations on existing RDDs. More files within a workload mean more metadata to parse and more tasks to schedule, which can significantly slow down processing.

Big Data

Big Data Data Process Process Hadoop

Journey to Event Driven – Part 4: Four Pillars of Event Streaming Microservices

Confluent

MAY 9, 2019

Storing events in a stream and connecting streams via stream processors provide a generic, data-centric, distributed application runtime that you can use to build ETL, event streaming applications, applications for recording metrics and anything else that has a real-time data requirement. Out of the Tar Pit, 2006.

Kafka

Kafka Pipeline-centric Architecture Database-centric

Data Engineering Digest

The Good and the Bad of Apache Spark Big Data Processing

Journey to Event Driven – Part 4: Four Pillars of Event Streaming Microservices

Webinars

Stay Connected