Data Engineering Podcast

JUNE 1, 2020

Summary Data lakes offer a great deal of flexibility and the potential for reduced cost for your analytics, but they also introduce a great deal of complexity. What used to be entirely managed by the database engine is now a composition of multiple systems that need to be properly configured to work in concert.

Data Lake 100

Data Lake Database Building Lambda Architecture 100

Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Data Engineering Podcast

MAY 11, 2020

You monitor your website to make sure that you’re the first to know when something goes wrong, but what about your data? Tidy Data is the DataOps monitoring platform that you’ve been missing. What motivates you to dedicate so much of your time and enery to Pulsar in particular, and the streaming data ecosystem in general?

Lambda Architecture 100

Lambda Architecture Cloud Kafka Hadoop 100

Data Engineering Weekly

JULY 9, 2023

The platform approach to enable the citizen machine learning engineers is a great perspective while building both the Data & ML platform. Architectural patterns like Lambda Architecture and Kappa Architecture emerged to bridge the gap between real-time and batch data processing.

Data Engineering 52

Data Engineering Data Engineer Engineering Lambda Architecture 52

Monte Carlo

MARCH 14, 2023

Data ingestion is the process of collecting data from various sources and moving it to your data warehouse or lake for processing and analysis. It is the first step in modern data management workflows. Data ingestion is the process of acquiring and importing data for use, either immediately or in the future.

Data Ingestion 52

Data Ingestion Data Warehouse Lambda Architecture Raw Data 52

Knowledge Hut

MAY 2, 2024

As per Apache, “ Apache Spark is a unified analytics engine for large-scale data processing ” Spark is a cluster computing framework, somewhat similar to MapReduce but has a lot more capabilities, features, speed and provides APIs for developers in many languages like Scala, Python, Java and R.

Scala 52

Scala Hospitality Healthcare Retail 52

ProjectPro

AUGUST 24, 2021

Data Warehousing: Data warehousing utilizes and builds a warehouse for storing data. A data engineer interacts with this warehouse almost on an everyday basis. Data Analytics: A data engineer works with different teams who will leverage that data for business solutions.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52