Data Engineering Podcast

AUGUST 21, 2022

Select Star’s data discovery platform solves that out of the box, with an automated catalog that includes lineage from where the data originated, all the way to which dashboards rely on it and who is viewing them every day. Go to dataengineeringpodcast.com/ascend and sign up for a free trial.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Knowledge Hut

MAY 2, 2024

As per Apache, “ Apache Spark is a unified analytics engine for large-scale data processing ” Spark is a cluster computing framework, somewhat similar to MapReduce but has a lot more capabilities, features, speed and provides APIs for developers in many languages like Scala, Python, Java and R.

Scala 52

Scala Hospitality Healthcare Retail 52

Data Engineering Podcast

NOVEMBER 20, 2021

Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. The batch model for processing is intuitive despite its latency problems.

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

Data Engineering Podcast

JUNE 16, 2019

This conversation was useful for getting a better idea of the challenges that exist in large scale data analytics, and the current state of the tradeoffs between data lakes and data warehouses in the cloud. What are some of the common antipatterns in data lake implementations and how does Delta Lake address them?

Data Lake 100

Data Lake Lambda Architecture Data Warehouse Hadoop 100

Cloudera

NOVEMBER 2, 2020

Users today are asking ever more from their data warehouse. As an example of this, in this post we look at Real Time Data Warehousing (RTDW), which is a category of use cases customers are building on Cloudera and which is becoming more and more common amongst our customers. What is Real Time Data Warehousing?

Data Warehouse 91

Data Warehouse Kafka Lambda Architecture Telecommunication 91

Data Engineering Podcast

JUNE 1, 2020

In order to bring the DBA into the new era of data management the team at Upsolver added a SQL interface to their data lake platform. How do those challenges influence the adoption or viability of a data lake? What are the advantages of a data lake over a data warehouse if everything is being managed via SQL anyway?

Data Lake 100

Data Lake Database Building Lambda Architecture 100

Knowledge Hut

OCTOBER 30, 2023

There are many uses and benefits for real-time traffic simulation and prediction projects using big data. This project is a Lambda Architecture program that tracks Chicago's streets' traffic conditions, including congestion and safety. If you are familiar with SQL, you should have no trouble completing this project.

Big Data 52

Big Data Coding Project Medical 52