François Nguyen

MARCH 22, 2021

TL;DR After setting up and organizing the teams, we are describing 4 topics to make data mesh a reality. With this 3rd platform generation, you have more real time data analytics and a cost reduction because it is easier to manage this infrastructure in the cloud thanks to managed services. What you have to code is this workflow !

Architecture 130

Architecture Technology Google Cloud Metadata 130

Data Engineering Podcast

AUGUST 28, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Engineering 100

Data Engineering Data Engineer MongoDB Metadata 100

Workfall

JUNE 12, 2023

Moreover, it facilitates the implementation of microservices architectures and event-driven systems, automating reactions to data changes without manual intervention. It captures incremental changes from transactional databases or other sources, efficiently loading them into data warehouses or data lakes.

Telecommunication 80

Telecommunication Metadata Healthcare Finance 80

Hepta Analytics

FEBRUARY 14, 2022

This week, we got to think about our data ingestion design. We looked at the following: How do we ingest – ETL vs ELT Where do we store the data – Data lake vs data warehouse Which tool to we use to ingest – cronjob vs workflow engine NOTE : This weeks task requires good internet speed and good compute.

Data Ingestion 52

Data Ingestion Data Engineering Data Engineer Engineering 52

Cloudera

AUGUST 21, 2020

Data-in-motion is predominantly about streaming data so enterprises typically have two different ways or binary ways of looking at data. However, in the context of time and geography, these two events point to a pattern of fraud. The card is being used for its purpose and the amounts are relatively insignificant.

Banking 60

Banking Kafka Cloud Storage Government 60

Data Engineering Podcast

NOVEMBER 26, 2023

Developing event-driven pipelines is going to be a lot easier - Meet Functions! Data lakes are notoriously complex. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Ascend.io

DECEMBER 11, 2023

Data orchestration is the process of efficiently coordinating the movement and processing of data across multiple, disparate systems and services within a company. This contrasts with data pipeline orchestration , which adopts a narrower focus, centering on the construction, operation, and management of data pipelines.

Data Workflow 52

Data Workflow Data Pipeline Data Lake Data 52