Remove event-driven-architecture-pub-sub
article thumbnail

Google Cloud Pub/Sub: Messaging on The Cloud

ProjectPro

With over 10 million active subscriptions, 50 million active topics, and a trillion messages processed per day, Google Cloud Pub/Sub makes it easy to build and manage complex event-driven systems. Google Pub/Sub provides global distribution of messages making it possible to send and receive messages from across the globe.

article thumbnail

Top 10 AWS Applications and Their Use Cases [2024 Updated]

Knowledge Hut

I will explore the top 10 AWS applications and their use cases in this blog. AWS Lambda AWS Lambda is a serverless computing service that enables developers to run code in response to events without needing to work with servers. ECS is mostly used in microservices architecture , batch processing, and CI/CD pipelines.

AWS 52
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

How to Use Kafka for Event Streaming in a Microservices Architecture?

Workfall

In this blog, we’ll explore how to harness the power of Kafka to streamline event streaming within a microservices architecture and unlock its full potential for building scalable and responsive systems. In the event of server failure, Kafka can autonomously restart itself, ensuring uninterrupted data flow.

Kafka 75
article thumbnail

API-First Approach to Kafka Topic Creation

DoorDash Engineering

Kafka is the pub-sub layer of the Iguazu pipeline, which provides real-time event delivery at DoorDash. Understanding the legacy architecture As shown in Figure 1 below, DoorDash’s legacy approach to Topic creation involved several steps within a Cadence workflow. Chronosphere : Cloud-native observability platform.

Kafka 91
article thumbnail

The Good and the Bad of Apache Kafka Streaming Platform

AltexSoft

But to understand why Kafka is omnipresent we have to look at how it works — in other words, to get familiar with its concepts and architecture. Kafka architecture. Let’s start with an asynchronous communication pattern Kafka revolves around — publish/subscribe or simply Pub/Sub. Kafka cluster architecture.

Kafka 93
article thumbnail

Data Contracts and 4 Other Ways to Overcome Schema Changes

Monte Carlo

As GoCardless team lead and senior data engineer Andrew Jones recently wrote in his blog post about data contracts , “What I found in my swim upstream were well meaning engineers modifying services unaware that something as simple as dropping a field could have major implications on dashboards (or other consumers) downstream.”

article thumbnail

20+ Data Engineering Projects for Beginners with Source Code

ProjectPro

And, out of these professions, this blog will discuss the data engineering job role. Top 20+ Data Engineering Projects Ideas for Beginners with Source Code [2023] We recommend over 20 top data engineering project ideas with an easily understandable architectural workflow covering most industry-required data engineer skills.