Cloudera

MARCH 15, 2023

Writing to a database and sending messages to a message bus is not atomic, which means that if one of these operations fails, the state of the application can become inconsistent. When the service receives a request, it not only persists the new entity, but also a record representing the message that will be published to the event bus.

PostgreSQL 91

PostgreSQL Kafka Database Data 91

AltexSoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka? How Apache Kafka streams relate to Franz Kafka’s books. What Kafka is used for.

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

Confluent

JULY 16, 2019

How cool would it be to build your own burglar alarm system that can alert you before the actual event takes place simply by using a few network-connected cameras and analyzing the camera images with Apache Kafka ® , Kafka Streams, and TensorFlow? I will show how to implement this use case in this blog post.

Machine Learning 100

Machine Learning Kafka Java Datasets 100

AltexSoft

NOVEMBER 7, 2022

An example of the workflow in the form of a directed acyclic graph or DAG. DevOps tasks — for example, creating scheduled backups and restoring data from them. However, the platform is compatible with solutions supporting near real-time and real-time analytics — such as Apache Kafka or Apache Spark. Source: Apache Airflow.

PostgreSQL 52

PostgreSQL Metadata Python MySQL 52

ProjectPro

JANUARY 25, 2022

Here’s What You Need to Know About PySpark This blog will take you through the basics of PySpark, the PySpark architecture, and a few popular PySpark libraries , among other things. PySpark is used to process real-time data with Kafka and Streaming, and this exhibits low latency. This is where Apache Spark PySpark comes in.

Big Data 52

Big Data Data Process Process Kafka 52

Confluent

APRIL 30, 2019

With the release of Apache Kafka ® 2.1.0, Kafka Streams introduced the processor topology optimization framework at the Kafka Streams DSL layer. In what follows, we provide some context around how a processor topology was generated inside Kafka Streams before 2.1, Kafka Streams topology generation 101.

Kafka 90

Kafka Coding Process Bytes 90

ProjectPro

NOVEMBER 22, 2021

One of the examples of giants embracing PySpark is Trivago. The main goal of this is to connect the Python API to the Spark core. It's useful when you need to do low-level transformations, operations, and control on a dataset. Trivago has been employing PySpark to fulfill its team's tech demands.

Hadoop 52

Hadoop Python Datasets Metadata 52