Blog - Data Engineering Digest

ksql-streaming-sql-for-apache-kafka

Blog

Analysis of Confluent Buying Immerok

Jesse Anderson

JANUARY 9, 2023

I’ve always been vocal about ksqlDB’s and Kafka Stream’s limitations. The Future of ksqlDB and Kafka Streams With this announcement, the future of primarily ksqlDB and, to a lesser extent, Kafka Streams comes into view. Since Kafka Streams is part of the Apache project, I don’t see it going away as quickly.

Kafka

Kafka Coding Technology SQL

Machine Learning with Python, Jupyter, KSQL and TensorFlow

Confluent

FEBRUARY 6, 2019

The blog posts How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka and Using Apache Kafka to Drive Cutting-Edge Machine Learning describe the benefits of leveraging the Apache Kafka ® ecosystem as a central, scalable and mission-critical nervous system.

Machine Learning

Machine Learning Python Kafka Java

Join 16,000+

Insiders

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Webinars

How to Optimize the Developer Experience for Monumental Impact

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Understanding User Needs and Satisfying Them

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Leading the Development of Profitable and Sustainable Products

MORE WEBINARS

Trending Sources

The Importance of Distributed Tracing for Apache-Kafka-Based Applications

Confluent

MARCH 26, 2019

Apache-Kafka ® -based applications stand out for their ability to decouple producers and consumers using an event log as an intermediate layer. This article describes how to instrument Kafka-based applications with distributed tracing capabilities in order to make dataflows between event-based components more visible.

Kafka

Kafka Transportation Metadata Consulting

Webinars

How to Optimize the Developer Experience for Monumental Impact

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Understanding User Needs and Satisfying Them

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Leading the Development of Profitable and Sustainable Products

MORE WEBINARS

KSQL Training for Hands-On Learning

Confluent

JULY 11, 2019

I’ve been using KSQL from Confluent since its first developer preview in 2017. Reading, writing, and transforming data in Apache Kafka ® using KSQL is an effective way to rapidly deliver event streaming applications for clients (e.g., streaming insurance events ). The course consists of 33 lectures in total.

Kafka

Kafka Insurance SQL Architecture

Reshaping Entire Industries with IoT and Confluent Cloud

Confluent

APRIL 18, 2019

And Confluent Cloud and Apache Kafka ® are at the center of this transformation. Kafka is business critical for us. So working with Confluent Cloud and the Kafka experts was an obvious choice,” says Stefan Frehse, Software Architect, Digitalization, for food processing machinery company BAADER.

Food

Food Cloud Retail Kafka

How to Use KSQL Stream Processing and Real-Time Databases to Analyze Streaming Data in Kafka

Rockset

MARCH 19, 2020

Intro In recent years, Kafka has become synonymous with “streaming,” and with features like Kafka Streams, KSQL, joins, and integrations into sinks like Elasticsearch and Druid, there are more ways than ever to build a real-time analytics application around streaming data in Kafka.

Kafka

Kafka Database Process SQL

Using Graph Processing for Kafka Stream Visualizations

Confluent

AUGUST 29, 2019

We know that Apache Kafka ® is great when you’re dealing with streams, allowing you to conveniently look at streams as tables. Stream processing engines like KSQL furthermore give you the ability to manipulate all of this fluently. The approach we’ll use works with any Kafka run though.

Kafka

Kafka Process Algorithm Cloud

Real-Time Analytics and Monitoring Dashboards with Apache Kafka and Rockset

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. However, Apache Kafka is more than just messaging. Rockset supports JDBC and integrates with other SQL dashboards like Tableau, Grafana, and Apache Superset.

Kafka

Kafka BI SQL Datasets

Deploying Kafka Streams and KSQL with Gradle – Part 3: KSQL User-Defined Functions and Kafka Streams

Confluent

JULY 10, 2019

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. KSQL user-defined functions. While the CASE syntax now available since KSQL 5.2.2 gradlew composeUp.

Kafka

Kafka Java Bytes SQL

Deploying Kafka Streams and KSQL with Gradle – Part 1: Overview and Motivation

Confluent

MAY 15, 2019

But the main gap is the way that traditional ETL tools reason about the events that needed processing—that they exist in batches, not in streams. Events move in streams: continuous, closed-loop processes without defined batches. The item picking process. Inventory auditing. Warehouse staff management. Other ancillary functions.

Kafka

Kafka ETL Tools Cloud Data Integration

Deploying Kafka Streams and KSQL with Gradle – Part 2: Managing KSQL Implementations

Confluent

MAY 29, 2019

In part 1 , we discussed an event streaming architecture that we implemented for a customer using Apache Kafka ® , KSQL from Confluent, and Kafka Streams. In part 3, we’ll explore using Gradle to build and deploy KSQL user-defined functions (UDFs) and Kafka Streams microservices.

Kafka

Kafka Management Bytes SQL

Journey to Event Driven – Part 2: Programming Models for the Event-Driven Architecture

Confluent

FEBRUARY 13, 2019

Once settled on the event streaming approach, I’ll provide a high-level dataflow of how we design systems for payment processing at scale using this approach. The paradigm shift to the event streaming architecture often leads to a lightbulb moment of inspiration and clarity. Event-driven, streaming architecture.

Architecture

Architecture Programming Kafka Database-centric

KSQL in Football: FIFA Women’s World Cup Data Analysis

Confluent

JULY 3, 2019

The idea in this blog post is to mix information coming from two distinct channels: the RSS feeds of sport-related newspapers and Twitter feeds of the FIFA Women’s World Cup. Ingesting Twitter data is very easy with Kafka Connect , a framework for connecting Kafka with external systems. Data sources. Ingesting Twitter data.

Data Analysis

Data Analysis Kafka Datasets Java

The Rise of Managed Services for Apache Kafka

Confluent

SEPTEMBER 20, 2019

As a distributed system for collecting, storing, and processing data at scale, Apache Kafka ® comes with its own deployment complexities. To simplify all of this, different providers have emerged to offer Apache Kafka as a managed service. How do you spot a true fully managed service for Apache Kafka?

Kafka

Kafka Management Cloud AWS

Analysis of Confluent Buying Immerok

Machine Learning with Python, Jupyter, KSQL and TensorFlow

Webinars

Trending Sources

The Importance of Distributed Tracing for Apache-Kafka-Based Applications

Webinars

KSQL Training for Hands-On Learning

Reshaping Entire Industries with IoT and Confluent Cloud

How to Use KSQL Stream Processing and Real-Time Databases to Analyze Streaming Data in Kafka

Using Graph Processing for Kafka Stream Visualizations

Real-Time Analytics and Monitoring Dashboards with Apache Kafka and Rockset

Deploying Kafka Streams and KSQL with Gradle – Part 3: KSQL User-Defined Functions and Kafka Streams

Deploying Kafka Streams and KSQL with Gradle – Part 1: Overview and Motivation

Deploying Kafka Streams and KSQL with Gradle – Part 2: Managing KSQL Implementations

Journey to Event Driven – Part 2: Programming Models for the Event-Driven Architecture

KSQL in Football: FIFA Women’s World Cup Data Analysis

The Rise of Managed Services for Apache Kafka

Stay Connected