article thumbnail

Data Engineering Annotated Monthly – June 2022

Big Data Tools

There are also multiple improvements for streaming support (for Kafka and Kinesis ), along with many other changes. It wouldn’t be quite right to call it “Kafka on steroids” because it includes lots of batteries. The State of Data Engineering 2022 – I like this kind of content.

article thumbnail

Data Engineering Annotated Monthly – June 2022

Big Data Tools

There are also multiple improvements for streaming support (for Kafka and Kinesis ), along with many other changes. It wouldn’t be quite right to call it “Kafka on steroids” because it includes lots of batteries. The State of Data Engineering 2022 – I like this kind of content.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data Engineering Annotated Monthly – May 2022

Big Data Tools

Kafka: Mark KRaft as Production Ready – One of the most interesting changes to Kafka from recent years is that it now works without ZooKeeper. This is possible thanks to implementations of KRaft, a Raft consensus protocol designed specifically for the needs of Kafka. Of course, the main topic is data streaming.

article thumbnail

Data Engineering Annotated Monthly – May 2022

Big Data Tools

Kafka: Mark KRaft as Production Ready – One of the most interesting changes to Kafka from recent years is that it now works without ZooKeeper. This is possible thanks to implementations of KRaft, a Raft consensus protocol designed specifically for the needs of Kafka. Of course, the main topic is data streaming.

article thumbnail

Data Engineering Annotated Monthly – April 2022

Big Data Tools

Kafka was the first, and soon enough, everybody was trying to grab their own share of the market. In the case of RocketMQ, their attempt is very interesting because, unlike Kafka and Pulsar, RocketMQ is closer to traditional MQs like ActiveMQ (which isn’t really surprising, seeing how it’s based on ActiveMQ).

article thumbnail

Data Engineering Annotated Monthly – April 2022

Big Data Tools

Kafka was the first, and soon enough, everybody was trying to grab their own share of the market. In the case of RocketMQ, their attempt is very interesting because, unlike Kafka and Pulsar, RocketMQ is closer to traditional MQs like ActiveMQ (which isn’t really surprising, seeing how it’s based on ActiveMQ).

article thumbnail

Data Engineering Annotated Monthly – September 2022

Big Data Tools

One of the use cases from the product page that stood out to me in particular was the effort to mirror multiple Kafka clusters in one Brooklin cluster! Ambry v0.3.870 – It turns out that last month was rich in releases from LinkedIn, all of them related in one way or another to data engineering. This is no doubt very interesting.