Remove 2021 Remove Big Data Tools Remove Building Remove Kafka
article thumbnail

Data Engineering Annotated Monthly – September 2021

Big Data Tools

Zingg is a tool that integrates with Spark and tries to answer this question automatically, without the quadratic complexity of the task! Kafka 3.0.0 – The Apache Software Foundation needed less than one month to go from Kafka version 3.0.0-rc0 Tools askgit – SQL is a native language for many data engineers.

article thumbnail

Data Engineering Annotated Monthly – September 2021

Big Data Tools

Zingg is a tool that integrates with Spark and tries to answer this question automatically, without the quadratic complexity of the task! Kafka 3.0.0 – The Apache Software Foundation needed less than one month to go from Kafka version 3.0.0-rc0 Tools askgit – SQL is a native language for many data engineers.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data Engineering Annotated Monthly – July 2021

Big Data Tools

Rack-aware Kafka streams – Kafka has already been rack-aware for a while, which gives its users more confidence. When data is replicated between different racks housed in different locations, if anything bad happens to one rack, it won’t happen to another. Most of the topics, from data quality to DWH architecture, are hot!

article thumbnail

Data Engineering Annotated Monthly – July 2021

Big Data Tools

Rack-aware Kafka streams – Kafka has already been rack-aware for a while, which gives its users more confidence. When data is replicated between different racks housed in different locations, if anything bad happens to one rack, it won’t happen to another. Most of the topics, from data quality to DWH architecture, are hot!

article thumbnail

Data Engineering Annotated Monthly – August 2021

Big Data Tools

It hasn’t had its first release yet, but the promise is that it will un-bias your data for you! rc0 – If you like to try new releases of popular products, the time has come to test Kafka 3 and report any issues you find on your staging environment! Change Data Capture at DeviantArt – I think we all know what Debezium is.

article thumbnail

Data Engineering Annotated Monthly – November 2021

Big Data Tools

It’s developed by LinkedIn, which means it has very tight integrations with other LinkedIn tools, like Apache Kafka! This release brings 2 big features: Segment Merge and Rollup, both of which can be used for better (i.e. And, unlike Kafka, it doesn’t need ZooKeeper and it supports message scheduling! Apache Pinot 0.9.0

article thumbnail

Data Engineering Annotated Monthly – November 2021

Big Data Tools

It’s developed by LinkedIn, which means it has very tight integrations with other LinkedIn tools, like Apache Kafka! This release brings 2 big features: Segment Merge and Rollup, both of which can be used for better (i.e. And, unlike Kafka, it doesn’t need ZooKeeper and it supports message scheduling! Apache Pinot 0.9.0