Event streaming: Kafka 3.9 says goodbye to ZooKeeper

The Kafka project managed by the Apache Software Foundation (ASF) has reached version 3.9. According to the release notes, this is the last major update of the 3.x series of the event and data stream processing platform. This version marks the end of support for the outdated ZooKeeper mode, as Kafka will be operated without ZooKeeper from version 4.0.

KRaft-Quoren replaces ZooKeeper

A key innovation in Kafka 3.9 is the introduction of dynamic KRaft quorums, which make it possible to change controller nodes without downtime. This should make it easier for users to manage and adapt the Kafka infrastructure. A KRaft quorum in Apache Kafka is a system that uses the Raft algorithm to manage metadata, eliminating the dependency on ZooKeeper. ZooKeeper is a service used in distributed systems to centrally manage configurations, name registries and synchronization services.

Read also

Java 23 extends import and patterns, but dispenses with string templates

According to the release notes, the development team has completed the adjustments to the process for the migration from ZooKeeper to KRaft. This migration has been continuously revised since Kafka 3.4. Administrators who continue to use ZooKeeper are encouraged to migrate to Kafka 3.9 before upgrading to Kafka 4.0.

Tiered Storage ready for production

In addition, the Tiered Storage feature, which has been in development since Kafka 3.6, is now ready for production use and brings new tools for managing storage loads. Improvements in Kafka Streams and Kafka Connect include revised logging, a new processing exception handler designed to allow users to handle specific exceptions that occur during message processing. This handler is initialized once per StreamTask. The update also includes further configuration options.

More information can be found in the release notes and the article on the Confluent blog.

(mdo)