Recently I faced an issue with my single node Kafka broker after a system reboot. The Kafka was not getting started and it was showing an error “Cluster ID doesn’t match stored clusterId in meta.properties”

This error is self explanatory and the we can solve this by doing a quick fix. The following steps fixed my Kafka’s problem.

Step 1: Get the new Cluster ID from the error logs.

You can check the error logs and find the new cluster id.

Sample cluster id format is given below

p2Ke6DSDzfdcxcfarkcxJscoQ

Step 2: Find the meta.properties file in your Kafka server

There is a file with the name meta.properties. This file is located within the kafka-logs directory. The location of kafka-logs directory can be located by checking the server.properties file in the config directory.

Location of config file –> $KAFKA_HOME/config/server.properties

In the server.properties, check for value of the parameter log.dirs.

cat  $KAFKA_HOME/config/server.properties  | grep log.dir

The meta.properties will be located in the log.dir path

Step 3: Update the Cluster ID in the meta.properties and restart the Kafka

Open the meta.properties, the contents will be similar to the one below. Update the cluster.id value with the new cluster id present in the error log.

#Wed May 26 11:21:15 EET 2021
cluster.id=P2Ka7bKGmJwBduCchqrhsP
version=0
broker.id=0

Restart the Kafka Broker. Problem solved.

Note: The above solution is just a temporary work around and not a permanent solution. The cluster id mismatch happens because of the difference in the cluster ids registered in Zookeeper and Kafka. Kafka uses zookeeper for broker to broker communication. By default zookeeper’s data directory is configured to /tmp location. So if the system reboots, the tmp directory gets cleared and this results in a complete new registration of the Kafka cluster which results in a new cluster id. Similarly Kafka also has a data directory. The permanent solution is to maintain a persistent data directory for kafka and zookeeper. This will ensure the same state even after a server reboot.

I hope this tip is helpful. Feel free to comment if you face any issues.