Installing Apache Kafka on Windows

Learn to install Apache Kafka on Windows 10 and execute ‘start server‘ and ‘stop server‘ scripts related to Kafka and Zookeeper. We will also verify the Kafka installation by creating a topic, producing a few messages to it and then using a Kafka consumer to read the messages.

1. Prerequisites

• Minimum Java 8 is required to run the latest downloads from the Kafka site. Install Java if you have not already.
• Zookeeper (to store metadata about the Kafka cluster) is also mandatory. Kafka comes with an inbuilt Zookeeper to start with. But it is recommended to install Zookeeper separately in the production environment. Download it from its official site.
• Kafka can be run on any operating system. Linux is the recommended OS. With Windows, Kafka has some known bugs.

This tutorial is for beginners and does not use separate zookeeper instance – to keep things simple and focused towards Kafka only.

2. Download and Install Kafka

• Download Kafka from the official site. I download the latest version which is 2.5.0, and the file name is “kafka_2.12-2.5.0.tgz“.
• Copy the downloaded file to some folder and extract it using tar command.
• Copy the extracted folder to the desired location. I have put it on location “E:\devsetup\bigdata\kafka2.5“.

tar -xzf kafka_2.12-2.5.0.tgz

Installation is pretty much complete, now.

3. Startup and Shutdown

To start Kafka, we first start Zookeeper and then Kafka. I am writing small batch files which navigate to the Kafka installation directory first and then execute the command in the new command prompt window.

3.1. Start Zookeeper

To start zookeeper, we need to run zookeeper-server-start.bat script and pass the zookeeper configuration file path.

cd E:\devsetup\bigdata\kafka2.5
start cmd /k bin\windows\zookeeper-server-start.bat config\zookeeper.properties

3.2. Start Kafka

To start Kafka, we need to run kafka-server-start.bat script and pass broker configuration file path.

cd E:\devsetup\bigdata\kafka2.5
start cmd /k bin\windows\kafka-server-start.bat config\server.properties

3.3. Shutdown Kafka

To stop Kafka, we need to run kafka-server-stop.bat script.

cd E:\devsetup\bigdata\kafka2.5
start cmd /k bin\windows\kafka-server-stop.bat

3.4. Shutdown Zookeeper

To stop Zookeeper, we need to run zookeeper-server-stop.bat script.

cd E:\devsetup\bigdata\kafka2.5
start cmd /k bin\windows\zookeeper-server-stop.bat

Do not stop zookeeper and kafka using CTRL+C commands. Always use above .bat files or commands. Otherwise the data corruption may occur.

4. Verify Kafka Installation

First, start Zookeeper and Kafka using the above scripts.

Open a new command prompt, and create new Kafka topic.

bin\windows\kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

Created topic test.

Now list all the topics to verify the created topic is present in this list. At this step, we have only one topic.

bin\windows\kafka-topics.bat --list --bootstrap-server localhost:9092

test

Now list all the topics to verify the created topic is present in this list. At this step, we have only one topic.

bin\windows\kafka-topics.bat --list --bootstrap-server localhost:9092

test

Produce some messages and submit to test topic. I added two messages i.e. “Hello” and “Kafka”.

bin\windows\kafka-console-producer.bat --bootstrap-server localhost:9092 --topic test

Consume the messages and submit to test topic.

bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning

Hello
Kafka

5. Conclusion

In this tutorial, we learned to install Kafka along with Zookeeper. We learned to start and stop both servers. Additionally, we verified the installation by creating a topic, posting some messages and then consuming using the console consumer script.

The important thing to note is that we should never stop the servers by killing processes or CTRL+C commands. Always use scripts to stop the servers.

Happy Learning !!

Sourcecode Download