In this tutorial, we will show you how to install Apache Kafka on Debian 10. For those of you who didn’t know, Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘
sudo‘ to the commands to get root privileges. I will show you through the step-by-step installation of the Apache Kafka on a Debian 10 (Buster).
- A server running one of the following operating systems: Debian 10 (Buster).
- It’s recommended that you use a fresh OS install to prevent any potential issues.
non-root sudo useror access to the
root user. We recommend acting as a
non-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.
Install Apache Kafka on Debian 10 Buster
Step 1. Before running the tutorial below, it’s important to make sure your system is up to date by running the following
apt commands in the terminal:
sudo apt update
Step 2. Installing Java.
Now install Java using the command below:
sudo apt install default-jdk
Verify the Java version by running the following command:
Step 3. Installing Apache Kafka on Debian 10.
Now we run the following command to download Apache Kafka packages from the official website:
Then, extract the downloaded archive file:
tar xzf kafka_2.12-2.8.0.tgz mv kafka_2.12-2.8.0 /usr/local/kafka
Step 4. Create Systemd Unit Files.
Now we create
systemd unit files for the Zookeeper and Kafka service. This will help to manage Kafka services to start or stop using the
Add the following file:
[Unit] Description=Apache Zookeeper server Documentation=http://zookeeper.apache.org Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh Restart=on-abnormal [Install] WantedBy=multi-user.target
Next, to create a Kafka
systemd unit file using the following command:
Add the below content:
[Unit] Description=Apache Kafka Server Documentation=http://kafka.apache.org/documentation.html Requires=zookeeper.service [Service] Type=simple Environment="JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64" ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh [Install] WantedBy=multi-user.target
Save the files and reload the
systemd daemon to apply new changes:
sudo systemctl daemon-reload sudo systemctl start zookeeper sudo systemctl start kafka
Step 5. Create a Topic in Kafka.
First, create a topic named “MakeTopic” with a single partition with a single replica:
cd /usr/local/kafka bin/kafka-maketopics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic MakeTopic
After that, you can see the created topics on Kafka by the running below command:
$ bin/kafka-maketopics.sh --list --zookeeper localhost:2181 MakeTopic
Step 6. Send Messages to Kafka.
Now we run the producer and then type a few messages into the console to send to the server:
$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic MakeTopic >Welcome to kafka >This is my first topic >
Step 7. Using Kafka Consumer.
Apache Kafka also has a command-line consumer to read data from the Kafka cluster and display messages to standard output.
$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic MakeTopic --from-beginning Welcome to kafka This is my first topic
Congratulations! You have successfully installed Apache Kafka. Thanks for using this tutorial for installing the latest version of the Apache Kafka on the Debian system. For additional help or useful information, we recommend you check the official Apache Kafka website.