In this tutorial, we will show you how to install Apache Kafka on Ubuntu 20.04 LTS. For those of you who didn’t know, Kafka is the most popular open-source software that provides a framework for storing, reading, and analyzing streaming data. Kafka was originally developed by LinkedIn and to be open-sourced in early 2011 under Apache Software Foundation.
This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘
sudo‘ to the commands to get root privileges. I will show you through the step-by-step installation of Apache Kafka on Ubuntu 20.04 (Focal Fossa). You can follow the same instructions for Ubuntu 18.04, 16.04, and any other Debian-based distribution like Linux Mint.
- A server running one of the following operating systems: Ubuntu 20.04, 18.04, 16.04, and any other Debian-based distribution like Linux Mint.
- It’s recommended that you use a fresh OS install to prevent any potential issues.
non-root sudo useror access to the
root user. We recommend acting as a
non-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.
Install Apache Kafka on Ubuntu 20.04 LTS Focal Fossa
Step 1. First, make sure that all your system packages are up-to-date by running the following
apt commands in the terminal.
sudo apt update sudo apt upgrade
Step 2. Installing Java.
Apache Kafka requires Java to be installed on your server. Run the following command to install OpenJDK on the Ubuntu system:
sudo apt install openjdk-11-jdk
Verify the Java version by running the following command:
$ java -version java version "1.8.0_251" Java(TM) SE Runtime Environment (build 1.8.0_251-b07) Java HotSpot(TM) 64-Bit Server VM (build 25.251-b07, mixed mode)
Step 3. Installing Apache Kafka on Ubuntu 20.04.
Now we download the latest version of Kafka on the official web page:
Then, un-tar the archive file and move to another location:
sudo tar xzf kafka_2.13-2.6.0.tgz sudo mv kafka_2.13-2.6.0 /opt/kafka
Step 4. Create Zookeeper and Kafka Systemd Unit Files.
systemd unit file for zookeeper service:
sudo nano /etc/systemd/system/zookeeper.service
Add the following lines:
[Unit] Description=Apache Zookeeper service Documentation=http://zookeeper.apache.org Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh Restart=on-abnormal [Install] WantedBy=multi-user.target
Next, create the
systemd unit file for Kafka service:
sudo nano /etc/systemd/system/kafka.service
Add the following lines:
[Unit] Description=Apache Kafka Service Documentation=http://kafka.apache.org/documentation.html Requires=zookeeper.service [Service] Type=simple Environment="JAVA_HOME=/opt/jdk/jdk1.8.0_251" ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties ExecStop=/opt/kafka/bin/kafka-server-stop.sh [Install] WantedBy=multi-user.target
Reload the daemon to take effect:
sudo systemctl daemon-reload sudo systemctl enable --now zookeeper sudo systemctl enable --now kafka
Step 5. Creating Topic in Kafka.
Now we will create a topic named “idroot” with a single replication-factor and partition:
cd /opt/kafka sudo bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-f
To send some messages for the created Topic:
sudo bin/kafka-console-producer.sh --broker-list localhost:9092 --topic idoot
It’s prompt for messages to type:
> Hi > How are you?
Using the below command we can see the list of messages:
sudo bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic idroot --from-beginning
Step 6. To Connect Kafka from remote machines.
To connect, create Topic, and send messages from a remote server. Please follow the below steps:
Now look for
server.properties and make some configuration changes:
sudo nano server.properties
listeners=PLAINTEXT://:9092 advertised.listeners=PLAINTEXT://<HOST IP>:9092
Congratulations! You have successfully installed Apache Kafka. Thanks for using this tutorial for installing Apache Kafka on Ubuntu 20.04 LTS Focal Fossa system. For additional help or useful information, we recommend you to check the official Apache Kafka website.