How To Install Apache Kafka on Ubuntu 22.04 LTS

In this tutorial, we will show you how to install Apache Kafka on Ubuntu 22.04 LTS. Apache Kafka, an open-source distributed event streaming platform, has emerged as a powerful solution for building real-time data pipelines and enabling high-performance data integration. Whether you’re working on big data analytics, microservices architecture, or IoT applications, Kafka provides a robust and scalable messaging system that ensures reliable data delivery. As the adoption of Kafka continues to grow, learning how to install and configure it on a popular Linux distribution like Ubuntu 22.04 is an essential skill for developers, DevOps engineers, and data professionals.

This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘sudo‘ to the commands to get root privileges. I will show you the step-by-step installation of Apache Kafka on Ubuntu 22.04. You can follow the same instructions for Ubuntu 22.04 and any other Debian-based distribution like Linux Mint, Elementary OS, Pop!_OS, and more as well.

Prerequisites

• A server running one of the following operating systems: Ubuntu 22.04, 20.04, and any other Debian-based distribution like Linux Mint.
• It’s recommended that you use a fresh OS install to prevent any potential issues.
• SSH access to the server (or just open Terminal if you’re on a desktop).
• An active internet connection. You’ll need an internet connection to download the necessary packages and dependencies for Apache Kafka.
• A non-root sudo user or access to the root user. We recommend acting as a non-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.

Install Apache Kafka on Ubuntu 22.04 LTS Jammy Jellyfish

Step 1. First, make sure that all your system packages are up-to-date by running the following apt commands in the terminal.

sudo apt update
sudo apt upgrade

Step 2. Java Development Kit (JDK) Installation.

Kafka relies on Java, so ensure you have the JDK installed. Let’s verify its presence and set the JAVA_HOME environment variable.

Confirm if Java is installed by running:

java -version

If not, install it with:

sudo apt install openjdk-11-jdk

To set the JAVA_HOME environment variable, open the .bashrc file:

nano ~/.bashrc

Add the following line at the end of the file (replace <your_java_home_path> with your Java installation path):

export JAVA_HOME=<your_java_home_path>

Save the file and run:

source ~/.bashrc

Step 3. Installing Apache Kafka on Ubuntu 22.04.

Navigate to the official Apache Kafka downloads page, select the version you want, and copy its URL. In your terminal, use wget to download Kafka:

wget https://downloads.apache.org/kafka/3.7.0/kafka-3.7.0-src.tgz

Extracting the downloaded archive:

tar -xzf kafka-3.7.0-src.tgz

Step 4. Configuring Apache Kafka.

First, creating a Kafka configuration directory:

sudo mkdir /etc/kafka

Edit the Kafka server.properties file:

sudo nano /etc/kafka/server.properties

Here are some essential properties to configure:

• Broker ID: Ensure each broker has a unique ID.
• Port Configurations: Default ports are 9092 for plain text and 9093 for SSL.
• Log Directories: Set the paths for Kafka logs.
• Zookeeper Connection: Specify the Zookeeper server address.

Step 5. Starting the Kafka Server.

To start Kafka in the background, run:

./kafka_3.7.0/bin/kafka-server-start.sh -daemon /etc/kafka/server.properties

You can verify that Kafka is running:

./kafka_3.7.0/bin/kafka-server-stop.sh

Step 6. Creating Kafka Topics.

Topics are essential for data organization in Kafka. To create a topic, use the kafka-topics.sh script:

./kafka_3.7.0/bin/kafka-topics.sh --create --topic test-topic --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092

Step 7. Testing Apache Kafka.

Now that your Kafka cluster is up and running, let’s test it by setting up a producer and a consumer.

Creating a Kafka Producer:

./kafka_3.7.0/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092

Creating a Kafka Consumer:

./kafka_3.7.0/bin/kafka-console-consumer.sh --topic test-topic --bootstrap-server localhost:9092 --from-beginning

Step 8. Managing Apache Kafka.

Starting and Stopping Kafka:

./kafka_3.7.0/bin/kafka-server-stop.sh

To start Kafka again:

./kafka_3.7.0/bin/kafka-server-start.sh /etc/kafka/server.properties

Step 9. Troubleshooting Common Issues.

Kafka Server Errors

• Check logs in the Kafka log directories.
• Ensure proper configuration of Kafka properties.

Kafka Producer and Consumer Issues

• Verify that producer and consumer configurations match.
• Check for network or firewall issues.

Resource Constraints and Performance Tuning

• Monitor system resources using tools like top and htop.
• Adjust Kafka configurations for optimal performance.

Congratulations! You have successfully installed Kafka. Thanks for using this tutorial for installing Apache Kafka on the Ubuntu system. For additional help or useful information, we recommend you check the official Apache Kafka website.