How To Install Apache Kafka on Debian 12

In this tutorial, we will show you how to install Apache Kafka on Debian 12. Apache Kafka, an open-source distributed event streaming platform, has become a cornerstone in the world of data processing. Designed by the Apache Software Foundation, Kafka’s robust architecture allows it to handle large-scale, real-time data streams efficiently and reliably. Its publish-subscribe model has made it a popular choice for businesses and organizations looking to process and analyze real-time data.

This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘sudo‘ to the commands to get root privileges. I will show you the step-by-step installation of the Apache Kafka on a Debian 12 (Bookworm).

Prerequisites

• A server running one of the following operating systems: Debian 12 (Bookworm).
• It’s recommended that you use a fresh OS install to prevent any potential issues.
• SSH access to the server (or just open Terminal if you’re on a desktop).
• You will need an active internet connection to download the Apache Kafka package.
• A non-root sudo user or access to the root user. We recommend acting as a non-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.

Install Apache Kafka on Debian 12 Bookworm

Step 1. Before we dive into the installation process, it’s crucial to ensure that your Debian 12 system is up-to-date. This step helps avoid potential conflicts between system packages and the GlassFish server. To update your system, open the terminal and execute the following commands:

sudo apt update
sudo apt upgrade

These commands will fetch the latest package lists from the repositories and upgrade any outdated packages on your system.

Step 2. Installing Java (OpenJDK).

Before you can install Apache Kafka, you need to ensure that Java is installed on your Debian 12 system. Kafka requires Java to run, so this is a crucial step. You can verify the presence of Java by running the following command in your terminal:

java --version

If Java is not installed, you will need to install it before proceeding. You can do this by running the following command:

sudo apt install default-jdk

Step 3. Installing Apache Kafka on Debian 12.

Once Java is installed, you can proceed to download the latest Kafka release archive. Navigate to the Kafka download page and grab the latest binary release archive. You can download it directly to your system using the wget command:

https://downloads.apache.org/kafka/3.6.1/kafka_2.13-3.6.1.tgz

With the Kafka binary release archive downloaded, you’re ready to install Apache Kafka. First, you need to extract the downloaded archive. We will install Kafka to the /opt/kafka directory. You can extract the downloaded archive using the tar command:

tar -xzf kafka_2.13-3.6.1.tgz -C /opt/kafka

Find the line that reads #delete.topic.enable=true, remove the # to uncomment the line, and save the file.

With the server configured, you can start Kafka. Kafka 2.8.0 introduced a new mode called KRaft (Kafka Raft metadata mode) that simplifies the operation of Kafka by removing the need for Apache ZooKeeper. You can start Kafka in KRaft mode with the following command:

./bin/kafka-server-start.sh ./config/kraft/server.properties

With Kafka installed and running, it’s time to test your setup. You can do this by creating a new topic and then producing and consuming messages from that topic. First, create a new topic named “test” with the following command:

./bin/kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092

Next, produce a message to the “test” topic:

echo "Hello, World" | ./bin/kafka-console-producer.sh --topic test --bootstrap-server localhost:9092

Finally, consume the message from the “test” topic:

./bin/kafka-console-consumer.sh --topic test --from-beginning --bootstrap-server localhost:9092

If everything is working correctly, you should see the message “Hello, World” in your terminal.

Step 4. Troubleshooting Common Issues.

Despite your best efforts, you may encounter issues when working with Apache Kafka. Here are a few common issues and their solutions:

• Out of memory errors: If Kafka is running out of memory, consider increasing the heap size. You can do this by editing the KAFKA_HEAP_OPTS environment variable in the Kafka startup script.
• Slow message delivery: If messages are being delivered slowly, it could be due to network issues or insufficient resources. Check your network connectivity and ensure Kafka has enough CPU, memory, and disk resources.
• Data loss: If you’re experiencing data loss, it could be due to insufficient replication or a misconfigured retention policy. Ensure you have enough replicas for each partition and that your retention policy is correctly configured.

Congratulations! You have successfully installed Apache Kafka. Thanks for using this tutorial to install the latest version of Apache Kafka on Debian 12 Bookworm. For additional help or useful information, we recommend you check the official Apache website.