How To Install ELK Stack on Ubuntu 24.04 LTS

In today’s data-driven world, managing and analyzing logs efficiently is crucial for system administrators and developers. The ELK Stack, comprising Elasticsearch, Logstash, and Kibana, has emerged as a powerful solution for centralized logging and real-time data analysis. This guide will walk you through the process of installing the ELK Stack on Ubuntu 24.04 LTS, providing you with a robust platform for log management and visualization.

The ELK Stack is an open-source suite of tools that work together to provide a comprehensive log management and analysis solution. Elasticsearch serves as the search and analytics engine, Logstash handles data processing and transformation, while Kibana offers a user-friendly interface for data visualization and exploration.

By implementing the ELK Stack, organizations can gain valuable insights from their log data, identify trends, troubleshoot issues more effectively, and make data-driven decisions. Whether you’re managing a small network or overseeing a large-scale infrastructure, the ELK Stack can significantly enhance your ability to monitor and optimize your systems.

Prerequisites

Before diving into the installation process, ensure that your system meets the following requirements:

• Ubuntu 24.04 LTS (Focal Fossa) installed and updated
• A minimum of 4GB RAM (8GB or more recommended for production environments)
• At least 2 CPU cores
• Sufficient disk space (at least 20GB free)
• Root or sudo privileges
• Terminal access
• OpenJDK 11 installed (required for Elastic Stack components)

To install OpenJDK 11, run the following commands:

sudo apt update
sudo apt install openjdk-11-jdk

Verify the Java installation by running:

java -version

Step 1: Install Elasticsearch

Elasticsearch is the heart of the ELK Stack, providing powerful search and analytics capabilities. Let’s begin by installing and configuring Elasticsearch on your Ubuntu 24.04 LTS system.

Add Elasticsearch Repository

First, we need to add the Elasticsearch GPG key and repository to our system:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

Install Elasticsearch

Now, update the package index and install Elasticsearch:

sudo apt update
sudo apt install elasticsearch

Configure Elasticsearch

Open the Elasticsearch configuration file for editing:

sudo nano /etc/elasticsearch/elasticsearch.yml

Modify the following lines to restrict Elasticsearch to localhost and set the cluster name:

network.host: localhost
cluster.name: my-elk-cluster

Start and Enable Elasticsearch

Start the Elasticsearch service and enable it to run on boot:

sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

Verify Elasticsearch Installation

To ensure Elasticsearch is running correctly, send a test HTTP request:

curl -X GET "localhost:9200"

You should receive a JSON response with Elasticsearch version information.

Step 2: Install Logstash

Logstash is responsible for collecting, processing, and forwarding log data to Elasticsearch. Let’s proceed with the Logstash installation.

Install Logstash

Install Logstash using the apt package manager:

sudo apt install logstash

Configure Logstash

Create a basic Logstash configuration file:

sudo nano /etc/logstash/conf.d/logstash.conf

Add the following configuration to process system logs and forward them to Elasticsearch:

input {
  file {
    path => "/var/log/syslog"
    start_position => "beginning"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}

Start and Enable Logstash

Start the Logstash service and enable it to run on boot:

sudo systemctl start logstash
sudo systemctl enable logstash

Verify Logstash Service

Check the status of the Logstash service:

sudo systemctl status logstash

Ensure that the service is active and running without errors.

Step 3: Install Kibana

Kibana provides a user-friendly web interface for visualizing and exploring data stored in Elasticsearch. Let’s set up Kibana to complete our ELK Stack installation.

Install Kibana

Install Kibana using the apt package manager:

sudo apt install kibana

Configure Kibana

Edit the Kibana configuration file:

sudo nano /etc/kibana/kibana.yml

Modify the following lines to restrict Kibana to localhost:

server.host: "localhost"
elasticsearch.hosts: ["http://localhost:9200"]

Start and Enable Kibana

Start the Kibana service and enable it to run on boot:

sudo systemctl start kibana
sudo systemctl enable kibana

Set Up Nginx as a Reverse Proxy

To access Kibana securely from a web browser, we’ll use Nginx as a reverse proxy. First, install Nginx:

sudo apt install nginx

Create a new Nginx server block configuration:

sudo nano /etc/nginx/sites-available/kibana

Add the following configuration:

server {
    listen 80;
    server_name your_domain.com;

    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Enable the new configuration and restart Nginx:

sudo ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx

Access Kibana Web Interface

Open a web browser and navigate to http://your_domain.com to access the Kibana interface. You should see the Kibana welcome page.

Step 4: Securing the ELK Stack

Security is crucial when deploying the ELK Stack, especially in production environments. Let’s implement some basic security measures to protect our installation.

Configure SSL/TLS for Nginx

To encrypt traffic between clients and Kibana, we’ll use Let’s Encrypt to obtain an SSL/TLS certificate:

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d your_domain.com

Follow the prompts to complete the certificate installation and Nginx configuration.

Set Up Basic Authentication for Kibana

Create a password file for Kibana access:

sudo apt install apache2-utils
sudo htpasswd -c /etc/nginx/.htpasswd admin

Update the Nginx configuration to include basic authentication:

sudo nano /etc/nginx/sites-available/kibana

Add the following lines within the location block:

auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/.htpasswd;

Restart Nginx to apply the changes:

sudo systemctl restart nginx

Configure Firewall

If you’re using UFW (Uncomplicated Firewall), allow necessary traffic:

sudo ufw allow 'Nginx Full'
sudo ufw enable

Step 5: Loading and Visualizing Data

With the ELK Stack installed and secured, it’s time to start loading data and creating visualizations.

Using Filebeat to Ship Logs

Filebeat is a lightweight log shipper that can send log files to Logstash or Elasticsearch. Install Filebeat:

sudo apt install filebeat

Configure Filebeat to collect system logs:

sudo nano /etc/filebeat/filebeat.yml

Enable the system module and set the output to Elasticsearch:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log

output.elasticsearch:
  hosts: ["localhost:9200"]

Start and enable Filebeat:

sudo systemctl start filebeat
sudo systemctl enable filebeat

Creating Visualizations in Kibana

Access Kibana through your web browser and follow these steps to create a basic visualization:

1. Click on “Visualize” in the left sidebar
2. Choose “Create new visualization”
3. Select a visualization type (e.g., “Line”)
4. Choose a source index pattern
5. Configure the visualization settings (metrics, buckets, etc.)
6. Save and add the visualization to a dashboard

Experiment with different visualization types and data sources to gain insights from your log data.

Congratulations! You have successfully installed ELK Stack. Thanks for using this tutorial for installing the ELK Stack on Ubuntu 24.04 LTS LTS system. For additional help or useful information, we recommend you check the official ELK Stack website.