CentOSLinuxTutorials

How To Install ELK Stack on CentOS 7

Install ELK Stack on CentOS 7

In this tutorial, we will show you how to install ELK Stack on CentOS 7. For those of you who didn’t know, ELK stack is a popular, open-source log management platform. It is used as centralized management for storing, analyzing, and viewing logs. Centralized management makes it easier to study the logs & identify issues if any for any number of servers.

This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘sudo‘ to the commands to get root privileges. I will show you the step-by-step install ELK Stack (Elasticsearch, Logstash, and Kibana) on CentOS 7 server.

Prerequisites

  • A server running one of the following operating systems: CentOS 7.
  • It’s recommended that you use a fresh OS install to prevent any potential issues.
  • SSH access to the server (or just open Terminal if you’re on a desktop).
  • A non-root sudo user or access to the root user. We recommend acting as a non-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.

Install ELK Stack on CentOS 7

Step 1. First, let’s start by ensuring your system is up-to-date.

yum clean all
yum -y update

Step 2. Installing Java.

You need a Java Runtime Environment (JRE) because Elasticsearch is written in Java programming language, you can install the OpenJDK package that includes JRE:

yum install java-1.8.0-openjdk.x86_64

Verify the Java version:

[root@idroot.us ~]# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)

Step 3. Installing Elasticsearch.

Elasticsearch can be installed with a package manager by adding Elastic’s package repository:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.0.0.rpm

Then install the RPM package that you just downloaded:

rpm -ivh elasticsearch-5.0.0.rpm

Start and enable the service:

systemctl enable elasticsearch
systemctl start elasticsearch

Now run the following command from the terminal to check if the Elasticsearch is working properly:

curl -X GET http://localhost:9200

You should get the following output:

{
"name" : "idroot.us",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "k27ZZFJPTaOtwg6_pyzEiw",
"version" : {
"number" : "5.5.0",
"build_hash" : "2cfe0df",
"build_date" : "2017-05-29T16:05:51.443Z",
"build_snapshot" : false,
"lucene_version" : "6.5.1"
},
"tagline" : "You Know, for Search"
}

Step 4. Installing Kibana.

Install Kibana is very simple, you can easily install it using an RPM package:

wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.0-x86_64.rpm

Now just execute the following command so you can start the Kibana service:

systemctl daemon-reload
systemctl start kibana

Kibana is now installed and working on our system. To check the web page, open the web browser & go to the URL mentioned below (use the IP address for your ELK host):

http://localhost:5601

Step 5. Configure ELK stack.

First, we need to create an SSL certificate. This certificate will be used for securing communication between logstash & filebeat clients. Before creating an SSL certificate, we will make an entry of our server IP address in openssl.cnf:

nano /etc/ssl/openssl.cnf

Look for a section with ‘subjectAltName’ & add your server IP to it:

subjectAltName = IP:10.20.30.100

Now change the directory to /etc/ssl and create an SSL certificate:

cd /etc/ssl
openssl req -x509 -days 365 -batch -nodes -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash_frwrd.crt

Step 6. Configure Logstash.

We will now create a configuration file for logstash under the folder ‘/etc/logstash/conf.d‘:

[root@idroot.us ~]# nano /etc/logstash/conf.d/logstash.conf

# input section
input {
 beats {
 port => 5044
 ssl => true
 ssl_certificate => "/etc/ssl/logstash_frwrd.crt"
 ssl_key => "/etc/ssl/logstash-forwarder.key"
 congestion_threshold => "40"
 }
}

The next section i.e. ‘filter section’ will parse the logs before sending them to Elasticsearch:

# Filter section
filter {
if [type] == "syslog" {
 grok {
 match => { "message" => "%{SYSLOGLINE}" }
 }
 date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]

}
 }
}

The last section is ‘output section’ & it defines the location for the storage of logs:

# output section
output {
 elasticsearch {
 hosts => localhost
 index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
 }
stdout {
 codec => rubydebug
 }
}

Now save the file and exit. Now start the logstash service & enable it at boot time:

systemctl start logstash
systemctl enable logstash

Step 7. Installing Filebeat on Clients.

Now to be able to communicate with the ELK stack, Filebeat needs to be installed on all the client machines:

$ nano /etc/yum.repos.d/filebeat.repo
[beats]
name=Elastic Beats Repository
baseurl=https://packages.elastic.co/beats/yum/el/$basearch
enabled=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
gpgcheck=1

Now install filebeat using the following command:

yum install filebeat

After the filebeat has been installed, copy the SSL certificate from the ELK stack server to ‘/etc/ssl’. Next we will make changes to filebeat configuration file to connect the client to the ELK server:

nano /etc/filebeat/filebeat.yml

Make the following changes to the file:

. . .
paths:
– /var/log/*.log
. . .

. . .
document_type: syslog
. . .

. . .
output:
logstash:
hosts: [“10.20.30.100:5044”]
tls:
certificate_authorities: [“/etc/ssl/logstash_frwrd.crt”]
. . .

Now start the service and enable it at boot time:

systemctl restart filebeat
systemctl enable filebeat

Configurations on both server end and client end are now complete. We can now log in to the Kibana web interface to look for analyzed logs.

http://your-ip-address:5601/

Congratulations! You have successfully installed ELK Stack on CentOS 7. Thanks for using this tutorial for installing ELK Stack (Elasticsearch, Logstash, and Kibana) on CentOS 7 systems. For additional help or useful information, we recommend you check the official ELK Stack website.

VPS Manage Service Offer
If you don’t have time to do all of this stuff, or if this is not your area of expertise, we offer a service to do “VPS Manage Service Offer”, starting from $10 (Paypal payment). Please contact us to get the best deal!

r00t

r00t is a seasoned Linux system administrator with a wealth of experience in the field. Known for his contributions to idroot.us, r00t has authored numerous tutorials and guides, helping users navigate the complexities of Linux systems. His expertise spans across various Linux distributions, including Ubuntu, CentOS, and Debian. r00t's work is characterized by his ability to simplify complex concepts, making Linux more accessible to users of all skill levels. His dedication to the Linux community and his commitment to sharing knowledge makes him a respected figure in the field.
Back to top button