How To Install ELK Stack on CentOS 7

Install ELK Stack on CentOS 7

In this tutorial, we will show you how to install ELK Stack on CentOS 7 server. For those of you who didn’t know, ELK stack is a popular, open-source log management platform. It is used as centralized management for storing, analyzing and viewing of logs. Centralized management makes it easier to study the logs & identify issues if any for any number of servers.

This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘sudo’ to the commands to get root privileges. I will show you through the step by step install ELK Stack (Elasticsearch, Logstash and Kibana) on CentOS 7 server.

Install ELK Stack on CentOS 7

Step 1. First, let’s start by ensuring your system is up-to-date.

yum clean all
yum -y update

Step 2. Installing Java.

You need a Java Runtime Environment (JRE) because Elasticsearch is written in Java programming language, you can install OpenJDK package that includes JRE:

yum install java-1.8.0-openjdk.x86_64

Verify the Java version:

[root@idroot.us ~]# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)

Step 3. Installing Elasticsearch.

Elasticsearch can be installed with a package manager by adding Elastic’s package repository:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.0.0.rpm

Then install the RPM package that you just downloaded:

rpm -ivh elasticsearch-5.0.0.rpm

Start and enable the service:

systemctl enable elasticsearch
systemctl start elasticsearch

Now run the following command from the terminal to check if the elasticsearch is working properly:

curl -X GET http://localhost:9200

You should get the following output:

{
"name" : "idroot.us",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "k27ZZFJPTaOtwg6_pyzEiw",
"version" : {
"number" : "5.5.0",
"build_hash" : "2cfe0df",
"build_date" : "2017-05-29T16:05:51.443Z",
"build_snapshot" : false,
"lucene_version" : "6.5.1"
},
"tagline" : "You Know, for Search"
}

Step 4. Installing Kibana.

Install Kibana is very simple, you can easily install it using an RPM package:

wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.0-x86_64.rpm

Now just execute the following command so you can start the Kibana service:

systemctl daemon-reload
systemctl start kibana

Kibana is now installed and working on our system. To check the web-page, open the web browser & go to the URL mentioned below (use the IP address for your ELK host):

http://localhost:5601

Step 5. Configure ELK stack.

First, we need to create an SSL certificate. This certificate will be used for securing communication between logstash & filebeat clients. Before creating an SSL certificate, we will make an entry of our server IP address in openssl.cnf:

nano /etc/ssl/openssl.cnf

Look for section with ‘subjectAltName’ & add your server IP to it:

subjectAltName = IP:10.20.30.100

Now change the directory to /etc/ssl and create SSL certificate:

cd /etc/ssl
openssl req -x509 -days 365 -batch -nodes -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash_frwrd.crt

Step 6. Configure Logstash.

We will now create a configuration file for logstash under the folder ‘/etc/logstash/conf.d‘:

[root@idroot.us ~]# nano /etc/logstash/conf.d/logstash.conf

# input section
input {
 beats {
 port => 5044
 ssl => true
 ssl_certificate => "/etc/ssl/logstash_frwrd.crt"
 ssl_key => "/etc/ssl/logstash-forwarder.key"
 congestion_threshold => "40"
 }
}

Next section i.e. ‘filter section’ will parse the logs before sending them to elasticsearch:

# Filter section
filter {
if [type] == "syslog" {
 grok {
 match => { "message" => "%{SYSLOGLINE}" }
 }
 date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]

}
 }
}

The last section is ‘output section’ & it defines the location for the storage of logs:

# output section
output {
 elasticsearch {
 hosts => localhost
 index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
 }
stdout {
 codec => rubydebug
 }
}

Now save the file and exit. Now start the logstash service & enable it at boot time:

systemctl start logstash
systemctl enable logstash

Step 7. Installing Filebeat on Clients.

Now to be able to communicate with the ELK stack, Filebeat needs to installed on all the client machines:

$ nano /etc/yum.repos.d/filebeat.repo
[beats]
name=Elastic Beats Repository
baseurl=https://packages.elastic.co/beats/yum/el/$basearch
enabled=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
gpgcheck=1

Now install filebeat using following command:

yum install filebeat

After the filebeat has been installed, copy the ssl certificate from the ELK stack server to ‘/etc/ssl’. Next we will make changes to filebeat configuration file to connect the client to ELK server:

nano /etc/filebeat/filebeat.yml

Make the following changes to file:

. . .
paths:
– /var/log/*.log
. . .

. . .
document_type: syslog
. . .

. . .
output:
logstash:
hosts: [“10.20.30.100:5044”]
tls:
certificate_authorities: [“/etc/ssl/logstash_frwrd.crt”]
. . .

Now start the service and enable it at boot time:

systemctl restart filebeat
systemctl enable filebeat

Configurations on both server end and client end are now complete. We can now login to the kibana web interface to look for analysed logs.

http://your-ip-address:5601/

Congratulations! You have successfully installed ELK Stack on CentOS 7. Thanks for using this tutorial for installing ELK Stack (Elasticsearch, Logstash and Kibana) on CentOS 7 systems. For additional help or useful information, we recommend you to check the official ELK Stack website.

VPS Manage Service Offer
If you don’t have time to do all of this stuff, or if this is not your area of expertise, we offer a service to do “VPS Manage Service Offer”, starting from $10 (Paypal payment). Please contact us to get the best deal!