In this tutorial, we will show you how to install ELK Stack on CentOS 7. For those of you who didn’t know, ELK stack is a popular, open-source log management platform. It is used as centralized management for storing, analyzing, and viewing logs. Centralized management makes it easier to study the logs & identify issues if any for any number of servers.
This article assumes you have at least basic knowledge of Linux, know how to use the shell, and most importantly, you host your site on your own VPS. The installation is quite simple and assumes you are running in the root account, if not you may need to add ‘sudo‘ to the commands to get root privileges. I will show you the step-by-step install ELK Stack (Elasticsearch, Logstash, and Kibana) on CentOS 7 server.
Prerequisites
- A server running one of the following operating systems: CentOS 7.
- It’s recommended that you use a fresh OS install to prevent any potential issues.
- SSH access to the server (or just open Terminal if you’re on a desktop).
- A
non-root sudo useror access to theroot user. We recommend acting as anon-root sudo user, however, as you can harm your system if you’re not careful when acting as the root.
Install ELK Stack on CentOS 7
Step 1. First, let’s start by ensuring your system is up-to-date.
yum clean all yum -y update
Step 2. Installing Java.
You need a Java Runtime Environment (JRE) because Elasticsearch is written in Java programming language, you can install the OpenJDK package that includes JRE:
yum install java-1.8.0-openjdk.x86_64
Verify the Java version:
[root@idroot.us ~]# java -version openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-b12) OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
Step 3. Installing Elasticsearch.
Elasticsearch can be installed with a package manager by adding Elastic’s package repository:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.0.0.rpm
Then install the RPM package that you just downloaded:
rpm -ivh elasticsearch-5.0.0.rpm
Start and enable the service:
systemctl enable elasticsearch systemctl start elasticsearch
Now run the following command from the terminal to check if the Elasticsearch is working properly:
curl -X GET http://localhost:9200
You should get the following output:
{
"name" : "idroot.us",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "k27ZZFJPTaOtwg6_pyzEiw",
"version" : {
"number" : "5.5.0",
"build_hash" : "2cfe0df",
"build_date" : "2017-05-29T16:05:51.443Z",
"build_snapshot" : false,
"lucene_version" : "6.5.1"
},
"tagline" : "You Know, for Search"
}
Step 4. Installing Kibana.
Install Kibana is very simple, you can easily install it using an RPM package:
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.0-x86_64.rpm
Now just execute the following command so you can start the Kibana service:
systemctl daemon-reload systemctl start kibana
Kibana is now installed and working on our system. To check the web page, open the web browser & go to the URL mentioned below (use the IP address for your ELK host):
http://localhost:5601
Step 5. Configure ELK stack.
First, we need to create an SSL certificate. This certificate will be used for securing communication between logstash & filebeat clients. Before creating an SSL certificate, we will make an entry of our server IP address in openssl.cnf:
nano /etc/ssl/openssl.cnf
Look for a section with ‘subjectAltName’ & add your server IP to it:
subjectAltName = IP:10.20.30.100
Now change the directory to /etc/ssl and create an SSL certificate:
cd /etc/ssl openssl req -x509 -days 365 -batch -nodes -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash_frwrd.crt
Step 6. Configure Logstash.
We will now create a configuration file for logstash under the folder ‘/etc/logstash/conf.d‘:
[root@idroot.us ~]# nano /etc/logstash/conf.d/logstash.conf
# input section
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/ssl/logstash_frwrd.crt"
ssl_key => "/etc/ssl/logstash-forwarder.key"
congestion_threshold => "40"
}
}
The next section i.e. ‘filter section’ will parse the logs before sending them to Elasticsearch:
# Filter section
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
The last section is ‘output section’ & it defines the location for the storage of logs:
# output section
output {
elasticsearch {
hosts => localhost
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
Now save the file and exit. Now start the logstash service & enable it at boot time:
systemctl start logstash systemctl enable logstash
Step 7. Installing Filebeat on Clients.
Now to be able to communicate with the ELK stack, Filebeat needs to be installed on all the client machines:
$ nano /etc/yum.repos.d/filebeat.repo [beats] name=Elastic Beats Repository baseurl=https://packages.elastic.co/beats/yum/el/$basearch enabled=1 gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch gpgcheck=1
Now install filebeat using the following command:
yum install filebeat
After the filebeat has been installed, copy the SSL certificate from the ELK stack server to ‘/etc/ssl’. Next we will make changes to filebeat configuration file to connect the client to the ELK server:
nano /etc/filebeat/filebeat.yml
Make the following changes to the file:
. . . paths: – /var/log/*.log . . . . . . document_type: syslog . . . . . . output: logstash: hosts: [“10.20.30.100:5044”] tls: certificate_authorities: [“/etc/ssl/logstash_frwrd.crt”] . . .
Now start the service and enable it at boot time:
systemctl restart filebeat systemctl enable filebeat
Configurations on both server end and client end are now complete. We can now log in to the Kibana web interface to look for analyzed logs.
http://your-ip-address:5601/
Congratulations! You have successfully installed ELK Stack on CentOS 7. Thanks for using this tutorial for installing ELK Stack (Elasticsearch, Logstash, and Kibana) on CentOS 7 systems. For additional help or useful information, we recommend you check the official ELK Stack website.
