Install ELK on CentOS8 Using YUM
Couple weeks ago I was assigned a ELK project. The goal was to integrate varies Azure service’s logs/metrics with ELK. I will talk about the integration part next time.
Installing ELK was easy but I encountered couple issues while setting it up. I will go through the installation process briefly but mainly talk about the issues I encountered. Let’s get started.
Setting up ELK repository
There are various ways to setup ELK on different platforms. Since I am using CentOS8 so I use YUM to install ELK cause YUM will take care all the dependencies for me.
My standard procedure to set everything up:
- Create ELK.repo file with content below
- Refresh repositories to make sure ELK repo is added
- Install Elasticsearch, Kibana, Logstash, java and all kinds of beats if needed
Once repo is setup, install ELK with command below. Java is needed as well since ELK is written in Java.
yum install elasticsearch kibana logstash metricbeat java -y
Before we startup all services, we should do some configuration according to our network structure. Since this is a demo and I have all services installed on the same VM, therefore all network settings can be localhost besides kibana. We have to set kibana’s network setting to 0.0.0.0 so we can access kibana using it’s public ip.
Next, start all services with systemctl start command at the order of Elasticsearch, Kibana and lastly metricbeat / logstash. We started in such order cause metricbeat needs E.S and kibana. Kibana needs E.S.
At this point, you will encounter a issue that logstash is not controlled by systemctl command. Use command below to solve it
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
At this point, you will encounter another logstash issue that it keeps restarting itself. After some debugging and found that it needs at least one configuration file at logstash_location/conf.d/ but there is none by default, so let’s make a quick one to make it works.
Logstash configuration file is composed by three sections, which are Inputs, Filters and Outputs. Use the config below, restart logstash to send the installed metricbeat metrics to installled E.S through logstash.
Pay attention that by default, metricbeat sends metrics to E.S directly! The config file below is only meant to make logstash functional. If you want to send metricbeat metrics to E.S through logstash, you need to modify metricbeat config file’s output section.
If everything is setup correctly, you would be able to see metrics from kibana as below. Since we change kibana’s network setting to 0.0.0.0, so we can access it through it’s public IP address at 5601 port.
At this point, ELK stack is done installed and working. I will share another post regarding ELK configuration.
Another issue that I encountered is that, if I have metricbeat installed on other servers, as mentioned before, E.S location needs to be configured in metricbest’s config file, therefore, E.S must be intranet or internet accessible by metricbeat. And with such configuration, E.S throws another error
After some debugging, it turns out that the default discovery settings are unsuitable for production use, the explanation is here. The solution is to add the line below into E.S configuration file.
discovery.type: single-node
Once done, restart E.S and it will function correctly. For more information regarding the solution, please refer discovery.type and Single-node discovery.
That’s it for today, see you guys around!
