ELK stack consists of a set of applications for retrieving and managing log files. In the software development industry, log files play a vital role to identify the problem and troubleshoot the issue. ELK stack is a collection of different open-source application tools such as Elasticsearch, Kibana, and Logstash. ELK can be used to collect, search and visualize the logs generated from any source in any pattern using a query. In this article, we will learn how to install and configure the ELK stack on Ubuntu and Debian.

Prerequisites:

  1. Fresh Ubuntu 20.04 or Debian 10 Server
  2. Root Privileged Account
  3. Proper Internet connection

Install Java

Installation of ELK stack requires a Java environment. Run the following command to install java on Ubuntu/Debian

$ sudo apt install openjdk-8-jdk

Verify the installation by checking the java version

$ java -version

Output:

Check the installed Java version

Install and configure Elasticsearch

Once java is installed, now it’s time to install and configure Elasticsearch. Since Elasticsearch packages are not available by default on Ubuntu/Debian we need to add the elasticsearch apt repository. Run the following command to add the GPG repository key.

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Now create the repository file using the command.

$ echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Once the repository file is created, elasticsearch can be installed using the command.

$ sudo apt update
$ sudo apt install elasticsearch

The default configuration file of elasticsearch is located at /etc/elasticsearch/elasticsearch.yml . Use any text editor and uncomment the lines:

network.host: localhost
http.port: 9200

Configure Elasticsearch

Start and enable elasticsearch

$ sudo systemctl start elasticsearch
$ sudo systemctl enable elasticsearch

Run the following command to view Elasticsearch status and details

$ curl -X GET "localhost:9200"

Output:

Test Elasticsearch

Install and configure Logstash

Logstash package is available by default in Ubuntu/Debian systems. Run the following command to install.

$ sudo apt install logstash

Start and enable the service

$ sudo systemctl start logstash
$ sudo systemctl enable logstash

Check the service using the command

$ systemctl status logstash

Configure Logstash

Default configuration directory of logstash is /etc/logstash/conf.d/ . Once the installation is completed INPUT, FILTER and OUTPUT pipelines can be configured based on required use cases.

Install and configure Kibana

Kibana is a web based GUI tool used for analyzing and parsing the collected logs. Kibana is available in the default repository of Ubuntu/Debian. Run the following command to install the package.

$ sudo apt install kibana

To configure kibana, go to the default configuration directory and uncomment the following lines

$ sudo vim /etc/kibana/kibana.yml
server.port: 5601
server.host: "localhost"
elasticsearch.hosts: ["http://localhost:9200"]

Configure Kibana

Start and enable the service

$ sudo systemctl start kibana
$ sudo systemctl enable kibana

Allow the kibana port in the firewall

$ sudo ufw allow 5601/tcp

Now access the Kibana dashboard using the url http://localhost:5601

Kibana dashboard

Install and configure filebeat

Filebeat is used for sending logs to elasticsearch and logstash for parsing. Filebeat is available by default in Ubuntu/Debian repository. Run the following command to install.

$ sudo apt install filebeat -y

To configure the filebeat, go to the default configuration directory and comment out the following.

$ sudo vim /etc/filebeat/filebeat.yml

# output.elasticsearch:
# Array of hosts to connect to .
# hosts: ["localhost:9200"]

Uncomment the following line and save the file

output.logstash:
hosts: [“localhost:5044”]

Configure Filebeat

In the next step, enable filebeat system module

$ sudo filebeat modules enable system

Enable filebeat modules

Now run the following command to load the index template

$ sudo filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

Load filebeat template

Start and enable filebeat service

$ sudo systemctl start filebeat
$ sudo systemctl enable filebeat

Check the status

$ sudo systemctl status filebeat

Check Filebeat status

Conclusion

In this article, I have covered how to install and configure the ELK stack on Debian/Ubuntu in the correct way. Also, we have learned how to use different components such as Kibana, Logstash, and Kibana to analyze and visualize the logs from any source.

 

How to Install and Configure ELK Stack on Ubuntu and Debian

Karim Buzdar

About the Author: Karim Buzdar holds a degree in telecommunication engineering and holds several sysadmin certifications. As an IT engineer and technical author, he writes for various web sites. You can reach Karim on LinkedIn