Home » Linux » CentOS » How to install and configure Elasticsearch, Logstash, Kibana (ELK Stack) on Ubuntu/Debian/Centos

How to install and configure Elasticsearch, Logstash, Kibana (ELK Stack) on Ubuntu/Debian/Centos

Large information systems generate a huge amount of logs that needs to be stored somewhere. I will talk about how to set up a repository for logging based on Elasticsearch, Logstash and Kibana, which is often called the ELK Stack. In this repository, you can store any logs in different formats and large volumes.

Introduction

I just want to warn you that I do not have much experience using this stack. He came to my attention, so I set up one system. I looked at it, tested it, I liked it. Set up another one. At the moment I am studying it. This article will be the basics – how to make a quick installation and basic setup so that you can collect logs in Elasticsearch and watch them in Kibana.

Installation instructions will be completely based on official documentation. The problem with documentation is that all information is scattered in different articles. If you are setting up an ELK Stack for the first time, it will not be very easy to figure it all out right away. In my article, I gathered in one place the necessary minimum for running Elasticsearch, Logstash, Kibana and the Filebeat and Winlogbeat agents for sending logs from servers.

What is ELK Stack

I will tell in my own words about what we will install. Earlier on my website I already talked about centralized logging using syslog-ng. This is a working solution, but when you have a lot of logs, storing them in text files is undesirable. It is necessary to somehow handle and display in a convenient form for viewing. These problems are solved by the presented stack of programs:

  • Elasticsearch is used for storage, analysis, search by logs.
  • Kibana is web panel for working with logs.
  • Logstash service for collecting logs and sending them to Elasticsearch. In the simplest configuration, you can do without it and send logs directly to Elasticsearch. But with logstash it is more flexible to do it.
  • Beats – agents to send logs to Logstash. They are different. I will use Filebeat to send data from linux and Winlogbeat text logs to send logs from Windows logs.

Nginx, which proxies connections to Kibana, is added to this bundle. In the simplest case, it is not needed, but it’s more flexible. You can, for example, add an authorization or an ssl certificate, in nginx it is easy to manage the domain name. In the case of a heavy load, different services are spread across different servers or clusters. In my example, I will install everything on one server. Schematically, the work of this system can be represented like this:

ELC STACK operation scheme (Elasticsearch, Logstash, Kibana)

Let’s begin in stages to install and configure all the components of our future system for storing and processing logs of various information systems.

Install Java 8

All system components, with the exception of agents on servers, are written in java, so we will start our setup with installing Oracle Java 8. This version is chosen because it is supported by all versions of Elasticsearch. More information about the compatibility of software products, see a separate table from the documentation.

CentOS

Go to http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html, press “Accept License Agreement” and copy lonk on file *-linux-x64.rpm.

Install Java Oracle on Centos 7

In the server console, download and install Java 8.

# cd ~
# wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u181-b13/96a7b8442fe848ef90c96a2fad6ed6d1/jre-8u181-linux-x64.rpm"
# yum localinstall jre-*

Check the installed version:

# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Ubuntu

Add Java 8 repository.

# add-apt-repository -y ppa:webupd8team/java

Update the package list and install Oracle Java 8.

# apt update
# apt install oracle-java8-installer

Check the installed version of java:

# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Debian

The installation is almost the same as that of Ubuntu. Add repository first.

# apt install software-properties-common
# add-apt-repository "deb http://ppa.launchpad.net/webupd8team/java/ubuntu xenial main"

Install Java 8 Oracle in Debian 9.

# apt update
# apt install oracle-java8-installer

Check the installed version of java in console:

# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Install Elasticsearch

Install the base to collect logs – Elasticsearch. Its installation is very simple due to ready-made packages for all popular platforms.

Centos 7

Copy the public key of the repository:

# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add Elasticsearch repository:

# mcedit /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1

Start install elasticsearch:

# yum install elasticsearch

At the end of the installation, add elasticsearch to the autostart and launch it with default settings:

# systemctl daemon-reload
# systemctl enable elasticsearch.service
# systemctl start elasticsearch.service

Check if the service has started:

# systemctl status elasticsearch.service

Installation elasticsearch

Ubuntu/Debian

We copy the public key of the repository:

# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

If you do not have apt-transport-https package, then you need to install:

# apt install apt-transport-https

Add Elasticsearch repository:

# echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list

Install Elasticsearch on Debian or Ubuntu:

# apt update && apt-get install elasticsearch

After installation we add elasticsearch to autostart and launch it.

# systemctl daemon-reload 
# systemctl enable elasticsearch.service 
# systemctl start elasticsearch.service

Check if it started:

# systemctl status elasticsearch.service

If everything is OK, then proceed to configure Elasticsearch.

Configure Elasticsearch

Elasticsearch settings are in the /etc/elasticsearch/elasticsearch.yml file. At the start, we will be interested in the following parameters:

path.data: /var/lib/elasticsearch # data directory
network.host: 127.0.0.1 # listen only localhost

By default, Elasticsearch listens on all available network interfaces. We do not need this, since the data will be transferred to it by logstash, which will be installed locally. Pay special attention to the parameter for the data directory. The logs will hold a lot of  datastore. Think in start where you will store the logs. All other settings I leave default.

After changing the settings, you need to restart the service:

# systemctl restart elasticsearch.service

Check result:

# netstat -tulnp | grep 9200
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      14130/java

Elasticsearch start on the local interface. And I see that he is listening to ipv6, but nothing about ipv4. But he listens to him too, so that’s all right. Go to the installation of kibana.

Kibana installation

Next, install the Kibana web panel to visualize the data obtained from Elasticsearch. There is also nothing difficult here, the repository and ready-made packages are available for all popular platforms. The repositories and public key for the Kibana installation will be the same as in the Elasticsearch installation. But I will repeat everything once again for those who will install only Kibana, without everything else. This product is finished and is used not only with Elasticsearch.

Centos

Import the repository key:

# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add repository config:

# mcedit /etc/yum.repos.d/kibana.repo
[kibana-6.x]
name=Kibana repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Starting the Kibana installation:

# yum install kibana

Adding Kibana to autoload and starting it:

# systemctl daemon-reload
# systemctl enable kibana.service
# systemctl start kibana.service

Check the status of the running service:

# systemctl status kibana.service

By default, Kibana listens on port 5601. Just do not rush to check it after launch. Kibana starts long. Wait about a minute and check.

# netstat -tulnp | grep 5601
tcp        0      0 127.0.0.1:5601          0.0.0.0:*               LISTEN      27401/node

Ubuntu/Debian

Installing Kibana on Debian or Ubuntu is the same as on centos – we connect the repository and install from the deb package. Add a public key:

# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Add a kibana repository:

# echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list

Start the Kibana installation:

# apt update && apt install kibana

Add Kibana to autoload and run it:

# systemctl daemon-reload
# systemctl enable kibana.service
# systemctl start kibana.service

Check the status of the running service:

# systemctl status kibana.service

By default, Kibana listens on port 5601. Just do not rush to check it after launch. Kibana starts long. Wait about a minute and check.

# netstat -tulnp | grep 5601
tcp        0      0 127.0.0.1:5601          0.0.0.0:*               LISTEN      27401/node

Kibana setup

The file with the settings of the Kiban is located along the path – /etc/kibana/kibana.yml. At the initial stage, you can not touch anything at all and leave everything as it is. By default, kibana listens only to localhost and does not allow to connect remotely. This is a normal situation if you have nginx installed on the same server as a reverse proxy, which will accept the connections and proxy pass them to the kibana. This is what should be done in production, when the system will be used by different people from different places. With the help of nginx, it will be possible to differentiate access, use a certificate, configure a domain name, etc.

If you have this test installation, then you can do without nginx. To do this, you must allow Kibana to listen to the external interface and accept connections. Change the server.host parameter by specifying the ip address of the server, for example like this:

server.host: "10.1.4.114"

If you want kibana to listen to all interfaces, specify 0.0.0.0 as the address. After that, Kibana must be restarted:

# systemctl restart kibana.service

Now you can go to the web interface athttp://10.1.4.114:5601.

Kibana setup

You can continue to configure and test, and when everything is finished, run nginx and configure the proxying. I will leave the nginx setup to the very end. During the setup process I will connect directly to Kibana.

Installing and Configuring Logstash

Logstash is installed as easy as Elasticsearch and Kibana, from the same repository. I will not show again how to add it. Just install it and add it to autoload.

Installing Logstash в Centos 7:

# yum install logstash

Installing Logstash в Debian/Ubuntu:

# apt install logstash

Add logstash в autostart:

# systemctl enable logstash.service

We will not run until we need to configure it first. The main logstash config is at /etc/logstash/logstash.yml. I will not touch it, and all the settings will be divided according to different configuration files in the /etc/logstash/conf.d directory. We create the first config input.conf, which will describe the receiving of information from beats agents.

input {
  beats {
    port => 5044
  }
}

Everything is simple. I specify that we accept information on port 5044. It’s enough. If you want to use ssl certificates to transfer logs over secure connections, ssl parameters are added here. I will collect data from the closed perimeter of the local network, I do not need to use ssl.

Now we will specify where we will transfer the data. Here, too, everything is relatively simple. We create a config output.conf which describes data transmission in Elasticsearch.

output {
        elasticsearch {
            hosts    => "localhost:9200"
            index    => "nginx-%{+YYYY.MM.dd}"
        }
	#stdout { codec => rubydebug }
}

Everything is simple here – to transfer all data to elasticsearch under the specified index with a mask in the form of a date. As I understand it, the breakdown of indices by day and by data type is flexible for data management. Then it will be easy to perform data cleaning for these indexes. I commented out the last line. She is responsible for logging. If it is enabled, all incoming data will be sent by logstash to the system log additionally. In centos this is /var/log/messages. Use it only during debugging, otherwise the log will quickly grow with duplicates of incoming data.

There remains the last config with the description of data processing. Here begins a little magic, in which I understood for a while. I’ll tell you below. We create a config file filter.conf.

filter {
 if [type] == "nginx_access" {
    grok {
        match => { "message" => "%{IPORHOST:remote_ip} - %{DATA:user} \[%{HTTPDATE:access_time}\] \"%{WORD:http_method} %{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent_bytes} \"%{DATA:referrer}\" \"%{DATA:agent}\"" }
    }
  }
  date {
        match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
  }
  geoip {
         source => "remote_ip"
         target => "geoip"
         add_tag => [ "nginx-geoip" ]
  }
}

The first thing that this filter does is to parse the nginx logs with the grok, if the log type is specified, and extracts key data from the log that it writes to certain fields so that it is convenient to work with them later. At first I did not understand why this was necessary. The documentation for the filebeat describes well the modules that come with it, which all this can already do, you just need to connect the module.

The poin is that filebeat modules work only if you send data directly to elasticsearch. On it, you also put the appropriate plugin and get the formatted data. But we have an intermediate logstash that accepts data. With it, as I understand it, the filebeat plugins do not work, so you have to parse the data separately in logstash. It is not very difficult, but nonetheless. As I understand it, this is a fee for the convenience provided by logstash. If you have a lot of separate data, then send them directly to elasticsearch is not as convenient as using preprocessing in logstash. If I’m wrong, please correct me.

For the grok filter that logstash uses, there is a convenient debugger where you can see how your data will be parsed. I will show on the example of one line from the nginx config. For example, take the following line from the nginx log:

180.163.220.100 - travvels.com [05/Sep/2018:14:45:52 +0300] "GET /assets/galleries/26/1.png HTTP/1.1" 304 0 "https://travvels.com/en/glavnaya/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"

And let’s see how it is parsed by the rule grok, which I used in the config above.

%{IPORHOST:remote_ip} - %{DATA:user} \[%{HTTPDATE:access_time}\] \"%{WORD:http_method} %{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent_bytes} \"%{DATA:referrer}\" \"%{DATA:agent}\"

Actually, you can see the result yourself in the debugger. The filter will parse the log and generate json at the output, where each value will be assigned its own field, according to which then it will be convenient to create reports in elasticsearch to make selections. Just do not forget about the format of the logs. The rule I quoted corresponds to the default format of the main logs in nginx. If you have somehow modified the format of the logs, make changes to the grok filter.

I hope you understand how to work with grok. You can thus parse any logs and transfer them to elasticsearch. Then, on the basis of this data, build reports, charts, dashboards. I plan to parse as I need postfix and dovecot mail logs.

Next, the date module is used to extract the date from incoming logs and use it as the document date in elasticsearch. This is done in order to avoid confusion, if there are delays in the delivery of logs. In the system, messages will be with one date, and inside the log will be another date. Inconvenient to disassemble incidents.

In the end, I use the geoip filter, which, based on the ip address that we received earlier using the grok filter and recorded in the remote_ip field, determines the geographical location. He adds new tags and writes geographic data there. For his work, it was enough for me to describe it in the config and download the database of addresses in the /etc/logstash folder.

# wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
# gunzip GeoLite2-City.mmdb.gz

Finished logstash setup. Run it:

# systemctl start logstash.service

You can check the log /var/log/logstash/logstash-plain.log to make sure everything is OK. Now configure the agents to send data.

Installing Filebeat to send logs to Logstash

Install the first Filebeat agent on the server with nginx to send web server logs to the server with ELK. You can install both from the common repository, which we connected before, and separately packages. How to set – you decide. In the first case, it is necessary to add a repository to all hosts, but then it is convenient to update the packages. If you do not want to connect the repository, you can simply download the package and install it.

Install on Centos 7.

# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.4.0-x86_64.rpm
# rpm -vi filebeat-6.4.0-x86_64.rpm

Install on Debian/Ubuntu:

# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.4.0-amd64.deb
# dpkg -i filebeat-6.4.0-amd64.deb

Or just so:

# yum install filebeat
# apt install filebeat

After installation we create config /etc/filebeat/filebeat.yml for transfer logs to logstash.

filebeat.inputs:
- type: log
  enabled: true
  paths:
      - /var/log/nginx/*-access.log
  fields:
    type: nginx_access
  fields_under_root: true
  scan_frequency: 5s

- type: log
  enabled: true
  paths:
      - /var/log/nginx/*-error.log
  fields:
    type: nginx_error
  fields_under_root: true
  scan_frequency: 5s

output.logstash:
  hosts: ["10.1.4.114:5044"]

xpack.monitoring:
  enabled: true
  elasticsearch:
    hosts: ["http://10.1.4.114:9200"]

Some explanations for the configuration, as it is not entirely default and minimalist. I modified it a little for convenience. First, I separated the access and error logs using a separate type field, where I write the corresponding log type: nginx_access or nginx_error. Depending on the type, the processing rules in logstash change. Plus, I turned on monitoring and for this I specified the address elastichsearch, where the filebeat sends the monitoring data directly. Showing it to you just to demonstrate the opportunity. I always have a separate monitoring on zabbix, so there’s not much point in a separate monitoring. But you look at him, maybe he will come in handy.

In order for monitoring to work, it must be activated in the appropriate section in Kibana – Monitoring. And do not forget to run elasticsearch on the external interface. In the initial setup, I indicated to listen only to the local interface.

Let’s start filebeat and add to autoload.

# systemctl start filebeat
# systemctl enable filebeat

Check the log at /var/log/filebeat/filebeat. He is very informative. If everything is OK, you will see a list of all the logs in the /var/log/nginx directory that the filebeat found and began to prepare for sending. If done correctly, the data has already transferred into Elasticsearch. We can see them in Kibana. To do this, open the web interface and go to the Discover section. Since there is no index there yet, we will be redirected to the Management section, where we can add it.


Adding an index to Kibana

You should see the index that started logging logstash into elasticsearch. In the Index pattern field, type nginx-* and click Next Step. In the next step, select the field name for the temporary filter. You will have only one option – @timestamp, select it and click Create Index Pattern.

Creating an index with data

New index has been added. Now when you go to the Discover section, it will open by default with all the data that goes to it.

View Elasticsearch logs in Kibana

Getting logs from linux servers configured. Now do the same for windows logs.

Installing and configuring Winlogbeat

To configure a centralized server for collecting logs from Windows servers, install the winlogbeat system log collector. To do this, download it and install it as a service. Go to the download page and download the version for your architecture – 32 or 64 bits.

Unpack the downloaded archive and copy its contents to the C:\Program Files\Winlogbeat directory. Create a logs folder for storing the logs of winlogbeat.

Installing Winlogbeat to collect logs from Windows system logs

Run the PowerShell console from the administrator and execute the commands:

PS C:\Users\Administrator> cd 'C:\Program Files\Winlogbeat'
PS C:\Program Files\Winlogbeat> .\install-service-winlogbeat.ps1

I had this console output:

Status   Name               DisplayName
------   ----               -----------
Stopped  winlogbeat         winlogbeat

The service has been installed and is currently stopped. Edit the configuration file winlogbeat.yml. I created it with such content:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
  - name: System

tags: ["winsrv"]

output.logstash:
  hosts: ["10.1.4.114:5044"]

logging.level: info
logging.to_files: true
logging.files:
  path: C:/Program Files/Winlogbeat/logs
  name: winlogbeat
  keepfiles: 7

xpack.monitoring:
  enabled: true
  elasticsearch:
    hosts: ["http://10.1.4.114:9200"]

I take 3 system logs – Application, Security, System and send them to the server with logstash. Let’s set up log storage and enable monitoring by analogy with filebeat. Separately pay attention to the tags: [“winsrv”]. I mark all sent messages with this tag in order to process them in logstash and send it to elasticsearch with a separate index. I could not use the type field, by analogy with the filebeat, because the winlogbeat in the type field is set wineventlog and cannot be changed. If you have data from all servers going to one index, then you can not add the tag, but use the specified type field for sorting. If you want to place data from different servers in different indexes, then separate them with tags.

After setup, you can start the winlogbeat service, which will appear in the list of windows services. But in order for the logs of the Windows logs to go to Elasticsearch not in one heap along with the nginx logs, we need to set up a separate index for them in the logstash in the output section. Go to the server with logstash and edit the config output.conf.

output {
    if [type] == "nginx_access" {
        elasticsearch {
            hosts     => "localhost:9200"
            index    => "nginx-%{+YYYY.MM.dd}"
        }
    }
    else if [type] == "nginx_error" {
        elasticsearch {
            hosts     => "localhost:9200"
            index    => "nginx-%{+YYYY.MM.dd}"
        }
    }
    else if "winsrv" in [tags] {
        elasticsearch {
            hosts     => "localhost:9200"
            index    => "winsrv-%{+YYYY.MM.dd}"
        }
    }
    else {
        elasticsearch {
            hosts     => "localhost:9200"
            index    => "unknown_messages"
        }
    }
	#stdout { codec => rubydebug }
}

I think the text makes sense. I divided the logs of nginx, the system logs of Windows servers for different indexes and added a separate index unknown_messages, which will get everything that did not fall into the previous ones. This is just a reserve for the future, when the configuration is more complicated, you can catch messages that for some reason did not fall into any of the above rules.

I couldn’t put both types of nginx_error and nginx_access in one rule, because I didn’t understand at once how to write it down correctly, and according to the documentation I was already tired of looking for information. So turned out a working version. After that, I restarted logstash, waited a bit until new logs appeared on the Windows server, went into kibana and added new indices. Let me remind you that this is done in the section Management -> Kibana -> Index Patterns.

Configure Index to collect logs from Windows system logs

You can go to the Discover section and view logs from Windows servers.

View Windows system logs in ELK Stack

At this point, I finished setting up the stack for Elasticsearch, Logstash, Kibana, Filebeat, and Winlogbeat. Described the main functionality for collecting logs. Then you can work with them on the situation – build visualizes, reports, dashboards, etc. Perhaps I will describe it separately.

Proxy pass connections to Kibana via nginx

I will not talk in detail about what proxying in nginx is. Here is just an example of a config for sending requests from nginx to kibana. I recommend using ssl certificates to access Kibana. Even if there is no objective need for this, browser notifications are annoying that you have an insecure connection.

Here is an example nginx config for proxying Kibana requests with password access restriction:

server {
	listen 443;

	server_name kibana.site.ru
	ssl_certificate /etc/letsencrypt/live/kibana.site.ru/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/kibana.site.ru/privkey.pem;

	location / {
		auth_basic "Restricted Access";
		auth_basic_user_file /etc/nginx/htpasswd.kibana;
		proxy_pass http://localhost:5601;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection 'upgrade';
		proxy_set_header Host $host;
		proxy_cache_bypass $http_upgrade;
	}
}

Create a file for users and passwords:

# htpasswd -c /etc/nginx/htpasswd.kibana kibanauser

After that, you will be prompted to enter a password. Using the above command, we created a file with user and password information kibanauser to restrict access to the web panel of the kibana.

Conclusion

I tried to tell in detail and clearly about the complete configuration of the ELK Stack. The information is mainly gathered in the official documentation. It seems to be such a popular and effective tool, but I hardly saw any articles more than just a default installation.

I did not describe any verified moments in the article, since I considered them inconvenient and did not use it myself. For example, if you refuse logstash and send data from the beats directly to elasticsearch, then at first glance, everything becomes easier. The standard modules beats the output lines themselves, install ready-made visualizations and dashboards in Kibana. You just have to go and see the beauty 🙂 But in fact, everything goes not so beautiful as we would like. Custom configuration is complicated. Changing the fields in the logs leads to more complex settings for entering these changes into the system. All settings of the incoming logs are transferred to each beats, changed in the configs of individual agents. It is not comfortable.

At the same time, when using logstash, you simply send data from all beats to it and manage everything in one place, distribute indexes, change fields, etc. All settings are moved to one place. This is a more convenient approach. Plus, with a heavy load, you can take out the logstash to a separate machine.

I did not said in my article such moments as the creation of visualizations and dashboards in Kibana. Perhaps I will do it separately, since the material is already big, I am tired of writing this article 🙂 We still need to think about the presentation, a lot of pictures need to be understood. There everything is not very clear, so you need to carefully understand.

I also did not consider such a moment. Logstash can receive data directly via syslog. You can, for example, in nginx configure sending logs to syslog, bypassing files and beats. This may be more convenient than the scheme I described. This is especially true for collecting logs from various network devices on which it is impossible to install an agent. Syslog stream can also be parsed on the go using grok. Separately, it is necessary to consider the auto-cleaning of old indices in elasticsearch.

Summing up, I would say that you need to very carefully deal with this system of log storage. To be convenient to use, you need to configure a lot of things. I described only a slightly customized data collection, their visualization is a separate conversation. I am currently studying the system, so I will be glad to any advice, comments, interesting links and everything that will help to look into the topic.

One comment

  1. nice tutorial

Leave a Reply

Your email address will not be published.

Нажимая кнопку "Отправить комментарий" Я даю согласие на обработку персональных данных.