Skip to main content

ELK Stack

ELK is the tool for the feature of Kubernetes real-time nodes monitoring tools. You can simply start ELK by running the following command:

if [ ! -f .env ]; then cp -r .env.default .env; fi
if [ ! -d config ]; then cp -r config.example config; fi
if [ ! -f docker-compose.yml ]; then cp -r docker-compose.prd.yml docker-compose.yml; fi
make elk

Ports​

By default, the stack exposes the following ports:

  • 5044: Logstash Beats input
  • 50000: Logstash TCP input
  • 9600: Logstash monitoring API
  • 9200: Elasticsearch HTTP
  • 9300: Elasticsearch TCP transport
  • 5601: Kibana

Features​

  • Kubernetes Nodes Monitoring (Must be reachable from the server)
  • Cloud Platform Logs (in progress)

Setup prerequisite​

You should install the following prerequisite:

  • docker (Tested version 18.09.7)
  • docker-compose (Tested version: 1.25.4, build 8d51620a)

Setup ELK​

  • Run docker compose --profile elk up -d or make elk
  • Please update the dashboard links in frontend. They are referenced to 2 dashboard "[Metricbeat System] Overview ECS" and "[Metricbeat Kubernetes] Overview ECS" (You can find the dashboard in "Analytics > Dashboards" from the Kibana Sidebar)

Elasticsearch Engine Configure​

Kibana URL: http://127.0.0.1:5601/

Goto "Stack Management" ==> "Index Lifecycle Policies" ==> click on metricbeat

Change the Index Lifecycle Policies of metricbeat to

  • Hot phase ==> Maximum primary shard size=100 megabytes, Maximum age=1hour, Maximum index size=100megabytes
  • Delete phase ==> Move data into phase when:2 hours old

The password of Elasticsearch​

The password is stored in the .env file.

Account: elastic Password: configured using the ELASTIC_PASSWORD environment variable in the .env file

Tasks​

  • Include Beats for Cloud Platform Logs

API Reference​

See Elasticsearch API Reference for more details.

Reference​