Monitor the state of your infra with dashboards

ELK is perfect for small business who need to monitor the logs of their infra. Coming from a Splunk setup, I can say that even if ELK is lacking few minor functionalities, it will convince 90% of companies with it price (free! ;) and community support & content!

1. What we use

• ELK stack (Elasticsearch, Logstash, Kibana). This is just a database, a collector/log parser, and a GUI to visualize the data. Your actual infra servers will then be able to send logs via syslog to ELK, or we can use filebeat to parse local server log file and send to the stack. A big release is out now, with the setup below you can try the powerfull kibana version 5!
• docker-compose: to configure and deploy quickly and easily

2. Show me what you got

Here is some final production dashboards. All of this could be running on one big screen.

2.1 Infra

We can monitor here all logs with errors, by vm/container. ON 24 hours we can detect app failing, database errors, incoming high traffic, node stop syncing.

2.2 Ssh access and servers performance

All ssh success and failures are displayed, impossible then to miss any attack (hopefully ;-).

Below is the performance of servers, basically we run each 1 min a "top" on all servers to collect metrics. We display only the top 6 servers high CPU or disk, which is very powerfull because in half of a screen we can monitor 20+ servers.
Here we can see that vm ethworker is swapping badly... :-O

3. Deploy

3.1 Get the code

Follow the github doc to get a full ELK stack running:
https://github.com/gregbkr/elk-dashboard-docker

I will add later commands to easily visualise via ELK: VM performances via a simple "Top", and the parsing for ssh security.

3.2 Cadvisor logs to ELK

If you are interested to forward container performance logs from cadvisor to elk, please have a look at this doc. (carefull, old version of elk)
https://github.com/gregbkr/docker-elk-cadvisor-dashboards