Monitor the state of your infra with dashboards

ELK is perfect for small business who need to monitor the logs of their infra. Coming from a Splunk setup, I can say that even if ELK is lacking few minor functionalities, it will convince 90% of companies with its price (free! ;) and community support & content!

1. What we use

• ELK stack (Elasticsearch, Logstash, Kibana). This is a database, a collector/log parser, and a GUI to visualize the data. Your actual infra servers will then be able to send logs via syslog to ELK, or we can use Filebeat to parse local server log files and send them to the stack. A big release is out now, with the setup below you can try the powerful Kibana version 5!
• Docker-Compose: to configure and easily deploy

2. Show me what you got

Here is some final production dashboards. All of this could be running on one big screen.

2.1 Infra

We can monitor here all logs with errors, by VM or container. ON 24 hours we can detect app failing, database errors, incoming high traffic, node stop syncing.

2.2 SSH access attempts and server performances

All SSH connections with success or failure are displayed, impossible then to miss any attacks (hopefully ;-).

Below is the performance of servers. Basically, we run each 1 minute a "top" on all servers to collect metrics. We display only high CPU or disk of the top 6 servers, which is very powerful because in half of a screen we can monitor 20+ servers.
Here we can see that VM ethworker is swapping badly... :-O

3. Deploy

3.1 Get the code

Follow the github to get a full ELK stack running.

3.2 CAdvisor logs to ELK

If you are interested to forward container performance logs from CAdvisor to ELK, please have a look at this repo (careful: older version of ELK).

Thank you for reading :-) See you in the next post!