I want to setup a master/satellite cluster to monitor 1000 devices each having 10 service on average with a polling interval of 300 seconds. Most of the services check are through SNMP while few are based on ssh.
Features that need to be enabled in icinga are DB-IDO,Notification and Graphite
Can you please advice what hardware requirements(CPU,RAM,Disk) for Icinga master,satellite,mysql & graphite do i need to efficiently monitor my devices.
Hello
Asking such a question is difficult to answer but iβd try to give an estimation:
As you are planning to distribute the load to satellites, the load on any individual machine will not be too much so for the Master & Satellites iβd recommend a 2 Core machine with 8 GB Ram and 40 GB HD as the minimum.
If you plan to run the Graphite on the Master servers, iβd increase the HD size to min 60GB.
For the Mysql server iβd say 4 Core, 16 GB Ram and as big a HD as you can as the DB tend to grow, without any limit, unless you as the DBA perform table pruning.
I have no experience with such big setups, yet (sadly).
Just some figures from the biggest one I have set up until now:
Monitoring ~2100 hosts and ~3200 services.
HW setup:
Master-Cluster: 4vCPU, 8GB RAM, 60GB HDD each.
Satellite-Cluster: 4vCPU, 8GB RAM, 30GB HDD each.
DB: 4vCPU, 8GB RAM, 60GB HDD
Stats: Load:
Master
β is bored, load around 0.6 - 1.1
Satellites (doing the most checks ~1650 hosts ~2900 services)
β still bored 0.7 - 1.8 mostly
DB
β around 1.0 - 1.3
Memory:
Master
β master1 ~60%, master2 ~10% (assuming this is due to master1 being used as the main webinterface)
Satellites (doing the most checks ~1650 hosts ~2900 services)
β ~10%
DB
β ~20%
Disk-/var:
Master
β around 9 - 11 GB
DB
β around 6GB
The setup is running for about a year now.
As you plan to run mostly SNMP checks keep in mind that checks that create temporary cache files (like some check from check_nwc_health do) can have a considerable performance impact.
Iβd say a VM with 2 cores, 4gb memory and 50gb disk space should be enough.
Cores and memory could easily be increased if needed, if you configure this disk to be a LVM the space of the partitions can also be easily increased, in case you run out of space.
In case of using satellites, where they will do most of the controls, do you recommend VM sizing with the same specifications?
Let me explain better (for example) : 1 master = VM hosted on AWS configured with 2 cores, 4gb memory and 50gb. 3 satellites = Each one configured with 2 cores, 4gb memory and 50gb. Could this be enough?
Please note: each satellite will monitor about 100 hosts (about 500 monitored services)