Share your Icinga Environment

Dear all,

we use Icinga and its integrations in many different ways, and we love to hear more about it in your Icinga Camp/Meetup talks. Since we as the people behind Icinga also know that not everyone can do so, we’d like to start a challenge on here.

Share your Icinga environment

Include everything you think is worth sharing, by replying to this topic. Here’s a few pointers on what’s the basics you can add. Be creative, the list isn’t complete nor mandatory.

  • What criteria did you weigh up in deciding to engage Icinga?

  • How many hosts/services do you monitor? (Hint: icinga2 daemon -C)

  • Describe your Icinga Setup roughly (High Available, Satellites, Agents etc.)

  • How do you add new monitoring objects? Describe your configuration workflow roughly.

  • Name your most favourite Icinga Module

  • How do you integrate Icinga with your other Tools?

  • What’s that ONE piece your missing in Icinga?

  • Spice it up, attach some screenshots from your Tactical Overview

Is there anything else you would like to share? Don’t hesitate, add everything you think is interesting!


Share your Icinga Environment! Reward for the first 3 Icinga users taking part: One fancy blue Icinga drink bottle and a free ticket to Icinga Camp Berlin!

Until when?

Take your time with collecting all the details and screenshots, but as always - add them as as soon as possible. To give everyone the chance to step in, like and share, the first round targets the end of March, 2019.

Thanks for sharing!



Hi all,

Since people seem shy to post their Icinga setup, I will to post our small environment and inspire others to do the same.

First of all, we are a publicly funded institution, so our budget is tight. We tried both NewRelic and Datadog, which are fine in what they do but are too pricy for us plus we had some GDPR related issues. Some in the team had previous experience with Nagios and OP5, but we opted for Icinga because we wanted to pair it with our Graylog servers and have an overview of what is happening in our systems plus the interface seemed good enough.

Our environment is small, 124 servers and 1010 services are monitored. We have no need for HA and Satellites for the moment, so a single VM with Icinga and two containers with Grafana and influxDB. We gather most information through network plugins and SSH for load, memory, etc for Linux servers and WMI for windows servers.

We don’t use Director either. We wanted to keep it as simple as possible and went with plaintext files. We manage them with a combination of ansible playbooks mostly for adding/removing new servers and some manual work for adjusting services and variables.

Favourite icinga module is the Grafana module by Carsten Kobke. We love Grafana and praise Carsten daily for the plugin.

As for integrations, we use the graylog plugin to check our Graylog streams and we have written a custom plugin to integrate with Statuscake which checks our services externally. Previously it was also reporting issues to HipChat, but since we moved to Slack we stopped sending messages there due limits in the free version. We are also using dashing interface in a big screen, so we know quite fast when things break.

We would really like to have a proper reporting functionality for all sorts of cases but for the IT team a better way to handle notifications (maybe NoMa2 ?) is our number one request now.

Finally, a big thank you to all who have worked hard to make Icinga a great project, a top-level monitoring tool and fantastic community.



Hello all,

I just found out this, and I’d like to share our “not so small” platform.

We reviewed several monitoring tools a while ago and we decided to migrate from OpenNMS (our old platform) to Icinga because of the integration with other tools and the ease (but powerful) of configuration on Icinga.

As you can see on the screenshot, we’ve got ~5900 hosts in total and ~79k services. Sadly for us, it’s not all green.

Our infrastructure has around 35 satellites (internal and external ones) and 1 master node. We do most of our checks through SNMP, but we also use our own scripts to run some checks (using python, bash, and even Docker containers for specific services). We also have 1 node with Grafana and other with InfluxDB

At the moment, we don’t use Director, so our configuration is done directly on the files. Since there is a big group of people working on the configuration, we manage it through our SCM and pull requests to avoid issues.

Like @gkoutsog, I would say that Grafana module is the favourite for us.

Regarding to integration, we manage Icinga notifications through HipChap. We also integrated it with an internal NOC (w’ve got other tools like Splunk on the NOC as well)

At the moment, we’d like to have a configurable reporting tool that allows us to get metrics and maybe some graph reports.

I’m new on the community forum, so I’ll be checking the documentation and sharing my experience with all of you.



Hi all,

compared to the two other enviroment’s, our one is a little bit smaller :slight_smile:

We also reviewed some monitoring tools, but started with nagios first. We had a visit at the datacenter here in salzburg and we asked what monitoring system they use - they said icinga. So here we are :slight_smile:

We monitor 59 hosts and 803 services.

icinga daemon -C
information/cli: Icinga application loader (version: r2.10.3-1)
information/cli: Loading configuration file(s).
information/ConfigItem: Committing config item(s).
information/ApiListener: My API identity:
information/ConfigItem: Instantiated 803 Services.
information/ConfigItem: Instantiated 1 InfluxdbWriter.
information/ConfigItem: Instantiated 1 IcingaApplication.
information/ConfigItem: Instantiated 59 Hosts.
information/ConfigItem: Instantiated 1 FileLogger.
information/ConfigItem: Instantiated 4 NotificationCommands.
information/ConfigItem: Instantiated 881 Notifications.
information/ConfigItem: Instantiated 1 NotificationComponent.
information/ConfigItem: Instantiated 16 HostGroups.
information/ConfigItem: Instantiated 1 ApiListener.
information/ConfigItem: Instantiated 1 Comment.
information/ConfigItem: Instantiated 1 CheckerComponent.
information/ConfigItem: Instantiated 22 Zones.
information/ConfigItem: Instantiated 20 Endpoints.
information/ConfigItem: Instantiated 2 ApiUsers.
information/ConfigItem: Instantiated 3 Users.
information/ConfigItem: Instantiated 1 IdoMysqlConnection.
information/ConfigItem: Instantiated 230 CheckCommands.
information/ConfigItem: Instantiated 1 UserGroup.
information/ConfigItem: Instantiated 8 ServiceGroups.
information/ConfigItem: Instantiated 3 TimePeriods.
information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
information/cli: Finished validating the configuration file(s).

The master is running on a HPE ProLiant DL380 Gen7 with InfluxDB and Grafana.
Checks are done though the icinga agent, ssh and snmp. Also using some own check plugins written in perl, python and C#.
The configuration is done though .conf files - for version controlling we are using git (gitlab).

In addition to the standard mail notifications we get also notifications though sms for cirtical hosts/services. Using a simple umts stick, with a sim-card and the gammu package for sending.

New monitoring objects get at the moment added manually. We are planning to do this automated via chef for installing/configuring icinga2 agents.

For our employees we made a little system status page, so they can have a look if something isn’t working. Data gets requested via API. At our office we have two monitors to know whats happening in our infrastructure - the first one displays an icinga2 dashboard and the second one grafana dashboards with a playlist.

Like the others said, the grafana module is also the favorite for us. Thanks to @anon66228339!
Using also the x509 module for certs handling, thanks for that.
A build-in reporting tool would be nice, i already read that something is in progress - looking ahead to use it in the near future.

Thanks for reading and a big thanks to all who made icinga a great, maybe even the best monitoring tool - we love it!



here is my enviroment.

monitoring stuff:

  • 180-190 network switches with snmp
  • 20 esxi hosts
  • 12 Firewalls
  • Voice Gateway’s
  • Branch Office Gateway’s / PSTN Breakouts
  • a couple of SAN switches
  • and other infrastructure.
  • and all the VM’s which are required for Infrastructure services…
    including DC’s, Exchange, Backup, Antivir and other infrastrucutre servers/services.

enabled Features / Modules:

  • Director
  • x509
  • vSphere

so a really bug thx to the team from netways which is “Head of Developlent” for the Icinga Features/Modules.

U guys/ and girls as well do a realy good Job. :star_struck:


after reading the entries, I decided to show mine as well :smiley:

Decision for Icinga

I followed the icinga2 development from the early days of the beta, when there was no icingaweb2. After switching companies, the new deparment had many issues related to none existent or spotty monitoring, so the first thing I was tasked with was to fix all the problems which included setting up a monitoring system. Naturally I decided to use Icinga2 as it seemed the most powerful and accessible tool with a nice community.

Icinga Setup

Our setup is rather small, but it only encompasses our department of 14 people, nonetheless we have currently around 59 VMs that are dedicated to our department, which all need to be controlled and monitored.


Adding Hosts to Icinga2

Adding hosts is done automatically, when they are setup. We run ansible playbooks to fully configure our hosts, so it was a natural decision to also add them to icinga via a playbook, this is realised via a python script that talks to the director REST-API and adds them with configuration values for hostgroups, processes to survey etc.

Modifying Icinga Objects in the runtime

We have the need to modify Objects, without engaging the director. So I teamed up with @bodsch to port his API from ruby to python. you can find the API here: GitHub Repo it is currently rather stale, as I do not need any more functions.

Favorite Module

This is a hard one, as I really like the director and the grafana module and can’t decided which one has brought me more joy.


Icinga is currently integrated with:

  • Ansible
  • FreeIPA for authentication and soon Host Import

Missing Feature

I would really like a easy to use graphical interface for notifications, yes I’m looking at you NoMa :smiley:


Overview :-

We have hybrid cloud landscape which is combination of AWS Cloud & On Prem Infrastructure . As SRE our focus is to ensure 100% availability of all the OS |App | DB | Network| Middleware services. We had face lot of challenges in past due to gap in monitoring coverage. Main agenda behind implementing ICINGA is to ensure 100% monitoring coverage using continuous monitoring.

Currently our entire infrastructure is getting monitored from ICINGA which consist of ~1300 servers & ~24000 services . ICINGA master is in High availability which is connecting to 08 satellites.

All the new monitoring is happening via continuous monitoring automation. This automation ensure that all new discovered hosts will get onboarded along with respective services.(e.g. automatic monitoring onboarding OS | App | DB | MW as per technology)

Key Integration

  • ICINGA – influxDB – Grafana : OS Metrics Dashboards | Component Dashboards (Redis | Zookeepr | NGINX | JVM etc)

  • ICINGA – OpsGenie : Auto escalation for Critical & Warning Alerts

  • Self Healing : Self Healing actions post critical alerts (automatic disk clearance | process restart) using Jenkins integration

We are looking forward to see out of box reporting capabilities from ICINGA . This is our primary monitoring tool & most of our incidents are getting captured proactively.


What criteria did you weigh up in deciding to engage Icinga?
About three years ago we started evaluating on how to replace our existing monitoring setup (icinga1 (+centreon as config ui), that we used for us and our customers. We had some 3-4 different solution with which we (tried to) built a testing environment.
Icinga2 made the race because of the rewritten code and not being a nagios fork anymore. Also it was easy to set up, offered a modern looking interface (with many integrated modules), a out-of-the-box distributed/HA-option and (for us) the biggest selling point was the announcement of the Icinga Director.

How many hosts/services do you monitor? (Hint: icinga2 daemon -C )

icinga2 daemon -C
# icinga2 daemon -C
[2019-03-12 08:30:56 +0100] information/cli: Icinga application loader (version: r2.10.3-1)
[2019-03-12 08:30:56 +0100] information/cli: Loading configuration file(s).
[2019-03-12 08:30:56 +0100] information/ConfigItem: Committing config item(s).
[2019-03-12 08:30:56 +0100] information/ApiListener: My API identity: :)
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1822 Services.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 306 Hosts.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 FileLogger.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 195 Dependencies.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 15 NotificationCommands.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 2412 Notifications.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 53 HostGroups.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 ApiListener.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 GraphiteWriter.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 7 Zones.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 6 Endpoints.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 3 ApiUsers.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 12 Users.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 258 CheckCommands.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 7 ServiceGroups.
[2019-03-12 08:30:58 +0100] information/ConfigItem: Instantiated 14 TimePeriods.

Describe your Icinga Setup roughly (High Available, Satellites, Agents etc.)
This monitoring system is just for our internal IT infrastructure and monitors our Windows servers (Exchange, SQL, System Center, ADFS, SOFS), some Linux servers, Cisco switches and ASAs, NetApps, Cisco UC equipment and various other hosts

  • two Master servers as a HA cluster in our headquarter, where most of our internal IT is located
  • three satellites in our bigger remote locations, that monitor the local internal IT infrastructure
    – the smaller locations get monitored via the Master servers
  • mariaDB on a separate server
  • graphite on a separate server

How do you add new monitoring objects? Describe your configuration workflow roughly.
There is no real automation going on. Windows servers get imported from our AD via Import/Sync from the Director. Anything else is added manually (though this happens rarely, as there is no big movement in out IT infrastructure).

Name your most favourite Icinga Module
:heart_eyes:Icinga Director:heart_eyes:

What’s that ONE piece your missing in Icinga?
The reporting module :smiley:

All in all I really like icinga2. It is nice/easy to set up, runs well, looks good, is highly modular/customizable, has a very good configuration interface :slight_smile: and a great community!
Thanks for developing it :+1:



thanks everyone for sharing the many details of your Icinga environment, much appreciated :heart:

Also, your kind words are a big motivation for developing and enhancing Icinga even further.

We’ve counted the seven of you and decided to send the Icinga bottle to all of you. Everyone has been contacted via PM and packages are already on the way.


PS: This topic doesn’t end here. Please continue sharing your environments with a chance of getting more Icinga goodies.


What criteria did you weigh up in deciding to engage Icinga?

Before I started at our company, there was a point where they had switched from Nagios to Icinga 1.x due to cultural changes (and everything ported directly over). They asked if I’d be interested in overhauling the checks and thresholds, I suggested that we needed to look at a newer system. After taking a brief look at Icinga2 and Nagios XI, I just loved the object oriented config syntax and saw it as an easy solution to scale out.

How many hosts/services do you monitor? (Hint: icinga2 daemon -C )

Always growing:

[2019-05-22 07:44:24 -0400] information/ConfigItem: Instantiated 47110 Services.
[2019-05-22 07:44:24 -0400] information/ConfigItem: Instantiated 2328 Hosts.
[2019-05-22 07:44:24 -0400] information/ConfigItem: Instantiated 1763 Zones.
[2019-05-22 07:44:24 -0400] information/ConfigItem: Instantiated 1765 Endpoints.
[2019-05-22 07:44:24 -0400] information/ConfigItem: Instantiated 303 CheckCommands.

Describe your Icinga Setup roughly (High Available, Satellites, Agents etc.)

2 masters in high-availability which also monitor the client nodes in the same data center they run in. Past that, 2 satellites in all of our other datacenters. Icingaweb is running on both with a vip we can failover if icinga-master1 is going to be dead for a while. In the freak event our main datacenter goes out entirely, I have a cronjob on the icinga satellite in our Vegas DC that just makes sure it can access the Icinga rest API remotely and will leverage pagerduty directly.

I went with postgresql for the database which is using logical replication and a hot standby. Since postgres doesn’t have an automated failover solution built in, I wrote an event handler where Icinga will fail over its own database if it can’t connect to it, and that’s the funniest thing in the world to me.

Our carbon cache daemons and relay run on one of the database servers, but we plan to get bare metal to replace these so we can have our whisper database be redundant as well (and it’s growing like the blob).

How do you add new monitoring objects? Describe your configuration workflow roughly.

I’ve written most of our config by hand, with the exception of new host/zone/endpoint objects (I would have lost my mind). I install and generate the basic configuration using a python script I wrote, but I should probably quit being a crazy person and just use Ansible. Templates and services are all a labor of love and at this point I’d be afraid of Director touching my code.

How do you integrate Icinga with your other Tools?

The custom checks we’ve been writing these days are super specific to us, so I’ve only put one hobby project on the exchange so far. A lot of our event handlers we’re writing use a Slack plugin my coworker cranked out that announces to the team what is broken and what it’s doing to fix it.

What’s that ONE piece your missing in Icinga?

The ability to run event commands on different endpoints than the check commands ran from.

There are super specific things we need to be able to do sometimes that Icingaweb isn’t really equipped for, so not being php developer, I was stoked to see that taking a couple minutes to hack a configuration.php file will iframe and create relevant links for external applications. Building a custom toolbox in Django now since we’re mostly python guys. I really like the API.

Anyway, I just happen to be in Atlanta replacing an old Centerity setup with Icinga the week of the monitoring conference, so I’ll be seeing some of you tomorrow.


What criteria did you weigh up in deciding to engage Icinga?

We had Logic Monitor before, and it was not helping us with what we needed. Not to mentioned it was not free, and hard to set up and automate.
We then decided to try out Icinga, and we were quite blown away by this amazing monitoring tool.

How many hosts/services do you monitor? (Hint: icinga2 daemon -C )


Describe your Icinga Setup roughly (High Available, Satellites, Agents etc.)

We use Icinga to monitor the VPN connections to our service providers, as well as basic Infrastructure services (disk, IO, load, api, etc).
I am using PostgreSQL for the DB with a streaming replication slave for backup purposes.

  • 1x icinga master in New Zealand. Icingaweb and Postgres are also running here.
  • 2x Icinga satellites -
    • 1x in Australia
    • 1x in AWS Sydney
  • graphite running on the master

We have also integration with PagerDuty where we have our on-call escalation policies.

How do you add new monitoring objects? Describe your configuration workflow roughly.
In AWS, I am using CloudFormation to deploy new instances. So, in that case, I am automating the icinga2 node setup using CFN.
Apart from that, there is no automation as we have a small environment and we deploy new instances not that often.

Name your most favorite Icinga Module

What’s that ONE piece your missing in Icinga?
The reporting module :slight_smile:

I would like to thank the Icinga2 community. Every time I needed help, I got help. We are really happy with Icinga and its performance, as well as the DB size which does not grow as much as Zabbix.

Thanks guys!


What criteria did you weigh up in deciding to engage Icinga?

We startet with nagios and switched 2017 from icinga1 to icinga2.

How many hosts/services do you monitor?

Describe your Icinga Setup roughly

We have a HA-Cluster:

  • 3 master (2 for icinga2, 1 for icingaweb2, graphite, grafana)
  • 34 satellites

How do you add new monitoring objects?

manually on command line

Name your most favourite Icinga Module

Grafana in Icingaweb2

What’s that ONE piece your missing in Icinga?



What criteria did you weigh up in deciding to engage Icinga?
I worked with Nagios (core) since 2004. I started at a new company on May 2019 and there was a suggestion to try Icinga2 instead.

How many hosts/services do you monitor? (Hint: icinga2 daemon -C)

[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 8147 Services.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 InfluxdbWriter.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 6272 Hosts.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 2 EventCommands.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 FileLogger.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 2 NotificationCommands.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 25898 Notifications.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 88 HostGroups.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 2 Downtimes.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 9 Comments.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 3 Zones.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 2 Endpoints.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 8 ApiUsers.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 20 Users.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 251 CheckCommands.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 5 UserGroups.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 14 ServiceGroups.
[2019-05-24 08:39:35 +0200] information/ConfigItem: Instantiated 6 TimePeriods.

Describe your Icinga Setup roughly (High Available, Satellites, Agents etc.)
Started with only one server, we switched to one master with one satellite. Agents (NSCLient, NRPE) on most clients.

How do you add new monitoring objects? Describe your configuration workflow roughly.
Because our monitoring is location based, I’ve created an Excel sheet which gives me all definitions for copy/paste to the config files based on a few parameters we fill in the Excel sheer

Name your most favourite Icinga Module
Map, Grafana

How do you integrate Icinga with your other Tools?
We use Dashing to get a nice overview on the central monitor in the IT department.

What’s that ONE piece your missing in Icinga?
better reporting


I’m having one monitor at work showing the grid view on the left and unhandled problems on the right I’d like to share with you.

To spot new problems very quickly, I’ve desaturated the box icons in the grid that show OK or handled problem states. Having the “old” items almost white, the new problems can be discovered very easily. Example:

For the right panel, I’ve created a new dashboard pane (“Unhandled”) that shows unhandled host and service problems only:

Maybe someone finds it useful. By the way, the grid should fit into the left panel size :slightly_smiling_face:

Desatured item boxes

I’m using the User JavaScript and CSS extension for Chrome browser so I don’t have to deal with different themes. There are other extensions, too, so you don’t have to use exactly this one.

Thanks to the designer, the grid items use CSS classes so it’s very easy to overwrite the colors:

.icinga-module.module-monitoring .service-grid-link.state-ok {
	background-color: #dfd;

.icinga-module.module-monitoring .service-grid-link.state-critical.handled {
	background-color: #fee;

.icinga-module.module-monitoring .service-grid-link.state-unknown.handled {
	background-color: #edf;

.icinga-module.module-monitoring .service-grid-link.state-warning.handled {
	background-color: #efe;
Browser view / dashboard pane

Using this URL for fullscreen mode:!/icingaweb2/dashboard?pane=Unhandled

Dashboard pane “Unhandled” having 2 dashlets:

  • Unhandled services:
  • Unhandled hosts:

I’ve recreated my config using the official Icinga2 standalone Vagrant box.

Kind regards,

1 Like