Excludes out of import source or sync rule

Hi there,
at the moment I try to setup import sources and sync rules in order to evaluate a new monitring landscape in our company.
I want to a achive that we can import from different sources to build up the monitoring spots.
To get there I need a lot of information - unfortunately these information are stored in multiple places.

Now - my problem is that I get my host / ip information from our old ldap (icinga1 / lconf) but for example when I try to source the os of these hosts I have to query a sql database with much more information I want.
To query the sql database isn’t a problem at all, but in this database there are hosts I do not want to monitor (external systems I can’t reach, dev-systems that should not be part of >this< monitoring, …).

Now I have set up the sql query as import source - no problem.
I can also import the os to all my hosts - no problem as well.
My problem is that the sync rule will also create new hosts out of the data it gets from the import source - but I dont want to add new hosts that the director doesn’t already know of.

Is there anything I can do to compare if this row of the import source that I (not me - the sync rule) am processing is already part of the known hosts and if not to blacklist this row?
Or is there anything I can do to exclude them while importing?

To remember:
The expalined problem with the os is just an examlpe - I have multiple sources with different information.
As long as the new source has less information than the director - everything is fine.
For the other case I need a solution - and I realy need a solution and not just a workarround because I want to automate the hole monitoring setup in the future with just fetching the information I need and let the apply rules take care of the rest :slight_smile:

BR and thanks in advance,
Chris

Tbh I got quite confused while reading :smiley:

Normally you have the option to merge, replace or ignore the information from the sync rule to existing objects.
What I do with importing and syncing hosts from our AD is, that i set the sync rule to “merge”. This way an already existing host object (has to have the same name!) will only be edited if some of the fields (e.g. host address) changes.

Can you show some screenshots of your import and sync rules? Maybe that clears things up :slight_smile:

1 Like

Hi log1c,

first of all - thanks for your reply :slight_smile:
But I guess you got me wrong … I try to explain a bit better - hope it will clear things out ^^

Merging (new) information into hosts is no problem at all :wink:
My problem is to >not< adding new hosts if the host does not exist.

Example:

  1. “Base-Import” (where I get all hosts + ip that I need to monitor):
    => host-a - ip 172.0.0.1
    => host-b - ip 172.0.0.2
    This will add the two hosts with a chosen host-template (settings like the correct cluster zone and some basic settings), adding the ip address to it and so on.

  2. “OS-Import” (CMDB database where (for example) the os is stored for every host - not only the hosts I want to monitor):
    => host-a - os CentOS7
    => host-b - os CentOS6
    => host-c - os RHEL7
    Now the os information will be added for host “a” and “b”.
    But (and this is my big problem) host “c” will be added to the director - and I dont want it to be in there.
    I just want to update the hosts that the direcor knows of - not to add any new hosts.

I need one source that is importing my hosts to monitor - and many other sources to just expand the configuration with specific information.
The expanding sources are never allowed to add new hosts.
Is there any way to achieve this with the director?

BR
Chris

Ok, got it.

The only idea I have:

  • Can you filter out the not-wanted hosts in some way?
    – On the SQL query level?
    – the sync rule also allows for filtering with an expression, that is executed on the imported objects.
1 Like

Yeah, that was my thought, too …
But can I access somehow the existing hosts (known by the director) in my sql statement or in these filters?
I already tried yesterday to set filters like “hostname=host.name”, “hostname=$host.name$” or things like this (first colum is the name out of the import source).

No luck here :frowning:

Seting filters will work if i type the actual hostnames, but I need this to be dynamic.
Typing in all 1500 Servers and adding new ones to the filters every time someone wants to have a new one monitored won’t work for me :wink:

Afaik these filters in the sync rule only work on the objects from the import rule, not the Director database itself.
If you can’t filter out the not-wanted hosts in your import rule, so that they are not even there, when the sync rule accesses the objects, I’m out of ideas.

Damn, but ok - thanks anyway for your input :slight_smile:

Maybe someone else got an idea.

Otherwise I have to do these steps by my self and first import everything by scripts into a new “facts” database, merge everything there and import this database from the director …
I realy hoped that I can avoid the effort building this ^^

Hi everyone,

in case there is anyone out there who got the same requirements I do (regarding the import sources and excludes) - I wrote myself a sql-statement that will delete every row out of the imported data the director doen’t already know:

delete from imported_row_property where row_checksum in (select checksum from imported_row where checksum in (select row_checksum from imported_rowset_row where rowset_checksum = (select rowset_checksum from import_run where source_id = 2 and start_time = (select max(start_time) from import_run where source_id = 2) and succeeded = 'y')) and object_name not in (select distinct object_name from icinga_host where object_type = 'object'));

Unfortunately after this the import source won’t realy work anymore (because it compares the checksums and not the actual content) - so you need to delete every old data to get it up an running again.

My Import / Sync looks like this now:

  1. Import via importsource:
    icingacli director importsource run --id=2

  2. Delete all hosts the director doen’t know:
    delete from imported_row_property where row_checksum in (select checksum from imported_row where checksum in (select row_checksum from imported_rowset_row where rowset_checksum = (select rowset_checksum from import_run where source_id = 2 and start_time = (select max(start_time) from import_run where source_id = 2) and succeeded = 'y')) and object_name not in (select distinct object_name from icinga_host where object_type = 'object'));

  3. Sync via syncrule:
    icingacli director syncrule run --id=2

  4. Remove DB entries regarding the import source
    4.1. delete from import_run where source_id = 2;
    4.2. delete from imported_rowset where checksum not in (select distinct rowset_checksum from import_run);
    4.3. delete from imported_property where checksum not in (select distinct property_checksum from imported_row_property);
    4.4. delete from imported_row where checksum not in (select distinct row_checksum from imported_row_property);

  5. Deploy configuration on icinga-master
    icingacli director config deploy

It is certainly not the best thing you can do to edit the database itself - but at the moment it works for me and it is the only way I could came up with to do the job like I want it to be done.

Btw.: Thanks for the posibility to start import / syncy and deployments with the icingacli - it realy makes things much easier :slight_smile:

BR,
Chris

2 Likes