Client via REST API does not appear icingaweb2

Icinga 2
1

Hi.

I have a scenario High-Availability Master with Clients.
I’m provisioning clients via REST API in my cluster. Object host and object services are located in /var/lib/icinga2/api/packages/ip-172-31-36-127/249f6fd5-761e-4074-891f-57b3ec4699ab/zones.d/master and object zone and endpoint are located in /var/lib/icinga2/api/packages/ip-172-31-36-127/249f6fd5-761e-4074-891f-57b3ec4699ab/conf.d. But the host doesn’t appear in icingaweb2.

I’ve watching logs too, but I have no found errors.

the curl command I send is:
curl -k -s -u ${ICINGA2_API_USER}:${ICINGA2_API_PASS} -H 'Accept: application/json' -X POST \ -d '{ "files": { "zones.d/master/'${ICINGA2_ENDPOINT}'_host.conf": "object Host \"'${ICINGA2_ENDPOINT_FQDN}'\" { check_command = \"ssh\", address = \"'${ICINGA2_ENDPOINT_IP}'\", zone = \"'${ICINGA2_ENDPOINT_FQDN}'\", display_name = \"'${ICINGA2_ENDPOINT}'\" }", "conf.d/'${ICINGA2_ENDPOINT}'_endpoint.conf": "object Endpoint \"'${ICINGA2_ENDPOINT_FQDN}'\" { host = \"'${ICINGA2_ENDPOINT_IP}'\" }", "conf.d/'${ICINGA2_ENDPOINT}'_zone.conf": "object Zone \"'${ICINGA2_ENDPOINT_FQDN}'\" { endpoints = [\"'${ICINGA2_ENDPOINT_FQDN}'\"], parent = \"master\" }", "zones.d/master/service_dev_check_load.conf": "object Service \"check_load\" { import \"generic-service\", host_name = \"'${ICINGA2_ENDPOINT_FQDN}'\", check_command = \"load\", command_endpoint = \"'${ICINGA2_ENDPOINT_FQDN}'\" }" } }' \ 'https://'${ICINGA2_MASTER1_IP}':5665/v1/config/stages/'${ICINGA2_ENDPOINT}'

My environment details:

  • icinga2 version: r2.9.1-1
  • Ubuntu 16.04.6 LTS

What can I do wrong?

Thanks in advanced

2

You’ll need to fetch the startup.log from the deployed stage in that package. Possibly it contains configuration errors.

Cheers,
Michael

3

That is the question. Before I had opened the topic, I reviewed this log and didn’t found error. Here it is:

root@ip-172-31-36-127:~# curl -k -s -u ${ICINGA2_API_USER}:${ICINGA2_API_PASS} 'https://18.196.217.24:5665/v1/config/files/ip-172-31-36-127/249f6fd5-761e-4074-891f-57b3ec4699ab/startup.log' [2019-03-27 14:04:42 +0000] information/cli: Icinga application loader (version: r2.9.1-1) [2019-03-27 14:04:42 +0000] information/cli: Loading configuration file(s). [2019-03-27 14:04:42 +0000] information/ConfigItem: Committing config item(s). [2019-03-27 14:04:42 +0000] information/ApiListener: My API identity: ip-172-31-14-0.eu-central-1.compute.internal [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 3 Services. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 3 ServiceGroups. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 3 HostGroups. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 FileLogger. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 NotificationComponent. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 2 NotificationCommands. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 IcingaApplication. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 4 Hosts. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 ApiListener. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 CheckerComponent. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 4 Zones. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 ExternalCommandListener. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 4 Endpoints. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 ApiUser. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 UserGroup. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 213 CheckCommands. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 3 TimePeriods. [2019-03-27 14:04:42 +0000] information/ConfigItem: Instantiated 1 User. [2019-03-27 14:04:42 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars' [2019-03-27 14:04:42 +0000] information/cli: Finished validating the configuration file(s).

What can I do?

Regards
Jesús

4

Hi,

is the service object available via REST API at /v1/objects/services?

Cheers,
Michael

5

I think so.

curl -k -s -u root:icinga 'https://localhost:5665/v1/objects/services' | grep load {"results":[{"attrs":{"__name":"ip-172-31-14-0.eu-central-1.compute.internal!cluster","acknowledgement":0.0,"acknowledgement_expiry":0.0,"action_url":"","active":true,"check_attempt":1.0,"check_command":"cluster","check_interval":5.0,"check_period":"","check_timeout":null,"command_endpoint":"","display_name":"cluster","downtime_depth":0.0,"enable_active_checks":true,"enable_event_handler":true,"enable_flapping":false,"enable_notifications":true,"enable_passive_checks":true,"enable_perfdata":true,"event_command":"","flapping":false,"flapping_current":0.0,"flapping_last_change":0.0,"flapping_threshold":0.0,"flapping_threshold_high":30.0,"flapping_threshold_low":25.0,"force_next_check":false,"force_next_notification":false,"groups":[],"ha_mode":0.0,"host_name":"ip-172-31-14-0.eu-central-1.compute.internal","icon_image":"","icon_image_alt":"","last_check":1553764133.1204199791,"last_check_result":{"active":true,"check_source":"ip-172-31-29-56.eu-central-1.compute.internal","command":null,"execution_end":1553764133.1204199791,"execution_start":1553764133.1201550961,"exit_status":0.0,"output":"Icinga 2 Cluster OK: 1 endpoints are connected.\n(ip-172-31-14-0.eu-central-1.compute.internal)","performance_data":[{"counter":false,"crit":null,"label":"api_num_conn_endpoints","max":null,"min":null,"type":"PerfdataValue","unit":"","value":1.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_endpoints","max":null,"min":null,"type":"PerfdataValue","unit":"","value":1.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_http_clients","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_clients","max":null,"min":null,"type":"PerfdataValue","unit":"","value":1.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_relay_queue_item_rate","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.55000000000000004441,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_relay_queue_items","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_sync_queue_item_rate","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_sync_queue_items","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_work_queue_count","max":null,"min":null,"type":"PerfdataValue","unit":"","value":2.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_work_queue_item_rate","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.48333333333333328152,"warn":null},{"counter":false,"crit":null,"label":"api_num_json_rpc_work_queue_items","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"api_num_not_conn_endpoints","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"checkercomponent_checker_idle","max":null,"min":null,"type":"PerfdataValue","unit":"","value":1.0,"warn":null},{"counter":false,"crit":null,"label":"checkercomponent_checker_pending","max":null,"min":null,"type":"PerfdataValue","unit":"","value":1.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_queries_rate","max":null,"min":null,"type":"PerfdataValue","unit":"","value":3.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_queries_1min","max":null,"min":null,"type":"PerfdataValue","unit":"","value":180.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_queries_5mins","max":null,"min":null,"type":"PerfdataValue","unit":"","value":901.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_queries_15mins","max":null,"min":null,"type":"PerfdataValue","unit":"","value":2705.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_query_queue_items","max":null,"min":null,"type":"PerfdataValue","unit":"","value":0.0,"warn":null},{"counter":false,"crit":null,"label":"idomysqlconnection_ido-mysql_query_queue_item_rate","max":null,"min":null,"type":"PerfdataValue","unit":"","value":3.0666666666666668739,"warn":null}],"schedule_end":1553764133.1204199791,"schedule_start":1553764133.1199998856,"state":0.0,"ttl":0.0,"type":"CheckResult","vars_after":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0},"vars_before":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0}},"last_hard_state":0.0,"last_hard_state_change":1553531860.221681118,"last_reachable":true,"last_state":0.0,"last_state_change":1553531860.221681118,"last_state_critical":1553531854.0804870129,"last_state_ok":1553764133.1219758987,"last_state_type":1.0,"last_state_unknown":0.0,"last_state_unreachable":0.0,"last_state_warning":0.0,"max_check_attempts":3.0,"name":"cluster","next_check":1553764137.49198699,"notes":"","notes_url":"","original_attributes":null,"package":"_etc","paused":true,"retry_interval":1.0,"severity":8.0,"source_location":{"first_column":0.0,"first_line":1.0,"last_column":23.0,"last_line":1.0,"path":"/etc/icinga2/zones.d/master/cluster-master.conf"},"state":0.0,"state_type":1.0,"templates":["cluster"],"type":"Service","vars":null,"version":0.0,"volatile":false,"zone":"master"},"joins":{},"meta":{},"name":"ip-172-31-14-0.eu-central-1.compute.internal!cluster","type":"Service"},{"attrs":{"__name":"ip-172-31-37-252.eu-central-1.compute.internal!check_load","acknowledgement":0.0,"acknowledgement_expiry":0.0,"action_url":"","active":true,"check_attempt":1.0,"check_command":"load","check_interval":60.0,"check_period":"","check_timeout":null,"command_endpoint":"ip-172-31-37-252.eu-central-1.compute.internal","display_name":"check_load","downtime_depth":0.0,"enable_active_checks":true,"enable_event_handler":true,"enable_flapping":false,"enable_notifications":true,"enable_passive_checks":true,"enable_perfdata":true,"event_command":"","flapping":false,"flapping_current":0.0,"flapping_last_change":0.0,"flapping_threshold":0.0,"flapping_threshold_high":30.0,"flapping_threshold_low":25.0,"force_next_check":false,"force_next_notification":false,"groups":[],"ha_mode":0.0,"host_name":"ip-172-31-37-252.eu-central-1.compute.internal","icon_image":"","icon_image_alt":"","last_check":1553764135.6605529785,"last_check_result":{"active":true,"check_source":"ip-172-31-37-252.eu-central-1.compute.internal","command":["/usr/lib/nagios/plugins/check_load","-c","10,6,4","-w","5,4,3"],"execution_end":1553764135.6605110168,"execution_start":1553764135.6579940319,"exit_status":0.0,"output":"OK - load average: 0.00, 0.00, 0.00","performance_data":["load1=0.000;5.000;10.000;0;","load5=0.000;4.000;6.000;0;","load15=0.000;3.000;4.000;0;"],"schedule_end":1553764135.6605529785,"schedule_start":1553764135.6605529785,"state":0.0,"ttl":0.0,"type":"CheckResult","vars_after":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0},"vars_before":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0}},"last_hard_state":0.0,"last_hard_state_change":1553757162.0596940517,"last_reachable":true,"last_state":0.0,"last_state_change":1553757162.0596940517,"last_state_critical":0.0,"last_state_ok":1553764135.654640913,"last_state_type":1.0,"last_state_unknown":0.0,"last_state_unreachable":0.0,"last_state_warning":0.0,"max_check_attempts":5.0,"name":"check_load","next_check":1553764195.6500000954,"notes":"","notes_url":"","original_attributes":null,"package":"ip-172-31-37-252","paused":false,"retry_interval":30.0,"severity":8.0,"source_location":{"first_column":0.0,"first_line":1.0,"last_column":26.0,"last_line":1.0,"path":"/var/lib/icinga2/api/packages/ip-172-31-37-252/27b1e443-43a0-409d-bef7-2da9bd913456/zones.d/master/service_dev_check_load.conf"},"state":0.0,"state_type":1.0,"templates":["check_load","generic-service"],"type":"Service","vars":null,"version":0.0,"volatile":false,"zone":"master"},"joins":{},"meta":{},"name":"ip-172-31-37-252.eu-central-1.compute.internal!check_load","type":"Service"}]}

:frowning:

6

Well. If i stop service icinga2 in master2, the host and services appear. Then, I suppose the problem is the sync between masters. But i don’t understand why they don’t syncronize.

7

I’vd found the problem, I think. Service and host objects sychronize between master1 and master2. But endopoint and zone don’t.

Then, my doubt is. Do I have to configure endpoint and zone objects in each master separately or is there some way to configure in master 1 and sync to master2?

Thanks for your help.

8

How did you come to that conclusion?

9

First of all, sorry if I explained myself wrong. Actually it is an explanation what it is happening in my tests. I’m assuming from the beginning this topic I’m doing something wrong. If I configure a client in a manual way, following the steps from https://icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring/#high-availability-master-with-clients all is Ok. The problem is when I use REST API to configure the client.

I did the following steps to confirm my conclusion :wink::

  • In the tests that I’ve been doing I didn’t found errors neither master1 log neither startup.log.
  • Then I stopped master2 and started master2. But it didn’t started.I run to verify configuration icinga2 daemon -C and appeared some messages telling that didn’t exits neither endpoint neither zone objects.
  • I replicated the endpoint and zone objects from master1 to master2.
  • Then, I restarted icinga2 master2 sucessfully.

Regards.
Jesús

10

But how do I replicate zone and endpoint objects between master1 and master2 ?

Regards