Check_by_ssh

szabolcssandor · January 24, 2020, 12:29pm

Hi!

I have a problem what I cannot managed to solv for days. I cannot use check_by_ssh if I use not nagios user.

Error message is the next:
Remote command execution failed: icinga@host-IP: Permission denied (publickey,keyboard-interactive).

What I checked already:

On server:
icinga@icinga:~$ ssh host-IP
Last login: Fri Jan 24 12:57:40 2020 from server-IP

icinga@icinga:~$ ssh host-IP “/usr/lib/nagios/plugins/check_users -w 2 -c 5”
USERS OK - 1 users currently logged in |users=1;2;5;0

On client:
icinga@ceg-irattar-backup:~> /usr/lib/nagios/plugins/check_users -w 2 -c 5
USERS OK - 1 users currently logged in |users=1;2;5;0
icinga@ceg-irattar-backup:~>

icingaconfig files:

command.cfg:

object CheckCommand “by_ssh_users_lib” {
import “by_ssh”
vars.by_ssh_command = “/usr/lib/nagios/plugins/check_users -v -w $by_ssh_users_wgreater$ -c $by_ssh_users_cgreater$”
vars.by_ssh_users_wgreater = “3”
vars.by_ssh_users_cgreater = “5”
}

services.cfg:

apply Service “users” {
import “generic-service”
check_command = “by_ssh_users_lib”
assign where host.vars.CheckType == “by_ssh” && host.vars.PluginPath == “lib”
}

object Host “ceg-irattar-backup.bktvsz.justice.hu” {
import “generic-host”
address = “host-IP”
vars.desciption = “ceg-irattar-backup”
vars.os = “Linux”
vars.CheckType = “by_ssh”
vars.by_ssh_logname = “icinga”
vars.PluginPath = “lib”
}

I have created a user named nagios on host before.
If I set vars.by_ssh_logname = “nagios” then everything is fine.

My problem is I would like to monitor with ssh some host with different user as nagios.

Thank You For Your Help.

gkoutsog · January 24, 2020, 12:52pm

Hi,

Could be failing due to SSH waiting for action on first usage.
Have you tried manually ssh’ing from icinga server/satellite to client with said (not nagios) account?

Cheers,
George

szabolcssandor · January 24, 2020, 4:49pm

Yes, and I managed to run check command as well.

szabolcssandor · January 24, 2020, 6:31pm

I can ssh and run check command over ssh in console with icings user, but I received error message as you see in previous write.

gkoutsog · January 24, 2020, 9:32pm

I suggest turning on debug log and see what is written there during execution of command.

Cheers,
George

log1c · January 27, 2020, 7:05am

If I see correctly you are missing the publickey that is authorizing your user to log in at the remote machine.

Here is a small tutorial(source: https://www.thomas-krenn.com/de/wiki/Icinga_check_by_ssh_Plugin?xtxsearchselecthit=1#):

On the monitoring server do:
Create a new folder for your keyfiles, e.g. /var/lib/icinga/.ssh and then create the keys (no password!)

cd /var/lib/icinga/
mkdir .ssh
ssh-keygen

Then make the user running Icinga 2 (icinga on CentOS/RHEL/SUSE, nagios on Debian/Ubuntu) the owner of the key folder
chown -R icinga.icinga /var/lib/icinga/.ssh

Then copy the pubkey to the remote host with the user you want to use to monitoring and have created on the
ssh-copy-id -i .ssh/id_rsa.pub user@remotehost_ip/fqdn
Then log in using the public key, no password query should come!
ssh -i /var/lib/icinga/.ssh/id_rsa user@remotehost_ip/fqdn

You don’t really need to create a new user on the remote server, you also can use an existing one, just modify the ssh-copy-id and ssh -i statements above according to the user on the remote host.

After this run the Icinga check_by_ssh from the CLI as the Icinga 2 user:

sudo -u icinga '/usr/lib/nagios/plugins/check_by_ssh' '-C' '/usr/lib/nagios/plugins/check_whatever' '-i' '/var/lib/nagios/.ssh/id_rsa' '-l' 'icinga'

szabolcssandor · January 27, 2020, 9:35am

I don’t know what I did wrong.
1.) first step create icinga user on server
root@icinga:/etc/icinga2/conf.d# groupadd -r icinga
root@icinga:/etc/icinga2/conf.d# useradd -r -m -d /var/lib/icinga -g icinga -s /bin/bash icinga
root@icinga:/etc/icinga2/conf.d# passwd icinga
root@icinga:/etc/icinga2/conf.d# su - icinga
2.) Create RSA key pairs
cinga@icinga:~$ ssh-keygen
2.) copy public rsa key to Host
icinga@icinga:~$ ssh-copy-id
3.) For test ssh to Remote host/
icinga@icinga:~$ ssh
icinga@:~>
/CRLT-D exit from host/
4.) for test run check command as icinga on server
root@icinga:/etc/icinga2/conf.d# sudo -u icinga ‘/usr/lib/nagios/plugins/check_by_ssh’ ‘-C’ ‘/usr/lib/nagios/plugins/check_users -v -w 3 -c 5’ ‘-H’ ‘’ ‘-l’ ‘icinga’
USERS OK - 1 users currently logged in |users=1;3;5;0
/I’v received the correct check output/

When I set
host.conf:

object Host “ceg-irattar-backup.bktvsz.justice.hu” {
import “generic-host”
address = “<Host_IP>”
vars.desciption = “ceg-irattar-backup”
vars.os = “Linux”
vars.location = “M3”
vars.CheckType = “by_ssh”
vars.by_ssh_logname = “icinga”
vars.PluginPath = “lib”
vars.location = “BKT M3”
vars.notification[“mail”] = {
users = [ “sandorsz” ]
}
}

command.conf:

object CheckCommand “by_ssh_users_lib” {
import “by_ssh”
vars.by_ssh_command = “/usr/lib/nagios/plugins/check_users -v -w $by_ssh_users_wgreater$ -c $by_ssh_users_cgreater$”
vars.by_ssh_users_wgreater = “3”
vars.by_ssh_users_cgreater = “5”
}

services.conf:

apply Service “users” {
import “generic-service”
check_command = “by_ssh_users_lib”
assign where host.vars.CheckType == “by_ssh” && host.vars.PluginPath == “lib”
}

Unfortunately I received error message:
Remote command execution failed: icinga@10.15.40.26: Permission denied (publickey,keyboard-interactive).

in debug log:
[2020-01-27 10:21:52 +0100] notice/Process: PID 5374 (’/usr/lib/nagios/plugins/check_by_ssh’ ‘-C’ ‘/usr/lib/nagios/plugins/check_swap -v -w 75% -c 50%’ ‘-H’ ‘’ ‘-l’ ‘icinga’) terminated with exit
code 3

I have no idea what should I do?
If I run check command as nagios user it goes well.
[2020-01-27 10:13:27 +0100] notice/Process: PID 3991 (’/usr/lib/nagios/plugins/check_by_ssh’ ‘-C’ ‘/usr/lib/nagios/plugins/check_users -v -w 3 -c 5’ ‘-H’ ‘<Host_IP>’ ‘-l’ ‘nagios’) terminated with exit cod
e 0

I have read in https://www.thomas-krenn.com/de/wiki/Icinga_check_by_ssh_Plugin?xtxsearchselecthit=1# tutorial that
The setup assumes that Icinga runs under the user nagios:
It is possible that I can run only as nagios user check_by_ssh?

Thank Everyone for help!

log1c · January 27, 2020, 10:43am

On which hosts did you perform steps 1-4?

Step 1 should be done on server host you want to monitor by check_by_ssh (though just a useradd is sufficient)
Step 2-4 should be done on the monitoring server

I have updated my post above.
Then you are still missing the -i parameter in the check execution to use the public key to login without a password.

szabolcssandor · January 27, 2020, 10:53am

I took steps 1-4 on icinga server, and I create an icinga user on host before as well. I made connection between the two icinga users. Copy icinga user rsa public key to host icinga user, so I can ssh from icinga server to monitored host without password.

I am really confused.

szabolcssandor · January 27, 2020, 11:29am

I have managed to solve the problem. I have connected users wrongly. Icinga2 runs under the user nagios, so the next step solve the problem:

1.) on server
su - nagios
ssh-copy-id icinga@<host_ip>

Connect server nagios user to remote host icinga user.

Thank Everyone for help.

ls440 · September 15, 2022, 9:33am

Hi, facing issues

‘/usr/lib/nagios/plugins/check_by_ssh’ ‘-C’ ‘/usr/lib/nagio^Cplugins/check_script.sh’ ‘-i’ ‘/home/icinga/.ssh/id_rsa’ -H ‘hostname’

still asks for password.

what I did in a short note:

On the icinga server:

useradd icinga
2)ssh-keygen -b 4096 -t rsa -C “icinga@$(hostname) user for check_by_ssh” -f $HOME/.ssh/id_rsa

i dont want to create icinga users on remote hosts.
ssh-copy-id -i $HOME/.ssh/id_rsa root@clientname

i copied pubkey to remote server under /root/.ssh/authorized_keys
icinga@server:/root$ ssh root@remoteserver -----> works

my question is how can i monitor the scripts placed on remote hosts

icinga@server:/root$ ‘/usr/lib/nagios/plugins/check_by_ssh’ ‘-C’ ‘/usr/lib/nagios/plugins/check_script.sh’ ‘-H’ ‘hostname’ -i /home/icinga/.ssh/id_rsa
Password: CRITICAL - Plugin timed out after 10 seconds
– i think it asks me root password here which differs from host to host. whats the best way to work

log1c · September 16, 2022, 8:39am

Why?
Using the root user with a passwordless login is very insecure!
I’m not even sure if it is allowed without further configuration, hence the password query.

I’d say a dedicated monitoring user on the remote host.

ls440 · September 16, 2022, 8:41am

Dedicated monitoring user i tried . I tried creating icinga user on remote host and by copying the keys . but for some reason it always asks me password though the pubkey is added.

ssh-copy-id -i $HOME/.ssh/id_rsa icinga@clientname
password:

also when i try to issue password, it doesnot take instead the connection gets terminated,

log1c · September 16, 2022, 8:48am

Well, that is to be expected at this stage.
The remote host/user doesn’t know about the key yet, so it asks for a password.

here is a good how to (german, maybe use google translate if necessary): Icinga2 check by ssh - steviesblog.de

ls440 · September 16, 2022, 9:25am

tried in the sameway couple of times it didnot work
icinga@server:~$ ssh-copy-id -i $HOME/.ssh/id_rsa icinga@client
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: “/home/icinga/.ssh/id_rsa.pub”
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys
Unauthorized access to this machine is prohibited
Press if you are not an authorized user
Password:
Password:

though i copied pubkey manually to the authorized key file on user icinga authorizedkeys and test it always ask me password.