Monitoring Avaya Communication Management

Hello,

I am trying to figure out how can we monitor Avaya Communication management. We have set up SNMPV3 and its working. What is the nhext step to get more check to work. Do i need some OID from avaya?
What are the possibilities with Avaya monitors?

Never used Avaya before. Maybe it haves some kind of Health Endpoints in an API?

I use the monitoring-plugins/check-plugins/snmp at main · Linuxfabrik/monitoring-plugins · GitHub plugin with the following csv files.

acm.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.6.0,Server1 Duplication State,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.9.0,Server2 Duplication State,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.4.0,Server1 Name,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.7.0,Server2 Name,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.13.0,StandBY Server Refreshed,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.6.0,License Limit,int(value),,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.4.0,Licenses used,int(value),,,,,
,warnPercent,95,%,,,,
,critPercent,98,%,,,,
,License Usage,"round(values['Licenses used'] * 100.0 / values['License Limit'],1)",%,value > values['warnPercent'],value > values['critPercent'],True,

acm_freshness.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.13.0,StandBY Server Refreshed,,,,,,WARN

acm_license.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.6.0,License Limit,int(value),,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.4.0,Licenses used,int(value),,,,,
,warnPercent,95,%,,,,
,critPercent,98,%,,,,
,License Usage,"round(values['Licenses used'] * 100.0 / values['License Limit'],1)",%,value > values['warnPercent'],value > values['critPercent'],True,
OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.6.0,License Limit,int(value),,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.20.4.0,Licenses used,int(value),,,,,
,warnPercent,95,%,,,,
,critPercent,98,%,,,,
,License Usage,"round(values['Licenses used'] * 100.0 / values['License Limit'],1)",%,value > values['warnPercent'],value > values['critPercent'],True,

acm_server.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.6.0,Server1 Duplication State,,,,,,WARN
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.9.0,Server2 Duplication State,,,,,,WARN
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.4.0,Server1 Name,,,,,,
SNMPv2-SMI::enterprises.6889.2.73.8.1.4.7.0,Server2 Name,,,,,,

sbc_advanced_licence_in_use.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.77.11.4.0,SBC Advanced License in Use,,,,,,WARN

sbc_std_licence_in_use.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.77.11.2.0,SBC Standard License in Use,int(value),,value > 100,,True,

sbc_total_active_calls.csv

OID,Name,Re-Calc,Unit Label,WARN,CRIT,Show in 1st Line,Report Change as
SNMPv2-SMI::enterprises.6889.2.77.1.3.1.10.0,StandBY Server Refreshed,,,,,,WARN

I have no clue how useful this is as I don’t administer the Avaya system but this is what they wanted.

I also used some of my DSL magic to check over the simple checks aggregated on the cluster hosts.


object CheckCommand "116-cmd-min-halve" {
    import "plugin-check-command"
    command = [ "/usr/lib64/nagios/plugins/dummy" ]
    timeout = 10s
    arguments += {
        "--message" = {
            required = false
            value = {{
                var output_status = ""
                var up_count = 0
                var down_count = 0
                var cluster_nodes = macro("$116_cluster_nodes$")
                var min_halve_service_name = macro("$116-cluster-min-halve-service$")
            
                for (node in cluster_nodes) {
                  if (get_service(node, min_halve_service_name).state > 0) {
                    down_count += 1
                  } else {
                    up_count += 1
                  }
                }
            
                if (up_count >= down_count) {
                  output_status = "OK: "
                }
                if (up_count < down_count) {
                  output_status = "CRITICAL: "
                }
            
                var output = output_status
            
                for (node in cluster_nodes) {
                  output += node + ": " + min_halve_service_name + ": " + get_service(node, min_halve_service_name).last_check_result.output + " "
                }
            
                output += " | count_of_alive_" + min_halve_service_name +"="+up_count+";" + string((up_count + down_count) / 2 + 1) + ":;" + string((up_count + down_count) / 2 ) + ":;0;" + string(up_count + down_count)
                log(output)
                return output
            }}
        }
        "--state" = {{
            var up_count = 0
            var down_count = 0
            var cluster_nodes = macro("$116_cluster_nodes$")
            var min_halve_service_name = macro("$116-cluster-min-halve-service$")
        
            for (node in cluster_nodes) {
              if (get_service(node, min_halve_service_name).state > 0) {
                down_count += 1
              } else {
                up_count += 1
              }
            }
        
            if (up_count >= down_count) {
              return "ok" // halve up then down -> OK
            }
            if (up_count < down_count) {
              return "crit" // less up then down -> Critical 
            }
            return "unk" // should never reach this
        }}
    }
}
2 Likes

Thank you this is some very usefull information,

Can you explain me how to configure this in icinga?? Is it via web console or do you have to add/create something on the master?

I use the Icinga Director if you configure Icinga2 directly via the DSL then you don’t need the linuxfabrik’s dummy and can use the build in dummy for the code of the “116-cmd-min-halve” cluster CheckCommand.

For the SNMP check, the connection information is configured as every other check but the *.csv files need to be on the host where you placed the plugin in a directory at ./device-oids relative to the snmp check. Details are available at monitoring-plugins/check-plugins/snmp at main · Linuxfabrik/monitoring-plugins · GitHub the @linuxfabrik could also help if you have problems with setting it up.

Hi Dominik,

Thanks a lot for sharing this! Great information there!

However, I think I need several questions to be answered before I could reuse what you wrote:

  1. What are macro("$116_cluster_nodes$") and other “macro” commands doing, and what are the prerequisites for these to work?

  2. I suppose the 116 is some sort of naming convention. Could you please share how this naming convention works, and what problem it addresses?

  3. I thought the 116-cmd-min-halve CheckCommand would be associated with the Host, but I see it is using the Critical status (and not the Down status), so I am not sure. Can you please explicit how the 116-cmd-min-halve CheckCommand is related to the SNMP checks, and to the services we can see on the screenshots?

Thank you,

Jean

  1. it’s a field (director) I set on on the cluster host defining the hosts that act as cluster nodes.
    it looks like this in the Custom Variables:
cluster_nodes (Array)	2 items
[0]	ictavsbclp01.example.com
[1]	ictavsbclp02.example.com

or in the DSL

object Host "Avaya SBC SIP TRUNK" {
    import "tpl-host-cluster-dummy"

    vars["cluster_nodes"] = [ "ictavsbclp01.example.com", "ictavsbclp02.example.com" ]
    vars.teams = [ "Telefonie_Avaya" ]
}

https://icinga.com/docs/icinga-2/latest/doc/18-library-reference/#macro
2. yes, you can ignore all instances of 116 as it’s only a prefix to prevent overwriting variables and objects by the monitoring-plugins and lfops defaults. I removed it in this post.
3. it’s associated with the cluster host but it processes the status of the cluster-min-halve-service on the hosts defined in cluster_nodes. The only relation to the snmp checks is that I put them into cluster-min-halve-service, like so:

object Service "Min halve - SNMP - SBC - Advanced License in Use" {
    host_name = "Avaya SBC SIP TRUNK"
    import "cluster-min-halve"

    vars["cluster-min-halve-service"] = "SNMP - SBC - Advanced License in Use"
}
1 Like

Thanks a lot, it all makes sense, and this thread has been a real eye-opener for me!

I suppose the check to determine the Host status of the Cluster is not relevant (probably is a dummy check, or a ping check on the VIP).

1 Like

Correct, in this case it’s a dummy that’s also using the min-halve trick as I have no IP to ping, so I calculate the state by checking if at least one of the cluster_nodes is up. If there’s a cluster IP, I would use an other template that has the classical ping as host-alive check.

I have set up SNMPV3 command and sysuptime check. I got MIBS from the avaya and stored it in /usr/share/snmp/mibs/AVAYA-AURA-CM-MIB.txt.

I made new check for alarms but i am getting: SNMP OK - No Such Object available on this agent at this OID .

Use the Inspect action to get the actual command line and try to run it manually with debug and or verbose arguments.
Also try to get the OID you require with the net-snmp tools.

Wel with snmpwalk -v3 -l authPriv -u xxxxx -A xxxxxx -X xxxxxx -a SHA -x DES IP adress -m /usr/share/snmp/mibs/AVAYA-AURA-CM-MIB avCmAlarms

its give me the all availble OIDS for avCmAlarms

But the probleem is that te check_snmp using snmpget and not walk. And snmpget not working for me

/usr/lib/nagios/plugins/check_snmp -H xxxx -v 3 -U xxxxx -A xxxx -X xxxx -a SHA -x DES -L authPriv -m /usr/share/snmp/mibs/AVAYA-AURA-CM-MIB -o 1.3.6.1.4.1.6889 -v
/usr/bin/snmpget -Le -t 10 -r 5 -m /usr/share/snmp/mibs/AVAYA-AURA-CM-MIB -v 1 [context] [authpriv] :161 1.3.6.1.4.1.6889

Hello Dominik,

Can you show me the output of the check you monitoring for the Avaya?? Because i am using snmpget command but the not all OID from AVAYA-AURA-CM-MIB mibs are working for me.

OK: ictavsbclp01: SNMP - SBC - Advanced License in Use: Key                         ! Value ! State 
----------------------------+-------+-------
SBC Advanced License in Use ! 0     !  ictavsbclp02: SNMP - SBC - Advanced License in Use: Key                         ! Value ! State 
----------------------------+-------+-------
SBC Advanced License in Use ! 0     ! 
OK: ictavsbclp01: SNMP - SBC - Standard License in Use: SBC Standard License in Use: 0

Key                         ! Value ! State 
----------------------------+-------+-------
SBC Standard License in Use ! 0     !  ictavsbclp02: SNMP - SBC - Standard License in Use: SBC Standard License in Use: 2

Key                         ! Value ! State 
----------------------------+-------+-------
SBC Standard License in Use ! 2     ! 
Server1 Duplication State: "active" (WARN, changed from ""standby""), Server2 Duplication State: "standby" (WARN, changed from ""active"")

Key                       ! Value         ! State     
--------------------------+---------------+-----------
Server1 Duplication State ! "active"      !  
Server2 Duplication State ! "standby"     !  
Server1 Name              ! "ictavcmlp02" !       
Server2 Name              ! "ictavcmlp01" ! 
SBC TotalActiveCalls: 12 (WARN, changed from "0")

Key                  ! Value ! State     
---------------------+-------+-----------
SBC TotalActiveCalls ! 12    ! 

I suspect that SNMPv2-SMI is the name of the MIB file where you get the OID today. and the snmp is V2 and not V3 is it correct? Can you still forward the command itself from the check from the director. Thanks in advance

Do you mean that with command snmpwalk you do get a proper reply and with command snmpget you only have errors?

Could you share both commands and outputs/errors for snmpget and snmpwalk?

Not really, the SNMPv2-SMI::enterprises only shortens the the OID a bit.

I connect via SNMP v3. This should have no influence over which OIDs are readable if there were no different rights configured per connection method on the avaya side.

Can you still forward the command itself from the check from the director

Not sure, what you mean with this.

From our Avaya department I received this mib “AVAYA-AURA-CM-MIB” this mib placed in /usr/share/snmp/mibs.

They want to monitor all major, minor, critical, warning of avCmAlarms in icinga.

For the SNMP I use check_snmp command This uses snmpget.

With snmpwalk snmpwalk -v3 -l authPriv -u xxxxxx-A xxxxx-X Welkom01 -a SHA -x DES hostIP -m /usr/share/snmp/mibs/AVAYA-AURA-CM-MIB.txt avCmAlarms -

  • output of I am not allowed to upload (too big) but example of is

AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016370”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016372”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016373”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016374”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016375”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016384”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016385”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016386”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016387”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016405”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016413”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016413”.“DIG-IP-S”.2 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016416”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016416”.“DIG-IP-S”.2 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016417”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016418”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016419”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016468”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016497”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016498”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016516”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016545”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016564”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016576”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016612”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016613”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016656”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016660”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016661”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016662”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016691”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016729”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016754”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016755”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016788”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016802”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016803”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016817”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:
AVAYA-AURA-CM-MIB::avCmAlarmsAck.“S016818”.“DIG-IP-S”.1 = STRING: Ack 1: Ack 2:

With check_snmp ‘/usr/lib/nagios/plugins/check_snmp’ ‘-A’ ‘xxxx’ ‘-H’ ‘IP’ ‘-L’ ‘authPriv’ ‘-P’ ‘3’ ‘-U’ ‘xxxxx’ ‘-X’ ‘xxxxx’ ‘-a’ ‘SHA’ ‘-m’ ‘AVAYA-AURA-CM-MIB.txt’ ‘-o’ ‘.1.3.6.1.4.1.6889.2.73.8.1.5’ ‘-p’ ‘161’ ‘-t’ ‘10’ ‘-x’ ‘DES’
Do I get output

SNMP OK - No Such Object available on this agent at this OID |

Oh i ment can you show the icinga director > command you created for snmp

The code the director generates:

apply Service "SNMP - SBC - TotalActiveCalls" {
    import "tpl-service-snmp"

    enable_notifications = false
    assign where "sbc-sip-trunk" in host.vars.custom_tags || "sbc-uc-cisco" in host.vars.custom_tags
    vars.snmp_device = "sbc_total_active_calls.csv"
    vars.snmp_hostname = "$host_name$"
    vars.snmp_snmp_version = "3"
    vars.snmp_v3_auth_prot = "SHA"
    vars.snmp_v3_auth_prot_password = "asdfasdf"
    vars.snmp_v3_level = "authPriv"
    vars.snmp_v3_priv_prot = "AES"
    vars.snmp_v3_priv_prot_password = "asdfasdf"
    vars.snmp_v3_username = "nagios"
    zone = "master"

    import DirectorOverrideTemplate
}

will result in

'/usr/lib64/nagios/plugins/snmp' '--community' 'public' '--device' 'sbc_total_active_calls.csv' '--hostname' 'ictavsbclp03' '--mib-dir' '$HOME/.snmp/mibs:/usr/share/snmp/mibs' '--snmp-version' '3' '--timeout' '7' '--v3-auth-prot' 'SHA' '--v3-auth-prot-password' 'asdfasdf' '--v3-level' 'authPriv' '--v3-priv-prot' 'AES' '--v3-priv-prot-password' 'asdfasdf' '--v3-username' 'nagios'

sbc_total_active_calls.csv I posted above and asdfasdf isn’t a real password :wink: