I’ve just set-up graphite following the from scratch blogpost Setting up Graphite from Scratch on Icinga Web 2 however I’m getting not any metrics written to the Graphite server.
When I check the graph debug on a particular host I get the below;
+ Icinga check command: 'hostalive' + Obscured check command: NULL + Applying templates for check command 'hostalive' ++ Applying template 'hostalive-rta' +++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.rta.value' +++ Excluded 0 metric(s) +++ Combined 0 metric(s) to 0 chart(s) ++ Applying template 'hostalive-pl' +++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.pl.value' +++ Excluded 0 metric(s) +++ Combined 0 metric(s) to 0 chart(s) + Applying default templates, excluding previously used metrics ++ Applying template 'default-host' +++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.%2A.value' +++ Excluded 0 metric(s) +++ Combined 0 metric(s) to 0 chart(s) ++ Not applying template 'default-service'
Looking through my icinga.log I get the below
[2021-09-03 16:17:33 +0100] warning/GraphiteWriter: Can't connect to Graphite on host 'host_ip_address' port '2003.' Context: (0) Reconnecting to Graphite 'graphite' [2021-09-03 16:17:33 +0100] critical/GraphiteWriter: Exception during Graphite operation: Verify that your backend is operational!
If I check netstat on the server where Graphite is installed I can see the server is listening on this port.
root@graphiteserver ~]# netstat -tulnp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN 76670/python2
However if I try telnet on the server where graphite is installed I get a “no route to host error”
root@icingaserver ~]# telnet graphite_ip 2003 Trying graphite_ip... telnet: connect to address graphite_ip: No route to host
I think this may be a network related issue, but while I’m waiting on our networks team to get back to me I was wondering if anyone else had come across something similar?