Graphite Writer cant connect to host on default port

Visualization, Metrics & Reporting
1

Hi All,

I’ve just set-up graphite following the from scratch blogpost Setting up Graphite from Scratch on Icinga Web 2 however I’m getting not any metrics written to the Graphite server.

When I check the graph debug on a particular host I get the below;

+ Icinga check command: 'hostalive'
+ Obscured check command: NULL
+ Applying templates for check command 'hostalive'
++ Applying template 'hostalive-rta'
+++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.rta.value'
+++ Excluded 0 metric(s)
+++ Combined 0 metric(s) to 0 chart(s)
++ Applying template 'hostalive-pl'
+++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.pl.value'
+++ Excluded 0 metric(s)
+++ Combined 0 metric(s) to 0 chart(s)
+ Applying default templates, excluding previously used metrics
++ Applying template 'default-host'
+++ Fetched 0 metric(s) from 'http://graphite_ip/metrics/expand?query=icinga2.VPN-Router.host.hostalive.perfdata.%2A.value'
+++ Excluded 0 metric(s)
+++ Combined 0 metric(s) to 0 chart(s)
++ Not applying template 'default-service'

Looking through my icinga.log I get the below

[2021-09-03 16:17:33 +0100] warning/GraphiteWriter: Can't connect to Graphite on host 'host_ip_address' port '2003.'
Context:
	(0) Reconnecting to Graphite 'graphite'

[2021-09-03 16:17:33 +0100] critical/GraphiteWriter: Exception during Graphite operation: Verify that your backend is operational!

If I check netstat on the server where Graphite is installed I can see the server is listening on this port.

root@graphiteserver ~]# netstat -tulnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:2003            0.0.0.0:*               LISTEN      76670/python2

However if I try telnet on the server where graphite is installed I get a “no route to host error”

root@icingaserver ~]# telnet graphite_ip 2003
Trying graphite_ip...
telnet: connect to address graphite_ip: No route to host

I think this may be a network related issue, but while I’m waiting on our networks team to get back to me I was wondering if anyone else had come across something similar?

1 Like
2

Assuming that graphite is the thing owning that port, I would definitely say that it’s a firewall issue (either network or on the machine itself)

1 Like