A distro agnostic guide to Graphite with venv (examples: RHEL8, Debian10, Ubuntu18)

Thanks for the guide.

I recently made the experience that Graphite API is a great replacement for Graphite Web, as it removed all the Web UI and Django requirements.

Resources:

https://graphite-api.readthedocs.io/en/latest/

Unfortunately no activity since 2017

Hello Blake!
Great tutorial, thank you!
But I dare to point to a little mistake:

This seems to be a reference to SELinux, not Systemd, doesn’t it?

Typo discovery ftw. Will fix. Thanks.

Hi Blake,

Firstly, thanks for the guide, I used this to setup graphite with my icinga2.

Just thought I’d comment, I’m new to linux found the guide pretty tricky in some parts where steps were left out or not explained thoroughly (understandable as the target audience was probably people familiar with gnu/linux)

e.g. advising to create .conf from examples sometimes but not others
entering and existing virtual environment
editing carbon.conf was a bit confusing

and for anyone else’s benefit, i had a error when running PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb

turned out I needed to run sudo apt-get install libcairo2-dev first on my ubuntu 18.04.

but overall very thankful for the guide, great work Blake.

libcairo? Weird. I’ll spin up a VM at some point this weekend and update accordingly if I catch anything else missing.

I’ll look into the the flow a bit. Thanks for the feedback. I can’t make carbon.conf not confusing though, that thing is a monstrosity lol

Just went through your how to, so thanks at first! Very helpful and it works.

At first I tried to omit the “multiple” relay config and leave those settings on default, but that didn’t work.
So I just configured everything as in the how to, but removed the second relay.
Got it working then.

This was a bit confusing at first, as it goes into the same config file carbon.conf, but you mention carbon-cache config. So I spent some minutes looking for some carbon-cache.conf :smiley:

Also when trying to start the daemons I got an error:

Jul 11 10:57:11 blub carbon-relay.py[44875]: In carbon.conf, DESTINATION_PROTOCOL must be one of pickle, line. Invalid value: 'pickle
Jul 11 10:57:11 blub carbon-relay.py[44875]: DESTINATION_TRANSPORT = none'

Problem was a single space in front of DESTIONATION_PORT = none.
Maybe add some hint to not having space in front of the key-value pairs (for dummies like me ;))

This wasn’t needed for me on my Ubuntu 18.04 machine

Yea, I’m not sure.

I was just getting

File “/opt/graphite/webapp/cairocffi/init.py”, line 36, in dlopen
raise OSError(“dlopen() failed to load a library: %s” % ’ / '.join(names))
OSError: dlopen() failed to load a library: cairo / cairo-2 / cairo-gobject-2 / cairo.so.2

After trying to run PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb

but after I did apt-get install libcairo2-dev it worked.

So again not sure, very new to linux.

Likely the cairo devel package pulls in the actual library required by graphite web. Cairo in this regard is used to render the graph images. Could be a problem with pip dependencies.

@blakehartshorn maybe it is a good idea to add the check for dependencies into the installation details, like described here?

Chances are I just messed this up when consolidating my notes for Debian and Ubuntu as they’re pretty redundant. I’ll throw that in and if I find time this weekend do a dry run of all this on an Ubuntu VM.

Okay, this also affected Buster :man_facepalming:

Thanks for pointing out my errors. I got through install testing deb/ubuntu and finished the guide on RHEL. Since the carbon/carbonffi Python modules built without protest in pip, I figured they were fine.

Refer to how to install all errors, only use docker…

Hi @blakehartshorn
love that tutorial, escpecially as I had such a pain in my *** with getting graphite running…
Unfortunately I still had issues on my Ubu 18.04, I simply couldn’t get the graphite.wsgi running by gunicorn (always said module “graphite” not found).
Anyway, this may be also some nice way as it avoids especially my “running on my machine” issue by using docker: https://computingforgeeks.com/how-to-install-and-configure-graphite-on-ubuntu-18-04/
It provides a docker with graphite-carbon / whisper and graphite-web along with statsd and an nginx proxy in front.

Some other question regarding your template: would you mind to share those? I am wondering why my has “dark mode” appearance by default which doesn’t fit well into “light” icinga apperance :wink:

Best & Thanks,
Matthias

Hello,

I have problem with started carbon-relay.service

[root@lmaster ~]# cat /etc/systemd/system/carbon-relay.service
[Unit]
Description=Graphite Carbon Relay
After=network.target

[Service]
Environment=PATH=/opt/graphite/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
Environment=PYTHONPATH=/opt/graphite/webapp:/opt/graphite
Environment=VIRTUAL_ENV=/opt/graphite
Type=forking
StandardOutput=syslog
StandardError=syslog
ExecStart=/opt/graphite/bin/carbon-relay.py --config=/opt/graphite/conf/carbon.conf --pidfile=/var/run/carbon-relay.pid start
ExecReload=/bin/kill -USR1 $MAINPID
PIDFile=/var/run/carbon-relay.pid
Restart=always

[Install]
WantedBy=multi-user.target


May 21 13:20:46 lmaster.localdomain systemd[1]: carbon-relay.service: Control process exited, code=exited status=1
May 21 13:20:46 lmaster.localdomain systemd[1]: carbon-relay.service: Failed with result ‘exit-code’.
May 21 13:20:46 lmaster.localdomain systemd[1]: Failed to start Graphite Carbon Relay.


I am try start directly carbon-relay.py

[root@lmaster ~]# /opt/graphite/bin/carbon-relay.py --config=/opt/graphite/conf/carbon.conf --pidfile=/var/run/carbon-relay.pid start
Removing stale pidfile /var/run/carbon-relay.pid
Starting carbon-relay (instance a)
An error has occurred: b’socket.gaierror: [Errno -2] Name or service not known’
Please look at log file for more information.


In the icinga2.log is
[2020-05-21 12:23:46 +0200] warning/GraphiteWriter: Can’t connect to Graphite on host ‘127.0.0.1’ port ‘2013.’
Context:
(0) Reconnecting to Graphite ‘graphite’

[2020-05-21 12:23:46 +0200] information/GraphiteWriter: Finished reconnecting to Graphite in 0.00087595 second(s).
[2020-05-21 12:23:46 +0200] information/IdoMysqlConnection: MySQL IDO instance id: 1 (schema version: ‘1.14.3’)
[2020-05-21 12:23:46 +0200] information/IdoMysqlConnection: Finished reconnecting to ‘ido-mysql’ database ‘icinga’ in 0.0871511 second(s).
[2020-05-21 12:23:47 +0200] critical/GraphiteWriter: Cannot write to TCP socket on host ‘127.0.0.1’ port ‘2013’.


Do you have any idea ?

Thanks
Jozef

Hi everyone,
I am add info with debug for detailed problem start service carbon-relay.
And next info this problem is only CENTOS 8 (I tryed selinux disable. Not efect), on the UBUNTU service carbon-relay run fine.

Jozef

[root@lmaster ~]# /opt/graphite/bin/python3 -m pdb /opt/graphite/bin/carbon-relay.py --config=/opt/graphite/conf/carbon.conf --pidfile=/var/run/carbon-relay.pid start

/opt/graphite/bin/carbon-relay.py(14)()
→ limitations under the License.“”"
(Pdb)
(Pdb) n
/opt/graphite/bin/carbon-relay.py(16)()
→ import sys
(Pdb) n
/opt/graphite/bin/carbon-relay.py(17)()
→ import os.path
(Pdb) n
/opt/graphite/bin/carbon-relay.py(20)()
→ BIN_DIR = os.path.dirname(os.path.abspath(file))
(Pdb) n
/opt/graphite/bin/carbon-relay.py(21)()
→ ROOT_DIR = os.path.dirname(BIN_DIR)
(Pdb) n
/opt/graphite/bin/carbon-relay.py(25)()
→ LIB_DIR = os.path.join(ROOT_DIR, “lib”)
(Pdb) n
/opt/graphite/bin/carbon-relay.py(26)()
→ sys.path.insert(0, LIB_DIR)
(Pdb) n
/opt/graphite/bin/carbon-relay.py(28)()
→ from carbon.util import run_twistd_plugin # noqa
(Pdb) n
/opt/graphite/bin/carbon-relay.py(29)()
→ from carbon.exceptions import CarbonConfigException # noqa
(Pdb) n
/opt/graphite/bin/carbon-relay.py(31)()
→ try:
(Pdb) n
/opt/graphite/bin/carbon-relay.py(32)()
→ run_twistd_plugin(file)
(Pdb) n
Removing stale pidfile /var/run/carbon-relay.pid
Starting carbon-relay (instance a)
An error has occurred: b’socket.gaierror: [Errno -2] Name or service not known’
Please look at log file for more information.
[root@lmaster ~]#

An error has occurred: b’socket.gaierror: [Errno -2] Name or service not known’

carbon-relay.service has:
Environment=/opt/graphite/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

Should be changed to:
Environment=PATH=/opt/graphite/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

Also the PID file location is an issue on Debian 10 at least. My complete carbon-relay.service which works on Debian 10 and should work on other OS’es too:

[Unit]
Description=Graphite Carbon Relay
After=network.target

[Service]
Environment=PATH=/opt/graphite/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
Environment=VIRTUAL_ENV=/opt/graphite
User=carbon
Group=carbon
Type=forking
StandardOutput=syslog
StandardError=syslog
ExecStart=/opt/graphite/bin/carbon-relay.py --config=/opt/graphite/conf/carbon.conf --pidfile=/opt/graphite/storage/carbon-relay.pid start
ExecReload=/bin/kill -USR1 $MAINPID
PIDFile=/opt/graphite/storage/carbon-relay.pid
Restart=always

[Install]
WantedBy=multi-user.target

Thanks for spotting more typos. I’m going to do a dry run on this with Ubuntu 20 this weekend anyway, so I’ll try to make the corrections at the same time as adding that.

Fixed my PATH and PID errors in the systemd config, changed a few warnings about carbon.conf, and tested this guide on Ubuntu 20.04.

@blakehartshorn

(graphite) $ pip install --upgrade pip wheel setuptools gunicorn whisper
(graphite) $ pip install carbon --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/lib"
(graphite) $ pip install graphite-web --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp"

When installing on a new Ubuntu 20.04 vm these commands throw errors:
not sure what the problem with the first one is, as it seems to do what it is supposed to.

(graphite) carbon@icinga:~$ pip install --upgrade pip wheel setuptools gunicorn whisper
Collecting pip
  Downloading pip-20.3.3-py2.py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 3.2 MB/s
Collecting wheel
  Downloading wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Collecting setuptools
  Downloading setuptools-51.1.1-py3-none-any.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 6.7 MB/s
Collecting gunicorn
  Downloading gunicorn-20.0.4-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 5.4 MB/s
Collecting whisper
  Downloading whisper-1.1.7.tar.gz (37 kB)
Collecting six
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Building wheels for collected packages: whisper
  Building wheel for whisper (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/graphite/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-spt3z2lp/whisper/setup.py'"'"'; __file__='"'"'/tmp/pip-install-spt3z2lp/whisper/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-fw6z792c
       cwd: /tmp/pip-install-spt3z2lp/whisper/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for whisper
  Running setup.py clean for whisper
Failed to build whisper
Installing collected packages: pip, wheel, setuptools, gunicorn, six, whisper
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
  Attempting uninstall: setuptools
    Found existing installation: setuptools 44.0.0
    Uninstalling setuptools-44.0.0:
      Successfully uninstalled setuptools-44.0.0
    Running setup.py install for whisper ... done
Successfully installed gunicorn-20.0.4 pip-20.3.3 setuptools-51.1.1 six-1.15.0 wheel-0.36.2 whisper-1.1.7
(graphite) carbon@icinga:~$ pip install carbon --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/lib"
/opt/graphite/lib/python3.8/site-packages/pip/_internal/commands/install.py:234: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
  cmdoptions.check_install_build_global(options)
ERROR: Location-changing options found in --install-option: ['--prefix', '--install-lib'] from command line. This is unsupported, use pip-level options like --user, --prefix, --root, and --target instead.

pip has been updated in he meantime to not support these options:

As I am pretty inexperienced with python/pip I have no idea how to work around that :sweat:

As I am pretty inexperienced with python/pip I have no idea how to work around that :sweat:

Hi.

Recently I had the same problem.
For the sake of simplicity, I downgraded pip (e.g. to pip 9).


Greetings

1 Like

Thanks for the hint.
Did that and it worked. So basically one could simply leave out the “pip” from the first upgrade command and it should be ok (for now)

1 Like