Tutorial

Installing and Configuring Graphite and Statsd on an Ubuntu 12.04 VPS

Installing and Configuring Graphite and Statsd on an Ubuntu 12.04 VPS

Status: Deprecated

This article covers a version of Ubuntu that is no longer supported. If you are currently operate a server running Ubuntu 12.04, we highly recommend upgrading or migrating to a supported version of Ubuntu:

Reason: Ubuntu 12.04 reached end of life (EOL) on April 28, 2017 and no longer receives security patches or updates. This guide is no longer maintained.

See Instead:
This guide might still be useful as a reference, but may not work on other Ubuntu releases. If available, we strongly recommend using a guide written for the version of Ubuntu you are using. You can use the search functionality at the top of the page to find a more recent version.

Introduction

Graphite and statsd can be very valuable tools used to visualize data. Statsd collects data about running processes and graphite is a graphing library that can be used to create graphs.

We will be setting these tools up on an Ubuntu 12.04 VPS instance.

Installing Graphite

Graphite has many dependencies that we will need to fulfill. We also want to install the web server packages that will allow us to access the web interface.

First, update and upgrade your packages with apt-get:

sudo apt-get update
sudo apt-get upgrade

Install the needed packages:

sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 build-essential python3.2 python-dev libpython3.2 python3-minimal libapache2-mod-wsgi libaprutil1-ldap memcached python-cairo-dev python-django python-ldap python-memcache python-pysqlite2 sqlite3 erlang-os-mon erlang-snmp rabbitmq-server bzr expect libapache2-mod-python python-setuptools

We will use python-setuptool's "easy_install" utility to install a few more important python components:

sudo easy_install django-tagging zope.interface twisted txamqp

Next, we will acquire the Graphite components from the project's website.

Carbon is the data aggregator, Graphite web is the web component, and whisper is the database library:

cd ~
wget https://launchpad.net/graphite/0.9/0.9.10/+download/graphite-web-0.9.10.tar.gz
wget https://launchpad.net/graphite/0.9/0.9.10/+download/carbon-0.9.10.tar.gz
wget https://launchpad.net/graphite/0.9/0.9.10/+download/whisper-0.9.10.tar.gz

Use tar to extract the archives:

find *.tar.gz -exec tar -zxvf '{}' \;

Move into the whisper directory and install it with provided setup file:

cd whisper*
sudo python setup.py install

Now, move to the carbon directory and do the same:

cd ../carbon*
sudo python setup.py install

Finally, move to the Graphite directory and check dependencies. You should receive a message indicating that your dependencies are all correctly installed.:

cd ../graphite*
sudo python check-dependencies.py
All necessary dependencies are met.
All optional dependencies are met.

You are now safe to install Graphite:

sudo python setup.py install

Configuring Graphite

Next, we will configure the software that we have just installed.

We will move to the Graphite configuration directory copy or create some files that we will use for our applications:

cd /opt/graphite/conf
sudo cp carbon.conf.example carbon.conf
Create a file to handle our polling settings:
sudo nano storage-schemas.conf

Copy the following data into the file:

[stats]
priority = 110
pattern = .*
retentions = 10:2160,60:10080,600:262974

Save and close the file.

Next, we will configure the Graphite database. Go to the Graphite webapp directory and run the database script:

cd /opt/graphite/webapp/graphite/
sudo python manage.py syncdb

You will be prompted to create a superuser account. Type "yes" to continue and then follow the prompts.

Copy the local settings example file to the production version while you are in this directory:

sudo cp local_settings.py.example local_settings.py

Configuring Apache

Next, we will configure Apache to serve our web content.

Begin by copying some more example configuration files:

sudo cp ~/graphite*/examples/example-graphite-vhost.conf /etc/apache2/sites-available/default
sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi

Give the Apache web user ownership of Graphite's storage directory so that it can write data properly:

sudo chown -R www-data:www-data /opt/graphite/storage

Create a directory for our WSGI data:

sudo mkdir -p /etc/httpd/wsgi

We need to edit the Apache configuration file we copied earlier:

sudo nano /etc/apache2/sites-available/default

Change the WSGISocketPrefix to reflect the directory we just created. Also, change the ServerName property to reflect your domain name or IP address:

...
...
WSGISocketPrefix /etc/httpd/wsgi

<VirtualHost *:80>
	ServerName Your.Domain.Name.Here
	...
	...

Save and close the file.

Restart Apache to implement our changes:

sudo service apache2 restart

Installing and Configuring Statsd

To use statsd, we need to install node.js. To do this, we will install "python-software-properties", which contains a utility to add PPAs:

sudo apt-get install python-software-properties

We can now add the appropriate PPA:

sudo apt-add-repository ppa:chris-lea/node.js

Press "enter" to accept the repository change. Now update the apt package list again and install node.js:

sudo apt-get update
sudo apt-get install nodejs

We will now install statsd to feed Graphite data from our VPS.

First, install git so that we can download the project files:

sudo apt-get install git

Now clone the git repository into our optional software directory:

cd /opt
sudo git clone git://github.com/etsy/statsd.git

Create the configuration file for statsd within the new statsd directory:

sudo nano /opt/statsd/localConfig.js

Copy and paste the following information into the file:

{
	graphitePort: 2003,
	graphiteHost: "127.0.0.1",
	port: 8125
}

Starting the Services

Start the carbon data aggregator so that it will begin sending data:

sudo /opt/graphite/bin/carbon-cache.py start

Next, move to the statsd directory and run the files by using the node.js command:

cd /opt/statsd
node ./stats.js ./localConfig.js
19 Jul 21:15:34 - reading config file: ./localConfig.js
19 Jul 21:15:34 - server is up

You should receive a message informing you that the statsd server is up and running.

Viewing the Results

To see the graph data, open a web browser and navigate to your domain or IP address:

Your.Domain.Name.Here

You will see a blank box in the middle.

Graphite blank box

You will also see a navigation menu on the left. This is what we will address first

Graphite left navigation menu

Click on the "Graphite" box and you should have a number of different options. If you click through the file hierarchy, you will see many sets of data.

Graphite left navigation menu expanded

Choose one and click it. It will appear that no data has been graphed. We will need to change the scale to see any information.

Graphite data set with range too large

On the Graphite Composer, choose the second icon from the left to adjust the date range. Choose a range that only captures a few hours on either side of the current time.

Graphite change date range

Next Click on "Graph Options", mouse over "Line Mode" and select "Connected Line" in order to connect the data points.

Graphite select line drawing style Graphite drawing correctly

You might also want to select "Auto-Refresh" to have the graph update automatically as it receives data.

Be aware that every new data set you select adds that data to the graph. It does not switch to that data set. Click on "Graph Data" to remove data sets as necessary. You can also click on the data category on the left again to de-select it.

By Justin Ellingwood

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


About the authors

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
10 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

forgive the n00b questions, but how do I get the services to start automatically on reboot?

also, I ran into the following error:

root@A1-001:/opt# /opt/graphite/bin/carbon-cache.py start Traceback (most recent call last): File “/opt/graphite/bin/carbon-cache.py”, line 28, in <module> from carbon.util import run_twistd_plugin File “/opt/graphite/lib/carbon/util.py”, line 19, in <module> from twisted.scripts._twistd_unix import daemonize ImportError: cannot import name daemonize

and resolved it with the following:

$ apt-get install python-pip $ pip install daemonize then I opened /opt/graphite/lib/carbon/util.py and changed “from twisted.scripts._twistd_unix import daemonize” to “import daemonize”

Please advice how to solve this problem.

python /opt/graphite/webapp/graphite/manage.py syncdb

Could not import graphite.local_settings, using defaults! Could not import graphite.local_settings, using defaults! /usr/lib/python2.6/site-packages/django/conf/init.py:75: DeprecationWarning: The ADMIN_MEDIA_PREFIX setting has been removed; use STATIC_URL instead. “use STATIC_URL instead.”, DeprecationWarning) /usr/lib/python2.6/site-packages/django/conf/init.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty. warnings.warn(“The SECRET_KEY setting must not be empty.”, DeprecationWarning) /usr/lib/python2.6/site-packages/django/core/cache/init.py:82: DeprecationWarning: settings.CACHE_* is deprecated; use settings.CACHES instead. DeprecationWarning Traceback (most recent call last): File “/opt/graphite/webapp/graphite/manage.py”, line 11, in <module> execute_manager(settings) File “/usr/lib/python2.6/site-packages/django/core/management/init.py”, line 459, in execute_manager utility.execute() File “/usr/lib/python2.6/site-packages/django/core/management/init.py”, line 382, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File “/usr/lib/python2.6/site-packages/django/core/management/base.py”, line 196, in run_from_argv self.execute(*args, **options.dict) File “/usr/lib/python2.6/site-packages/django/core/management/base.py”, line 231, in execute self.validate() File “/usr/lib/python2.6/site-packages/django/core/management/base.py”, line 266, in validate num_errors = get_validation_errors(s, app) File “/usr/lib/python2.6/site-packages/django/core/management/validation.py”, line 30, in get_validation_errors for (app_name, error) in get_app_errors().items(): File “/usr/lib/python2.6/site-packages/django/db/models/loading.py”, line 158, in get_app_errors self._populate() File “/usr/lib/python2.6/site-packages/django/db/models/loading.py”, line 67, in _populate self.load_app(app_name) File “/usr/lib/python2.6/site-packages/django/db/models/loading.py”, line 88, in load_app models = import_module(‘.models’, app_name) File “/usr/lib/python2.6/site-packages/django/utils/importlib.py”, line 35, in import_module import(name) File “/opt/graphite/webapp/graphite/events/models.py”, line 6, in <module> from tagging.managers import ModelTaggedItemManager File “/usr/lib/python2.6/site-packages/tagging/managers.py”, line 6, in <module> from django.db.models.query import QuerySet, parse_lookup ImportError: cannot import name parse_lookup

pip freeze

argparse==1.2.1 async==0.6.1 backports.ssl-match-hostname==3.4.0.2 BeautifulSoup==3.0.8.1 beautifulsoup4==4.1.3 chardet==2.2.1 Conch==8.2.0 daemonize==2.3.1 Django==1.4.21 django-tagging==0.3.1 fedora-review==0.5.2 fedpkg==1.20 fpconst==0.7.3 gitdb==0.5.4 GitPython==0.3.2rc1 html5lib==0.90 importlib==1.0.2 iniparse==0.3.1 java-gcj-compat==1.0.79 kitchen==1.1.1 Magic-file-extensions==0.1 mercurial==1.4 munch==2.0.2 ordereddict==1.2 osbs==0.14 packagedb-cli==2.8.2 pcp==1.0 pycrypto==2.0.1 pycurl==7.19.0 pygpgme==0.1 pyliblzma==0.5.3 pyOpenSSL==0.13.1 pysqlite==2.3.5 python-bugzilla==1.2.1 python-fedora==0.5.6 python-ldap==2.3.10 python-memcached==1.43 PyXML==0.8.4 requests==2.6.0 rpkg==1.35 simplejson==2.0.9 six==1.9.0 smmap==0.8.1 SOAPpy==0.11.6 straight.plugin==1.4.0.post1 tagging==0.2.1 Twisted==15.4.0 Twisted-Core==8.2.0 Twisted-Lore==8.2.0 Twisted-Mail==8.2.0 Twisted-Names==8.2.0 Twisted-News==8.2.0 Twisted-Runner==8.2.0 Twisted-Web==8.2.0 Twisted-Words==8.2.0 txAMQP==0.6.2 urlgrabber==3.9.1 urllib3==1.10.2 whisper==0.9.10 yolk3k==0.8.7 yum-metadata-parser==1.1.2 zope.interface==4.1.3

The version of graphite-web in this tutorial (0.9.10) has known remote code execution exploits and shouldn’t be used.

@prag_2648: Hi, i am having the same problem you had. Did you find any fix of that problem?
File “manage.py”, line 2, in <module> from django.core.management import execute_manager ImportError: cannot import name execute_manager

@AndrewSB, thanks for the help. I’ll try it and if i have any problem, i’ll tell you!

Andrew SB
DigitalOcean Employee
DigitalOcean Employee badge
June 4, 2014

@rhuankarlus: There’s a more recent article covering installing Graphite on 14.04 here:

https://www.digitalocean.com/community/articles/how-to-install-and-use-graphite-on-an-ubuntu-14-04-server

Please, you need to update this tutorial… So many obsolete parts…

Andrew SB
DigitalOcean Employee
DigitalOcean Employee badge
May 29, 2014

@ravi: You can edit the SECRET_KEY for graphite in the app_settings.py file:

<pre> nano /opt/graphite/webapp/graphite/app_settings.py </pre>

Find the line that looks like:

<pre> SECRET_KEY = ‘’ </pre>

and change it to contain a unique value:

<pre> SECRET_KEY = ‘A_LONG_UNIQUE_STRING’ </pre>

The graphite web app uses Django. You can get more info about the use of SECRET_KEY in Django apps here:

https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECRET_KEY

anyone pls help, when i run “sudo python manage.py syncdb” getting fallowing error
/opt/graphite/webapp/graphite/settings.py:231: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security warn(‘SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security’) Creating tables … Installing custom SQL … Installing indexes … Installed 0 object(s) from 0 fixture(s)

If you encounter a problem with running graphite, with error: “OperationalError: unable to open database file”

Than, all you need to do is: sudo chown -R www-data:www-data /opt/graphite

Enjoy!

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel