Tutorial

How To Install and Configure Riak2 with Python3 on Ubuntu 14.04

Published on July 14, 2015
Default avatar

By Alvin Wan

AI PhD Student @ UC Berkeley

How To Install and Configure Riak2 with Python3 on Ubuntu 14.04

###Introduction

Riak is a noSQL, non-relational datastore with a focus on distributed database architecture. With the newest Riak 2.1.1 edition, all data can be made strongly consistent, in which data is up-to-date upon retrieval, as opposed to eventually consistent, in which data is more accessible but not up-to-date.

Riak is one choice in a family of key-value noSQL implementations, with competitors including Redis, MemcacheDB, and Aerospike. As a key-value database, it consequently is not optimized for SQL-esque queries that grab an entire dataset.

##Prerequisites

To follow this tutorial, you will need:

  • One fresh Ubuntu 14.04 Droplet

  • A sudo non-root user, which you can setup by following steps 2 and 3 of this tutorial

##Step 1 — Installing Riak

In this section, we will install Riak itself.

With the release of 2.0, Riak has migrated its packages from a self-hosted apt repository to the packagecloud.io service, so we’ll need to populate the apt index with Riak. Luckily, Riak provides a custom script which does just that.

First, we’ll download the script.

  1. curl -O https://packagecloud.io/install/repositories/basho/riak/script.deb.sh

Instead of directly executing it, first open the script to verify that it contains what we expect.

  1. less script.deb.sh

To fetch packages over HTTPS, the script needs to install apt-transport-https package. It also checks for a Certificate Authority, imports a public key, and updates your package index.

Press q to close the file, then execute the script.

  1. sudo bash script.deb.sh

Finally, install Riak.

  1. sudo apt-get install riak=2.1.1-1

Step 2 — Configuring and Launching Riak

In this section, we will configure and launch a Riak node.

To start, we will need to optimize Riak’s Erlang VM with some recommended settings. We will make two modifications: setting queue scan intervals and disabling scheduler compaction of load.

Open the new Riak 2.0’s configuration file using nano or your favorite text editor.

  1. sudo nano /etc/riak/riak.conf

Uncomment the erlang.schedulers.force_wakeup_interval = 500 line, highlighted below. Make sure that the leading space is also deleted, so that the e is the first character of the line.

/etc/riak/riak.conf
. . .

## Set scheduler forced wakeup interval. All run queues will be
## scanned each Interval milliseconds. While there are sleeping
## schedulers in the system, one scheduler will be woken for each
...
## Default: 500
##
## Acceptable values:
##   - an integer
## erlang.schedulers.force_wakeup_interval = 500

. . .

Repeat this process for the erlang.schedulers.compaction_of_load = false in the block directly after:

/etc/riak/riak.conf
. . .

## Enable or disable scheduler compaction of load. By default
## scheduler compaction of load is enabled. When enabled, load
## balancing will strive for a load distribution which causes as many
...
## Default: false
##
## Acceptable values:
##   - one of: true, false
## erlang.schedulers.compaction_of_load = false

. . .

Save and exit the file.

To start a Riak node, run:

  1. sudo riak start

You will see the following.

Output
!!!!
!!!! WARNING: ulimit -n is 1024; 65536 is the recommended minimum.
!!!!

The above message warns that our system has a low open file limit, which restricts the number of open file handles at any given moment. Think of each handle as a writing tool that we own. Every computer process requires a writing tool, to write

By default, the system’s limit on available writing tools is 1024; Riak recommends raising that limit to 65536. To raise this limit, see official Riak Open Files Limit documentation.

To double-check that your node is running, use the following.

  1. sudo riak ping

The command will output pong if the node is running and will return an error otherwise.

To run a sequence of pre-built Riak tests, run:

  1. sudo riak-admin test

This above command will output the following.

Output
Successfully completed 1 read/write cycle to 'riak@127.0.0.1'

Your Riak node is now up and running.

Step 3 — Building an Example Python Applcation (Optional)

The following is an optional series of steps that setup a sample Python-Riak application. The above instructions are language-agnostic and do not depend on the following to function normally. If you are not interested in a sample Python application, you can skip down to the Conclusion section.

First, check your current Python version.

  1. python --version

You should see the output:

Output
Python 2.7.6

We would like to have python run Python 3. So, let’s remove the old binary.

  1. sudo rm /usr/bin/python

Next, create a symbolic link to the Python 3 binary in its place.

  1. sudo ln -s /usr/bin/python3 /usr/bin/python

If you run python --version again now, you’ll see the output Python 3.4.0.

Next, we will install Pip, the recommended package installer for Python packages. Pip allows us to easily manage any Python3 package we would like to have. For an overview of Pip, you can read out this tutorial.

To install it, simply run the following:

  1. sudo apt-get install python3-pip

Now, we need to install the Python-Riak client. Several dependencies need to be satisfied first:

  1. sudo apt-get install python3-dev libffi-dev libssl-dev

Install the client.

  1. sudo pip3 install riak

Finally, we will code a sample application to test the Python-Riak combination. Create a new folder to house the application and create a new file in it.

  1. mkdir ~/test
  2. sudo nano ~/test/app.py

Paste the following inside. This is sample code from the official Riak documentation.

~/test/app.py
import riak

# connect to Riak
myClient = riak.RiakClient(pb_port=8087, protocol='pbc')

# create new Bucket
myBucket = myClient.bucket('test')

# store key-value pairs
val1 = 1
key1 = myBucket.new('one', data=val1)
key1.store()

val2 = "two"
key2 = myBucket.new('two', data=val2)
key2.store()

val3 = {"myValue": 3}
key3 = myBucket.new('three', data=val3)
key3.store()

# fetch the data
fetched1 = myBucket.get('one')
fetched2 = myBucket.get('two')
fetched3 = myBucket.get('three')

print('Value 1 correct: '+str(val1 == fetched1.data))
print('Value 2 correct: '+str(val2 == fetched2.data))
print('Value 3 correct: '+str(val3 == fetched3.data))

Now, run the following to test this application.

python ~/test/app.py

It will output the following warning, but this can be disregarded.

Python application warning
/usr/local/lib/python3.4/dist-packages/riak/security.py:54: UserWarning: Found OpenSSL 1.0.1f 6 Jan 2014 version, but expected at least OpenSSL 1.0.1g. Security may not support TLS 1.2. warnings.warn(msg, UserWarning)

Transport Layer Security (TLS) 1.2 simply a tighter security protocol built on top of TLS 1.1, and TLS in turn is generally an upgrade from SSL. However, Internet Explorer does not universally support TLS 1.1 and 1.2, and TLS 1.2 is disabled in early versions of all popular browsers. As a consequence, we can settle for SSL to govern connections between the application and the Riak datastore safely.

It should output the following:

output
Value 1 correct: True
Value 2 correct: True
Value 3 correct: True

That’s it!

##Conclusion

You have now configured Riak 2 and successfully connected it to Python3. This Riak 2 installation is not specific to Python, however, and may be easily adapted to other languages. For more information on securing Riak, see official Riak 2 recommendations.

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


About the authors
Default avatar
Alvin Wan

author

AI PhD Student @ UC Berkeley

I’m a diglot by definition, lactose intolerant by birth but an ice-cream lover at heart. Call me wabbly, witling, whatever you will, but I go by Alvin


Default avatar

staff technical writer

hi! i write do.co/docs now, but i used to be the senior tech editor publishing tutorials here in the community.


Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
2 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

I am trying to follow your tutorial to install and run a Python program on a RIAK single-node cluster with Ubuntu 14.04 OS. While I try running the python script python ~/test/app.py, I get this huge error message

/usr/local/lib/python3.4/dist-packages/riak/security.py:45: UserWarning: OpenSSL 1.0.1f 6 Jan 2014 (>= 1.0.1g required), TLS 1.2 support: True warnings.warn(msg, UserWarning) Traceback (most recent call last): File “/usr/local/lib/python3.4/dist-packages/riak/client/transport.py”, line 162, in _with_retries raise BadResource(e) riak.transports.pool.BadResource: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/root/test/app.py”, line 14, in <module> key1.store() File “/usr/local/lib/python3.4/dist-packages/riak/riak_object.py”, line 267, in store timeout=timeout) File “/usr/local/lib/python3.4/dist-packages/riak/client/transport.py”, line 223, in wrapper return self._with_retries(pool, thunk) File “/usr/local/lib/python3.4/dist-packages/riak/client/transport.py”, line 172, in _with_retries raise e.args[0] File “/usr/local/lib/python3.4/dist-packages/riak/client/transport.py”, line 153, in _with_retries return fn(transport) File “/usr/local/lib/python3.4/dist-packages/riak/client/transport.py”, line 221, in thunk return fn(self, transport, *args, **kwargs) File “/usr/local/lib/python3.4/dist-packages/riak/client/operations.py”, line 537, in put timeout=timeout) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 151, in put codec = self._get_codec(msg_code) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 76, in _get_codec codec = self._get_pbuf_codec() File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 53, in _get_pbuf_codec self.client_timeouts(), self.quorum_controls(), File “/usr/local/lib/python3.4/dist-packages/riak/transports/feature_detect.py”, line 145, in client_timeouts return self.server_version >= versions[1.4] File “/usr/local/lib/python3.4/dist-packages/riak/util.py”, line 98, in get value = self.fget(obj) File “/usr/local/lib/python3.4/dist-packages/riak/transports/feature_detect.py”, line 197, in server_version return LooseVersion(self._server_version()) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 81, in _server_version server_info = self.get_server_info() File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 115, in get_server_info resp_code, resp = self._request(msg, codec) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/transport.py”, line 549, in _request resp_code, data = self._send_recv(msg_code, data) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/connection.py”, line 36, in _send_recv self._send_msg(msg_code, data) File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/connection.py”, line 65, in _send_msg self._connect() File “/usr/local/lib/python3.4/dist-packages/riak/transports/tcp/connection.py”, line 241, in _connect self._socket = socket.create_connection(self._address) File “/usr/lib/python3.4/socket.py”, line 512, in create_connection raise err File “/usr/lib/python3.4/socket.py”, line 503, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel