In this guide, we will be designating the two of our machines as cluster members and the third as a client.

We will be configuring the computers we labeled as gluster0 and gluster1 as the cluster components. We will use gluster2 as the client.

On our cluster member machines (gluster0 and gluster1), we can install the GlusterFS server package by typing:

sudo apt-get install glusterfs-server

Once this is installed on both nodes, we can begin to set up our storage volume.

On one of the hosts, we need to peer with the second host. It doesn’t matter which server you use, but we will be preforming these commands from our gluster0 server for simplicity:

sudo gluster peer probe gluster1.droplet.com

peer probe: success

This means that the peering was successful. We can check that the nodes are communicating at any time by typing:

sudo gluster peer status

Number of Peers: 1

Hostname: gluster1.droplet.com
Port: 24007
Uuid: 7bcba506-3a7a-4c5e-94fa-1aaf83f5729b
State: Peer in Cluster (Connected)

At this point, our two servers are communicating and they can set up storage volumes together.

Now that we have our pool of servers available, we can make our first volume.

Because we are interested in redundancy, we will set up a volume that has replica functionality. This will allow us to keep multiple copies of our data, saving us from a single point-of-failure.

Since we want one copy of data on each of our servers, we will set the replica option to “2”, which is the number of servers we have. The general syntax we will be using to create the volume is this:

<pre> sudo gluster volume create <span class=“highlight”>volume_name</span> replica <span class=“highlight”>num_of_servers</span> transport tcp <span class=“highlight”>domain1.com:/path/to/data/directory domain2.com:/path/to/data/directory …</span> force </pre>

The exact command we will run is this:

sudo gluster volume create volume1 replica 2 transport tcp gluster0.droplet.com:/gluster-storage gluster1.droplet.com:/gluster-storage force

volume create: volume1: success: please start the volume to access data

This will create a volume called volume1. It will store the data from this volume in directories on each host at /gluster-storage. If this directory does not exist, it will be created.

At this point, our volume is created, but inactive. We can start the volume and make it available for use by typing:

sudo gluster volume start volume1

volume start: volume1: success

Our volume should be online currently.

Now that we have our volume configured, it is available for use by our client machine.

Before we begin though, we need to actually install the relevant packages from the PPA we set up earlier.

On your client machine (gluster2 in this example), type:

sudo apt-get install glusterfs-client

This will install the client application, and also install the necessary fuse filesystem tools necessary to provide filesystem functionality outside of the kernel.

We are going to mount our remote storage volume on our client computer. In order to do that, we need to create a mount point. Traditionally, this is in the /mnt directory, but anywhere convenient can be used.

We will create a directory at /storage-pool:

sudo mkdir /storage-pool

With that step out of the way, we can mount the remote volume. To do this, we just need to use the following syntax:

<pre> sudo mount -t glusterfs <span class=“highlight”>domain1.com</span>:<span class=“highlight”>volume_name</span> <span class=“highlight”>path_to_mount_point</span> </pre>

Notice that we are using the volume name in the mount command. GlusterFS abstracts the actual storage directories on each host. We are not looking to mount the /gluster-storage directory, but the volume1 volume.

Also notice that we only have to specify one member of the storage cluster.

The actual command that we are going to run is this:

sudo mount -t glusterfs gluster0.droplet.com:/volume1 /storage-pool

This should mount our volume. If we use the df command, you will see that we have our GlusterFS mounted at the correct location.

Testing the Redundancy Features

Now that we have set up our client to use our pool of storage, let’s test the functionality.

On our client machine (gluster2), we can type this to add some files into our storage-pool directory:

cd /storage-pool
sudo touch file{1..20}

This will create 20 files in our storage pool.

If we look at our /gluster-storage directories on each storage host, we will see that all of these files are present on each system:

# on gluster0.droplet.com and gluster1.droplet.com
cd /gluster-storage
ls

file1  file10  file11  file12  file13  file14  file15  file16  file17  file18  file19  file2  file20  file3  file4  file5  file6  file7  file8  file9

As you can see, this has written the data from our client to both of our nodes.

If there is ever a point where one of the nodes in your storage cluster is down and changes are made to the filesystem. Doing a read operation on the client mount point after the node comes back online should alert it to get any missing files:

ls /storage-pool

Now that we have verified that our storage pool can be mounted and replicate data to both of the machines in the cluster, we should lock down our pool.

Currently, any computer can connect to our storage volume without any restrictions. We can change this by setting an option on our volume.

On one of your storage nodes, type:

<pre> sudo gluster volume set volume1 auth.allow <span class=“highlight”>gluster_client_IP_addr</span> </pre>

You will have to substitute the IP address of your cluster client (gluster2) in this command. Currently, at least with /etc/hosts configuration, domain name restrictions do not work correctly. If you set a restriction this way, it will block all traffic. You must use IP addresses instead.

If you need to remove the restriction at any point, you can type:

sudo gluster volume set volume1 auth.allow *

This will allow connections from any machine again. This is insecure, but may be useful for debugging issues.

If you have multiple clients, you can specify their IP addresses at the same time, separated by commas:

<pre> sudo gluster volume set volume1 auth.allow <span class=“highlight”>gluster_client1_ip</span>,<span class=“highlight”>gluster_client2_ip</span> </pre>

When you begin changing some of the settings for your GlusterFS storage, you might get confused about what options you have available, which volumes are live, and which nodes are associated with each volume.

There are a number of different commands that are available on your nodes to retrieve this data and interact with your storage pool.

If you want information about each of your volumes, type:

sudo gluster volume info

Volume Name: volume1
Type: Replicate
Volume ID: 3634df4a-90cd-4ef8-9179-3bfa43cca867
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster0.droplet.com:/gluster-storage
Brick2: gluster1.droplet.com:/gluster-storage
Options Reconfigured:
auth.allow: 111.111.1.11

Similarly, to get information about the peers that this node is connected to, you can type:

sudo gluster peer status

Number of Peers: 1

Hostname: gluster0.droplet.com
Port: 24007
Uuid: 6f30f38e-b47d-4df1-b106-f33dfd18b265
State: Peer in Cluster (Connected)

If you want detailed information about how each node is performing, you can profile a volume by typing:

<pre> sudo gluster volume profile <span class=“highlight”>volume_name</span> start </pre>

When this command is complete, you can obtain the information that was gathered by typing:

Cumulative Stats: %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop

  0.00       0.00 us       0.00 us       0.00 us             20     RELEASE
  0.00       0.00 us       0.00 us       0.00 us              6  RELEASEDIR
 10.80     113.00 us     113.00 us     113.00 us              1    GETXATTR
 28.68     150.00 us     139.00 us     161.00 us              2      STATFS
 60.52     158.25 us     117.00 us     226.00 us              4      LOOKUP

Duration: 8629 seconds

Data Read: 0 bytes Data Written: 0 bytes . . . </pre>

You will receive a lot of information about each node with this command.

For a list of all of the GlusterFS associated components running on each of your nodes, you can type:

sudo gluster volume status

Status of volume: volume1
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick gluster0.droplet.com:/gluster-storage             49152   Y       2808
Brick gluster1.droplet.com:/gluster-storage             49152   Y       2741
NFS Server on localhost                                 2049    Y       3271
Self-heal Daemon on localhost                           N/A     Y       2758
NFS Server on gluster0.droplet.com                      2049    Y       3211
Self-heal Daemon on gluster0.droplet.com                N/A     Y       2825
 
There are no active volume tasks

If you are going to be administering your GlusterFS storage volumes, it may be a good idea to drop into the GlusterFS console. This will allow you to interact with your GlusterFS environment without needing to type sudo gluster before everything:

sudo gluster

This will give you a prompt where you can type your commands. This is a good one to get yourself oriented:

help

When you are finished, exit like this:

exit

At this point, you should have a redundant storage system that will allow us to write to two separate servers simultaneously. This can be useful for a great number of applications and can ensure that our data is available even when one server goes down.

<div class=“author”>By Justin Ellingwood</div>