Syncing files and directories between servers and local machines is a very common requirement when dealing with networked computers. One method of automatically syncing the contents of directories is with a technology called BitTorrent Sync. This software leverages the BitTorrent protocol that is commonly used for file sharing as a synchronization tool.
Communication through BitTorrent Sync is encrypted end-to-end based on a unique shared secret that is auto-generated. While BitTorrent as a file sharing mechanism is a public service, the way that BitTorrent Sync uses the protocol is private, meaning that files can be transferred securely.
In this guide, we will demonstrate how to install and use BitTorrent Sync on two Ubuntu 14.04 servers. We will show you how to set up your shared directories, and how to set up SSL encryption for the web interface to securely administer your servers.
The first step that we need to get started is to install the BitTorrent Sync software on both of our server instances. Many of the procedures in this guide will be mirrored across both machines, so make sure you duplicate your commands for each machine.
There is no official BitTorrent Sync package available in Ubuntu’s default repositories. However, there is a well-maintained PPA (personal package archive) created by Leo Moll (known as tuxpoldo) that we can use to get up-to-date packages.
On both of your servers, add this PPA so that our systems can pull down the packages:
sudo add-apt-repository ppa:tuxpoldo/btsync
Now, we need to update our local package index so that our systems know about the newly available software. We’ll then install BitTorrent Sync, as well as nginx to add SSL encryption to our web interface later on:
sudo apt-get update
sudo apt-get install btsync nginx
You will be asked quite a few questions in prompts when you attempt to install. For now, press ENTER through all of the prompts. We will be reconfiguring our services momentarily in a more in-depth manner.
Now that the software is installed, we’re actually going to run the configuration script that prompts us for values a second time. This time, however, we will have access to additional options that we require for our purposes.
To run the script again, this time choosing our settings, type this on each server:
sudo dpkg-reconfigure btsync
This will run you through even more prompts than during the initial installation. For the most part, we will be going with the default values and you can just press ENTER.
Below, I’ve outlined the values that you need to configure:
127.0.0.1
admin
account in this example.]password
for demonstration purposes.]002
As you can see, for the vast majority of settings, we can accept the defaults. The above choices though are very important. If you mis-configure these, run the command again to correct your selections.
Now, we have BitTorrent Sync set up for the most part. We will set up our sync directories in a bit. But for now, we need to set up our nginx web server with SSL.
You may have noticed that we configured our web interface to only be available on the local loopback interface (127.0.0.1
). This would normally mean that we would not have access to this when running BitTorrent Sync on a remote server.
We restricted access like this because, although the BitTorrent Sync traffic itself is encrypted, the traffic to the web interface is transmitted in plain text. This could allow anyone watching traffic between our server and local computer to see any communication sent between our machines.
We are going to set up nginx with SSL to proxy connections through SSL to our BitTorrent web interface. This will allow us to securely administer our BitTorrent Sync instance remotely.
Again, we will need to do all of these steps on both of our hosts.
The first step towards getting this set up is to create a directory to hold our SSL certificate and key. We’ll do this under the nginx configuration directory hierarchy:
sudo mkdir /etc/nginx/ssl
Now, we can create our SSL certificate and key in a single motion by issuing this command:
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
You will be asked to fill out some information for your certificate. Fill out the fields as best as you can. The only one that really matters is this one:
Common Name (e.g. server FQDN or YOUR name) []:
In this field, enter your server’s domain name or public IP address.
Now, we can configure our nginx server blocks to use our SSL certificates when communicating with remote clients. It will then the information to our BitTorrent Sync web interface listening on the local interface.
We will leave the default nginx server block file intact in case you need to use this in the future. Since BitTorrent Sync operates on port “8888” by default, we will use this as the front-end SSL port as well.
Create a new server block file by opening a new file with sudo privileges in your editor:
sudo nano /etc/nginx/sites-available/btsync
Inside, we need the to add the following lines:
<pre> server { listen <span class=“highlight”>server_domain_or_IP</span>:8888 ssl; server_name <span class=“highlight”>server_domain_or_IP</span>;
access_log /var/log/nginx/access.log;
ssl_certificate /etc/nginx/ssl/nginx.crt;
ssl_certificate_key /etc/nginx/ssl/nginx.key;
location / {
proxy_pass http://127.0.0.1:8888;
}
} </pre>
Make sure you change the red text to your server’s domain name or public IP address. This will tell nginx to bind to the same port that the BitTorrent Sync web interface is using on the local interface. The difference is that nginx will use the public address and require SSL.
It will use the SSL certificate that we created to encrypt the traffic to the client. It will then pass it to the BitTorrent Sync interface. In this way, the traffic between the server and the client will be encrypted, but the BitTorrent Sync interface will operate as if we were accessing it from the server itself.
When you are finished, save and close the file.
Now, we just need to link the file so that it will be enabled:
sudo ln -s /etc/nginx/sites-available/btsync /etc/nginx/sites-enabled/
We can now restart the service to implement our changes:
sudo service nginx restart
Make sure you go through these procedures on each of your two servers.
We now have BitTorrent Sync configured, and have set up SSL and nginx to encrypt our sessions with the web interface.
Before we begin to use the web interface, we should set up the directories that we want to sync. Because of the way that BitTorrent Sync creates files that it has mirrored from a remote host, our configuration for this portion is pretty important.
First, in this guide, we will be syncing directories located at /shared
on both servers. Let’s create these directories now:
sudo mkdir /shared
Once you have the directory, we are going to give our root account user ownership over the directory. At the same time, we will give the “btsync” group (this was created during the installation) group ownership of the directory:
sudo chown root:btsync /shared
There are many different ways you can configure this access, each with implications. We are demonstrating a fairly flexible system here that will minimize the permissions and ownership conflicts. To find out other alternatives, and their trade-offs, check out the shared folders configuration of this article.
After we assign ownership, we should adjust permissions. We will set the setgid
bit on the directory so that the btsync
group will be given group ownership to any files created in the directory. To make this work correctly, we’ll also need to give the group write permissions:
sudo chmod 2775 /shared
Finally, since our regular system account is not the user owner or group owner of the directory, we will need to add our regular account to the btsync
group. This will allow us to access and interact with the content in this directory as our regular user:
<pre> sudo usermod -a -G btsync <span class=“highlight”>your_user</span> </pre>
Note: At this point, you must log out and log back in for these changes to register in your current environment. Exit by typing:
exit
Now log back in.
Now that we have everything set up, we can begin taking a look at the administrative web interface to pull the pieces together.
To begin, you will need to access both servers in a web browser on port “8888” using the “https” protocol. This should look something like this:
<pre> https://<span class=“highlight”>server_domain_or_IP</span>:8888 </pre>
You will most likely see a warning displayed that looks like this:
This is only a warning telling you that your browser does not recognize the party that signed your SSL certificate. Since we generated self-signed SSL certificates, this makes perfect sense and is expected, and we can safely click “Proceed anyways” or whatever similar button your browser gives you.
You will be prompted for the username and password that you selected while configuring BitTorrent Sync. In our example, the credentials were admin
and password
, but yours (especially the password) may be different.
Once you authenticate, you should see the main BitTorrent Sync Web interface:
We can not begin to add the directory we configured to the web interface.
Click on the “Add Folder” button in the upper-right corner. You will be given a dialog box for adding a directory to the BitTorrent Sync interface:
Scroll to the /shared
directory that we created and click on it. It should populate the “Path” field with the correct value.
Next to the “Secret” field, click on the “Generate” button to create a secret key for the directory:
Click on the “Add” button in the lower-right corner. Your directory will be added to the BitTorrent Sync web UI.
Now, we have a new button available. Click on the “Secret/QR” button associated with the /shared
directory that you just added:
You will be presented with a dialog box that gives you the secret for this directory. This is the way to sync this directory with another instance of BitTorrent Sync.
The software allows you to set up full access to the directory (read and write access), or read-only access. For our guide, we will be configuring full access to allow two-way syncing, but this is simply a preference.
You will need to copy the “Full access” secret from this interface to set up the syncing with your second server.
Now that we have the first server configured to share its directory, we need to set up our second server.
We will go through most of the same steps, with some slight variations.
Once again, sign into the web interface, this time, using the second server’s domain name or IP address. Remember to use “https” and port “8888”:
<pre> https://<span class=“highlight”>second_server_domain_or_IP</span>:8888 </pre>
You will see the SSL warning again, and you will need to authenticate. You will come to the same empty interface that we saw before.
Click on the “Add Folder” button, as we did before. Select the /shared
directory that we created.
At this point, instead of generating a new secret, we want to use the secret that was generated on the first server. This will allow these two instances to communicate, as each secret is unique and randomly generated. Enter the secret from the first server:
Click on the “Add” button in the lower right corner when you are finished.
In a few moments, the “Connected devices and status” column in the main interface will populate with the information about the companion server:
This means that your servers are communicating with each other and can sync content.
Let’s test our current setup.
On either of your servers (it does not matter which one if you configured full access), move into the /shared
directory:
cd /shared
We can will create 10 sample files by typing:
touch file{1..10}
After a moment, on your other server, you should be able to see the files you created:
# On the second server
cd /shared
ls -l
total 0
-rw-rw-r-- 1 btsync btsync 0 May 19 17:07 file1
-rw-rw-r-- 1 btsync btsync 0 May 19 17:07 file10
-rw-rw-r-- 1 btsync btsync 0 May 19 17:07 file2
-rw-rw-r-- 1 btsync btsync 0 May 19 17:07 file3
. . .
As you can see, our files were synced over. If you look at the web interface though, this sync has not registered. This is because these files don’t contain any actual data.
We will test whether it can detect when we transfer files with content by writing data to those files from our second server. This will also allow us to test that we can sync changes back to the first server.
On the second server, you can write the phrase “some content” to each of the files you created by typing:
for item in /shared/file{1..10}; do echo "some content" > $item; done
After a few seconds, the files on the first server should show the content you added:
# On first server
cat /shared/file1
some content
You should also see that the web interface has also been updated to reflect the number of files and the amount of space that has been synced across the servers:
If this is working, you have successfully configured BitTorrent Sync to mirror your changes between servers.
You should now have a flexible setup that allows you to securely transfer files between remote servers. Furthermore, this configuration allows you to administer the service through a secure connection by leveraging SSL.
The application itself is quite flexible and can be used in a variety of ways. Some useful features are the ability to scan secrets as QR codes on your mobile device, the ability to configure read-only access to content, and the ability to provide clients with one-time use secrets. You can also configure your servers to only communicate with certain hosts.
The BitTorrent Sync service also provides a simple version control system, which utilizes a hidden ./SyncArchive
directory in shared directory to keep old versions of files. You can also implement restrictions like rate limiting if you want to make sure that your files are synced without affecting other services.
<div class=“author”>By Justin Ellingwood</div>
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Sign up for Infrastructure as a Newsletter.
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Greetings, all.
So I followed the steps above, and they work beautifully… on my Backupsy.com server. On my DigitalOcean server? Not so much. I get a “Can’t connect to the server” error.
Personally, it think it has something to do with nginx, because if I set btsync to listen on 0.0.0.0:8888 and stop nginx, I can get to the webui. As soon as I turn nginx on, no dice.
I’ve checked the config files for btsync and nginx on both servers, and they are identical. I’m at a total loss on this. Any help would be appreciated!
I ran sudo dpkg-reconfigure btsync but it never asked me for the username and password for the web interface during setup. Now I can’t access it… Any thoughts, guys?
Hi,
I’ve tried for this guide for apache server but i failed. First of all i tried @astarr’s suggesiton but it did not work. At this suggestion says ``` SSLCertificateFile /etc/apache2/ssl/file.pem SSLCertificateKeyFile /etc/apache2/ssl/file.key
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
Listen 8888
<VirtualHost *:8888> ServerName mydomian.com SSLEngine On SSLCertificateFile /etc/apache2/ssl/apache.crt SSLCertificateKeyFile /etc/apache2/ssl/apache.key
</VirtualHost>
Okay, somewhere along the line, I removed my loopback rule in iptables. Just had to add the following rule back in:
This and other cool stuff can be found at the following link.
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-iptables-on-ubuntu-14-04
Can’t get anything on http on 127.0.0.1:8888
Here’s my config file
Getting a timeout error in nginx:
Does the webui port and the listening port have to match?
Anyone?
@it: Latency in the same DC should always be <1ms.
Anyone who uses this, can you tell me what you’re seeing on latency for droplets in the same datacenter? Are we talking milliseconds for replication? Interested for wordpress cluster architecture.
@widrans: You will need to restart apache service to see changing config effect.
@mildpro88: BitTorrent Sync is not what you think it is. Think dropbox without the cloud, direct computer to computer syncing of a folder of any size you want.
@mildpro88: Legal torrents for personal use are fine.