Question

How to mount DigitalOcean Spaces on droplets with s3fs

Posted April 6, 2020 7.1k views
Block StorageDigitalOcean Spaces

I’ve recently got asked how to mount a space on a droplet and I would like to create a separate question here in the community where everyone can share their ideas and valuable information.

1 comment

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
7 answers

Hello, @TsungyingChen

You can mount a DigitalOcean space on your droplet and use it for storage of some of you data. In general, object storage solutions like Spaces are not meant to be accessed like a file system. They are API driven and most often used programmatically. You can check out this tutorial to learn more about object storage and if it’s right for your use case:

https://www.digitalocean.com/community/tutorials/object-storage-vs-block-storage-services#what-is-object-storage

I’ve recently posted a question in the community on how to manage DigitalOcean Spaces using s3cmd:

https://www.digitalocean.com/community/questions/how-to-manage-digitalocean-spaces-using-s3cmd

You can use s3cmd in order to mount the Space to your droplet and from there you can transfer files directly to your Spaces. s3cmd also have a feature to sync directories. In this way you can always make sure that you have all of your latest project files uploaded to your bucket:

You can also use other tools like s3fs in order to mount the space. Also you can mount the Space on multiple droplets, there is no issue in doing that.

You need to do few steps:

1 First is to install s3fs:

sudo apt-get update
sudo apt-get install s3fs

2 Once that is done you need to setup your DigitalOcean Space credentials:

echo <space_key>:<space_token> > ~/.passwd-s3fs
chmod 600 ~/.passwd-s3fs

3 You need to sort the user permissions:

By default, DigitalOcean Space is mounted for root user. Because of that, any files or folders that are created via the web control panel is not accessible by non-root users after it’s mounted, which is not ideal.
To make sure that we can access it via a non-root user, we need to update the config file:

sudo nano /etc/fuse.conf

Uncomment the part that says:

user_allow_other

Next, find out your user’s details by typing the following command:

id

You’ll see an output like this:

uid=1000(sammy) gid=1000(sammy) groups=1000(forge)…

In this case, the user is sammy, so you need to note down the uid and gid of the user for the next step.

4 Mount DigitalOcean Space

First, make a mounting folder:

mkdir <folder>

e.g replace the <folder> with the actual folder you would like to use, the command will be: mkdir space-storage

Then, let’s mount the Space to our new folder:

s3fs <space_name> <folder> -o url=https://sgp1.digitaloceanspaces.com -o use_cache=/tmp -o allow_other -o use_path_request_style -o uid=1000 -o gid=1000

Note: replace the space_name , folder and the uid , gid with your actual space name, created folder and the user IDs received from the id command in step 3

If you want to unmount the space you need execute this command:

fusermount -u <folder>

That’s all for using s3fs with DigitalOcean’s Spaces.

Hope this helps!

Regards,
Alex

by Brian Boucheron
Flexible and scalable data storage is a baseline requirement for most applications and services being developed today. With today's complex deployments, containers, and ephemeral infrastructure, the days of simply saving files to disk on a single server are gone. Cloud providers have developed services to fill the storage needs of modern application deployments, and they mostly fit into two categories: object storage, and block storage.
  • space_test is my folder name

    Example
    echo key:secretkey > ~/.passwd-s3fs
    chmod 600 ~/.passwd-s3fs

    s3fs spacename spacetest -o url=https://sgp1.digitaloceanspaces.com -o usecache=/tmp -o allowother -o usepathrequest_style -o uid=0 -o gid=0

    error show
    s3fs: unable to access MOUNTPOINT space_test: Transport endpoint is not connected

    • Hi, @pranaysoni

      Could you please let me know which version of s3fs that you’re using? You can check the running version using:

      sudo s3fs --version

      Have you tried to unmount and mount the directory again?

      If you want to unmount the space you need execute this command:

      fusermount -u <folder>

      Command to mount

      /usr/bin/s3fs -o allow_other bucketname /s3

      Let me know how it goes.
      Regards,
      Alex

      • s3fs --version
        --------------------------------------------------
        Amazon Simple Storage Service File System V1.79(commit:unknown) with GnuTLS(gcrypt)
        Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
        License GPL2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>
        This is free software: you are free to change and redistribute it.
        There is NO WARRANTY, to the extent permitted by law.
        
        edited by MattIPv4
        • The version seems fine. As mentioned have you tried to unmount and mount the directory again. You can check the commands that I’ve posted above.

          Regards,
          Alex

  • Hi Alex, seems like you copied most of my article from https://jianjye.medium.com/how-to-mount-digitalocean-spaces-to-droplets-on-ubuntu-dcba4cc16c78 and pasted it here. Appreciate at least some credits where it’s due.

    • Hello, @jianjye

      Thanks for reaching out. I was using the article as a reference point, also some of the steps are described exactly as in your post that is correct. I believe that the article should be listed here in the community as well. Kudos for the great post. Also, feel free to post other DigitalOcean related products articles here.

      Regards,
      Alex

  • I’ve followed this indications but I can’t access to the subfolder inside the spaces.

    I see this message when I use cd sub-folder-name:

    -bash: cd: sub-folder-name: Not a directory
    

    I’m sure that sub-folder-name is a directory becouse I’ve created it via DigitalOcean’s panel and I’ve putted inside it a file.

    • Hello there,

      Can you please check out the bash history and see if the folder had another name or if it was created in another directory, e.g in /root or your home dir - ~/$USER ?

      Regards,
      Alex

      • I’m using root user for create that folder and I’ve created it inside root/.
        Inside root/ there is folder-name and I use that folder here:
        s3fs <space_name> <folder-name> -o url=https://code.digitaloceanspaces.com -o use_cache=/tmp -o allow_other -o use_path_request_style -o uid=1000 -o gid=1000

        After this, inside folder-name I can see subfolder-name but it seems isn’t a folder. The problem is that I’ve created subfolder-name via Spaces panel, moreover I’ve putted inside it a jpg.

        Hope is useful

        root@GeoServerDrako:~# cd do-spaces/
        root@GeoServerDrako:~/do-spaces# ls -la
        total 5
        drwxrwxrwx 1 root root    0 Jan  1  1970 .
        drwx------ 8 root root 4096 Mar 21 18:13 ..
        -rwxr-x--- 1 root root    0 Mar 21 18:26 goeserver-data-folder
        root@GeoServerDrako:~/do-spaces# cd geoserver-data-folder
        -bash: cd: geoserver-data-folder: No such file or directory
        
        
        • Same problem with non root user

          drwxr-xr-x 3 drako drako 4096 Mar 22 17:31 .
          drwxr-xr-x 3 root  root  4096 Mar 22 17:28 ..
          -rw-r--r-- 1 drako drako  220 Mar 22 17:28 .bash_logout
          -rw-r--r-- 1 drako drako 3771 Mar 22 17:28 .bashrc
          -rw-r--r-- 1 drako drako    0 Mar 22 17:28 .cloud-locale-test.skip
          -rw------- 1 drako drako   65 Mar 22 17:30 .passwd-s3fs
          -rw-r--r-- 1 drako drako  807 Mar 22 17:28 .profile
          -rw-r--r-- 1 drako drako    0 Mar 22 17:31 .sudo_as_admin_successful
          drwxrwxrwx 1 drako drako    0 Jan  1  1970 do-spaces
          drako@GeoServerDrako:~$ cd do-spaces/
          drako@GeoServerDrako:~/do-spaces$ ls -la
          total 5
          drwxrwxrwx 1 drako drako    0 Jan  1  1970 .
          drwxr-xr-x 3 drako drako 4096 Mar 22 17:31 ..
          -rwxr-x--- 1 drako drako    0 Mar 21 18:26 goeserver-data-folder
          drako@GeoServerDrako:~/do-spaces$ cd geoserver-data-folder
          -bash: cd: geoserver-data-folder: No such file or directory
          drako@GeoServerDrako:~/do-spaces$ 
          
          
        • Hello there,

          Yes, I can that goeserver-data-folder is actually a file rather than a directory:

          -rwxr-x--- 1 drako drako    0 Mar 21 18:26 goeserver-data-folder
          

          You can manually create this folder if you wish, either via ssh or via the spaces menu in your control panel.

          Regards,
          Alex

          • Ok, thank you.
            It’s strange that I can create folder, subfolder and files without problems using terminal, but if I use the spaces menu the folders become files when I try to navigate inside of their via terminal.

This question was answered by @pflynn02:

Hello @alexdo - I was able to get this implemented but every time I power cycle it loses this mount. Is there a way to make it automatically remount after a droplet restart?

View the original comment

  • Hi, @pflynn02

    You need to alter the /etc/fstab file in order to make the space loaded into the operating system’s file system table.

    You can open the file in your favourite file editor and enter the following line:

    s3fs#SPACE_NAME /path_to_moint_point fuse allow_other,_netdev,nosuid,nodev,url=https://nyc3.digitaloceanspaces.com 0 0`
    

    an alternative one is:

    s3fs SPACE_NAME /path_to_mount_point -o url=https://nyc3.digitaloceanspaces.com -o allow_other
    

    Note: Make sure to change SPACENAME with the actual Space name and pathtomointpoint with the correct path to the mount point.

    Hope that this helps!
    Regards,
    Alex

Hi, @pflynn02

You need to alter the /etc/fstab file in order to make the space loaded into the operating system’s file system table.

You can open the file in your favourite file editor and enter the following line:

s3fs#SPACE_NAME /path_to_moint_point fuse allow_other,_netdev,nosuid,nodev,url=https://nyc3.digitaloceanspaces.com 0 0`

an alternative one is:

s3fs SPACE_NAME /path_to_mount_point -o url=https://nyc3.digitaloceanspaces.com -o allow_other

Note: Make sure to change SPACENAME with the actual Space name and pathtomointpoint with the correct path to the mount point.

Hope that this helps!
Regards,
Alex

Hello, @massimilianomoraca

That is really weird. Would you mind trying to replicate this using an alternative browser and if the issue persists you can also try reaching out to our support team?

Regards,
Alex

Hello @massimilianomoraca

Thanks for sharing this information. I also believe other users can find it useful if they run into the same situation.

Regards,
Alex