We hope you find this tutorial helpful. In addition to guides like this one, we provide simple cloud infrastructure for developers. Learn more →

How to Manage DigitalOcean Spaces with s3cmd

PostedSeptember 20, 2017 5.7k views Object Storage DigitalOcean

A Cheat Sheet

Introduction

s3cmd is a popular cross-platform command-line tool for managing S3 and S3-compatible object stores. This guide provides a quick reference to commands that are useful for managing DigitalOcean Spaces.

How to Use this Guide:

  • This guide is in cheat sheet format with self-contained command-line snippets
  • Jump to any section that is relevant to the task you are trying to complete

Important: If you followed How To Configure s3cmd 2.x To Manage DigitalOcean Spaces
and chose to create a separate profile, remember to append -c ~/nyc3 or whatever you named your specific profile to the end of each command.

Create

Create one or more Spaces: mb

Use the command mb, short for "make bucket", to create a new space:

  • s3cmd mb s3://spacename s3://secondspace

Create Directories

Object stores use a flat namespace and don't have directories. What look like directories in the path of an object are actually part of the object's file name. Graphical interfaces mimic the more familiar experience of directories for their users by creating an empty file for each element that doesn't already exist.

From the command line, there's no need to create empty files. Instead, type the path where you want the file to be as part of the put command.

List

List Spaces

Get a list of all the existing Spaces with ls:

s3cmd ls

List Contents of one or more specific Spaces

List all the contents of a one or more specific Spaces, recursively:

s3cmd ls s3://spacename s3://secondspace

List Contents of all Spaces
List all the content in every Space:

s3cmd la --recursive

Copy

Copy Objects into a Space: put

Use the put command to copy files from your local machine to a Space. In all of these commands, you must include the trailing slash.

Put one file

When you include the trailing slash, as in the example below, the original file name will be appended. If you omit the slash, then the file will be copied to the space with the new name, path.

  • s3cmd put file.txt s3://spacename/path/

Put multiple files

This example shows multiple specific files getting transferred to a Space:

s3cmd put file1.txt file2.txt path/to/file3.txt s3://spacename/path/

Put all the files in your current working directory

Using the * with the put command will copy everything in the current working directory, recursively, into your Space:

  • s3cmd put * s3://spacename/path/ --recursive

Put a file under a new name

You can change the name of a file at the same time you put it in a Space by typing the new name at the end of the path as follows:

  • s3cmd put file.txt s3://spacename/newname.txt

Copy Files to your Local Machine: get

The command get copies files from a Space to your local machine.

Get one file

  • s3cmd get s3://spacename/path/to/file.txt

Get all the files in directory

To get multiple files, the s3 address must end with a trailing slash, and the command requires the --recursive flag.

  • s3cmd get s3://spacename/path/ --recursive

Get a file and save it under a new name

Like the put command, the command get command allows you to give the file a different name.

  • s3cmd get s3://spacename/file.txt newfilename.txt

Set Permissions

Currently the setacl commands are supported by DigitalOcean Spaces.

Set directory listings on a Space

Note that s3cmd only provides output when the command you issue changes tha access. For example, when you change the ACL from private to public, you'll see output like s3://spacename/: ACL set to Public. If the ACL is already public, s3cmd will return silently to the command prompt.

Enable directory listings

  • s3cmd setacl s3://spacename/ --acl-public

Disable directory listings

  • s3cmd setacl s3://spacename/ --acl-private

Set permissions on file

Using the setacl command, files can be made private so that only someone connecting with a valid key pair will be able to read the file, or public so that anyone can read the file with either an S3 compatible client or via HTTPS.

Note that s3cmd only provides output when the command you issue changes tha access. For example, when you change the ACL from private to public, you'll see output like s3://spacename/test.txt: ACL set to Public [1 of 1]. If the ACL is already public, s3cmd will return silently to the command prompt.

Make a file public

  • s3cmd setacl s3://spacename/file.txt --acl-public

Make all the files at a path public, recursively
Use the --recursive flag to make multiple files public recursively:

  • s3cmd setacl s3://spacename/path/to/files/ --acl-public --recursive

Make a file private

  • s3cmd setacl s3://spacename/file.txt --acl-private

Make all the files at a path private, recursively
Use the --recursive flag to make multiple files private recursively:

  • s3cmd setacl s3://spacename/path/to/files/ --acl-private --recursive

Delete

The s3cmd commands del and rm are identical and can be used interchangeably.

Delete a Space

Use rb short for "remove bucket" to delete an empty Space. If you wish to remove the Space and all its contents, append --recursive. This will permanently remove all the files.

  • s3cmd rb s3://spacename

Delete a file

  • s3cmd rm s3://spacename/name/of/file

Delete all files in a Space

Use rm or del with both the --recursive and --force flags to remove all the files in a Space but not the Space itself.

  • s3cmd rm s3://spacename/ --recursive --force

Encrypt a file

Adding the -e or --encrypt flag when you put a file in a Space with s3cmd will encrypt the file to protect if from being read on the server or in transit. If you download the file using s3cmd and the same configuration file, s3cmd will automatically use the password to decrypt it. Other users would need to use gpg -d file.txt to decrypt it and would have to enter the password you supplied. s3cmd allows you to use only one password, so it's suitable for you and for other users with full administrative access. You can learn more about gpg's symmetric encryption in The GNU Privacy Handbook:

s3cmd put s3://path/to/file.txt -e

Conclusion

This guide covered some of the common commands used to manage Spaces with s3cmd version 2.0.0+. There are many other flags and commands that can be used in combination. For a comprehensive guide to what's available, see the s3cmd usage guide.

If there are common commands you think would be useful to add to the guide, please let us know in the comments!

4 Comments

Creative Commons License