s3cmd 2.x Usage

s3cmd is a popular cross-platform command-line tool for managing S3 and S3-compatible object stores. Once you’ve set up s3cmd, you can use it to manage your Spaces and files.

If you’re using an alternative configuration file, you must you must explicitly provide it at the end of each command by appending -c ~/path/to/config/file.

Create Spaces

Use the command mb, short for “make bucket”, to create a new Space:

s3cmd mb s3://spacename s3://secondspace

Object stores use a flat namespace and don’t have directories. What look like directories in the path of an object are actually part of the object’s file name. Graphical interfaces mimic the more familiar experience of directories for their users by creating an empty file for each element that doesn’t already exist.

From the command line, there’s no need to create empty files. Instead, type the path where you want the file to be as part of the put command.

List Spaces and Contents

List all existing Spaces

s3cmd ls

List all the contents of Spaces recursively

For one or more specific Spaces:

s3cmd ls s3://spacename s3://secondspace

List all content in all Spaces

s3cmd la --recursive

Upload Files to a Space

Use the put command to copy files from your local machine to a Space. In all of these commands, you must include the trailing slash.

Put one file

When you include the trailing slash, as in the example below, the original file name will be appended. If you omit the slash, then the file will be copied to the space with the new name, path.

s3cmd put file.txt s3://spacename/path/

Put a file under a new name

You can change the name of a file at the same time you put it in a Space by typing the new name at the end of the path as follows:

s3cmd put file.txt s3://spacename/newname.txt

Put multiple files

s3cmd put file1.txt file2.txt path/to/file3.txt s3://spacename/path/

Put all files in your current directory

Using the * with the put command will copy everything in the current working directory, recursively, into your Space:

s3cmd put * s3://spacename/path/ --recursive

You can set public permisisons for all files at the same time by adding --acl-public, and you can similarly set metadata with --add-header (like --add-header=Cache-Control:max-age=86400):

s3cmd put * s3://yourfolder --acl-public --add-header=Cache-Control:max-age=86400 --recursive

Save Files to your local computer

The command get copies files from a Space to your local computer.

Get one file

s3cmd get s3://spacename/path/to/file.txt

Get all the files in a directory

To get multiple files, the s3 address must end with a trailing slash, and the command requires the --recursive flag.

s3cmd get s3://spacename/path/ --recursive

Save a file under a new name

Like the put command, the command get command allows you to give the file a different name.

s3cmd get s3://spacename/file.txt newfilename.txt

Set Directory Listings on a Space

s3cmd only provides output when the command you issue actually changes access levels. For example, when you change the ACL from private to public, you’ll see output like s3://spacename/: ACL set to Public. If the ACL is already public, s3cmd will return silently to the command prompt.

Enable directory listings

s3cmd setacl s3://spacename/ --acl-public

Disable directory listings

s3cmd setacl s3://spacename/ --acl-private

Set Permissions on Files

Using the setacl command, files can be made private so that only someone connecting with a valid key pair will be able to read the file, or public so that anyone can read the file with either an S3 compatible client or via HTTPS.

Note that s3cmd only provides output when the command you issue changes the access. For example, when you change the ACL from private to public, you’ll see output like s3://spacename/test.txt: ACL set to Public [1 of 1]. If the ACL is already public, s3cmd will return silently to the command prompt.

Make a file public

s3cmd setacl s3://spacename/file.txt --acl-public

Make all the files at a path public recursively

Use the --recursive flag to make multiple files public recursively:

s3cmd setacl s3://spacename/path/to/files/ --acl-public --recursive

Make a file private

s3cmd setacl s3://spacename/file.txt --acl-private 

Make all the files at a path private recursively

Use the --recursive flag to make multiple files private recursively:

s3cmd setacl s3://spacename/path/to/files/ --acl-private --recursive

Delete Spaces and Files

The s3cmd commands del and rm are identical and can be used interchangeably.

Delete a Space

Use rb short for “remove bucket” to delete an empty Space. If you wish to remove the Space and all its contents, append --recursive. This will permanently remove all the files.

s3cmd rb s3://spacename

Delete a file

s3cmd rm s3://spacename/name/of/file

Delete all files in a Space

Use rm or del with both the --recursive and --force flags to remove all the files in a Space but not the Space itself.

s3cmd rm s3://spacename/  --recursive --force

Encrypt Files

Adding the -e or --encrypt flag when you put a file in a Space with s3cmd will encrypt the file to protect if from being read on the server or in transit. If you download the file using s3cmd and the same configuration file, s3cmd will automatically use the password to decrypt it. Other users would need to use gpg -d file.txt to decrypt it and would have to enter the password you supplied.

s3cmd put s3://path/to/file.txt -e

s3cmd allows you to use only one password, so it’s suitable for you and for other users with full administrative access. You can learn more about gpg’s symmetric encryption in The GNU Privacy Handbook.

More Information

For a comprehensive guide to s3cmd usage, see the s3cmd usage guide or access the help file from the command line with s3cmd --help.