How To Configure s3cmd 2.x To Manage DigitalOcean Spaces
Welcome to the DigitalOcean Spaces. In this article, we'll show you how to install, configure, and manage DigitalOcean Spaces with the command-line tool s3cmd.
To use this 3rd party client, you will need:
- A working installation of s3cmd version 2.0.0+. Note that package managers may be out of date. To get this version, visit the s3cmd Download page.
- An access key pair for your Spaces. To generate these, visit the API page in the DigitalOcean Control Panel. To learn more about access keys, see the Understanding Permissions and Access Keys with DigitalOcean Spaces.
When you have completed the prerequisites, you're ready to follow along.
Verify the Version
We'll begin by confirming our version of s3cmd: with the following command:
- s3cmd --version
s3cmd version 2.0.0+
By verifying that we're running the same version, we can expect that the output of our commands will match the diretions that follow.
Choose the Configuration File
By default, s3cmd stores its configuration file,
.s3cfg, in the home directory of the user who runs the configuration command. This is a plain text file of key/value pairs which can be edited directly once it has been created. You can choose between setting up DigitalOcean as the default configuration or creating a different configuration file:
Option 1: Make DigitalOcean the default connection
If DigitalOcean is the main or only provider you'll connect to with
s3cmd, configure it in the default
~/.s3cfg file with the following command:
By choosing this option, you won't have to specify the configuration file each time you run a command.
Option 2: Create an explicit configuration file for DigitalOcean
If you're already using s3cmd with another service, you may want to create an alternate configuration file, which you can do by adding the
-c flag and supplying a filename. The configuration file will be created in the directory where you issue the command, so specify the path if you want it created elsewhere.
In this example, we'll place a config file named
nyc3in our home directory:
- s3cmd --configure -c ~/nyc3
Important: To use this configuration file, it must be explicitly provided at the end of each command by appending
Whether you use the default configuration file or specify your own, when you issue the configuration command, it will launch an interactive multi-step configuration script. In this section, we'll go through each of the steps in the configuration process.
Enter Access Keys
The script begins by asking for an Access Key and Secret Key. If you don't already have keys, you can generate a set for s3cmd by visiting the Control Panel's API page.
We'll enter our keys, then accept
US for the Default Region since the region information isn't relevant to DigitalOcean.
Enter new values or accept defaults in brackets with Enter. Refer to user manual for detailed description of all options. Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables. Access Key : EXAMPLE7UQOTHDTF3GK4 Secret Key : b8e1ec97b97bff326955375c5example Default Region [US]:
Note: If you prefer, you can use the environment variables
AWS_SECRET_ACCESS_KEY to store a set of keys.
Set the Endpoint
Next, we'll enter the DigitalOcean endpoint,
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3. S3 Endpoint [s3.amazonaws.com]: nyc3.digitaloceanspaces.com
Since Spaces supports DNS-based buckets, at the next prompt, we'll supply the bucket value in the required format:
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars c an be used if the target S3 system supports dns based buckets. DNS-style bucket+hostname:port template for accessing a bucket : %(bucket)s.nyc3.digitaloceanspaces.com
Optional: Set an encryption password
Next, we're asked to supply an encryption password. Unlike HTTPS, which protects file only while in transit, GPG encryption prevents others from reading files while they are stored on DigitalOcean as well as in transit. Setting a password now won't cause objects to be automatically encrypted; that's done by adding a
-e flag to the
put command when you copy the object to a Space.
We'll enter a password so it will be available in the event we want to use encryption.:
Encryption password is used to protect your files from reading by unauthorized persons while in transfer to S3 Encryption password: secure_password Path to GPG program [/usr/bin/gpg]:
Connect via HTTPS
Next, we're prompted to connect via HTTPS, which protects data from being read while it is in transit. DigitalOcean Spaces do not support unencrypted transfer, so we'll press
ENTER to accept the default,
When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer Use HTTPS protocol [Yes]: Yes'
Optional: Set a Proxy Server
If your network requires you to use an HTTP Proxy server, enter its IP address or domain name without the protocol, e.g.
proxy.example.com Since we aren't using an HTTP Proxy server, we'll leave this question blank and press
On some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly HTTP Proxy server name:
Confirm and test settings
After the prompt for the HTTP Proxy server name, the configuration script presents a summary of the values it will use, followed by the opportunity to test them:
New settings: Access Key: EXAMPLES7UQOTHDTF3GK4 Secret Key: b8e1ec97b97bff326955375c5example Default Region: US S3 Endpoint: nyc3.digitaloceanspaces.com DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.n yc3.digitaloceanspaces.com Encryption password: secure_password Path to GPG program: /usr/bin/gpg Use HTTPS protocol: True HTTP Proxy server name: HTTP Proxy server port: 0 Test access with supplied credentials? [Y/n] Y
Save the settings
When the test completes successfully, enter
Y to save the settings:
Please wait, attempting to list all buckets... Success. Your access key and secret key worked fine :-) Now verifying that encryption works... Success. Encryption and decryption worked fine :-) Save settings? [y/N] Y
If the test fails or you choose
N you'll have the opportunity to retry the configuration. Once you save the configuration, you'll receive confirmation of its location:
- Configuration saved to '/home/sammy/nyc3'
The test step confirms that we can connect to our DigitalOcean Spaces.
In this tutorial we've configured and tested the