Tutorial

Automated Provisioning of DigitalOcean Cloud Servers with Salt Cloud on Ubuntu 12.04

Published on October 28, 2013
Default avatar

By Pablo Carranza

Automated Provisioning of DigitalOcean Cloud Servers with Salt Cloud on Ubuntu 12.04

Status: Deprecated

This article covers a version of Ubuntu that is no longer supported. If you are currently operate a server running Ubuntu 12.04, we highly recommend upgrading or migrating to a supported version of Ubuntu:

Reason: Ubuntu 12.04 reached end of life (EOL) on April 28, 2017 and no longer receives security patches or updates. This guide is no longer maintained.

See Instead: This guide might still be useful as a reference, but may not work on other Ubuntu releases. If available, we strongly recommend using a guide written for the version of Ubuntu you are using. You can use the search functionality at the top of the page to find a more recent version.

Introduction


This tutorial builds on How To Install Salt on Ubuntu 12.04 | DigitalOcean and aims to provide the reader with a simplified, basic setup of an operable Salt Cloud master server. DigitalOcean users can then use this employ to automate the process of creating 1, 2 or a fleet of cloud servers.

Total Infrastructure Management


Salt Stack is an open-source cloud deployment, configuration management, remote execution & monitoring package. You may not have heard of Salt, but you might be familiar with Puppet or Chef. Salt is a similar tool, but it is relatively lightweight in terms of resources and requirements, and a growing number of users consider it much easier to use. Tools like Salt, Puppet and Chef allow you to issue commands on multiple machines at once, as well as install and configure software. Salt has two main aspects: (i) configuration management and (ii) remote execution. Cloud deployment is left to Salt Cloud.

Salt Cloud is a public-cloud provisioning tool designed to automate the deployment of public cloud servers. It integrates Salt with DigitalOcean’s application programming interface (API) in a clean way – and (i) creates; (ii) provisions & (iii) manages droplets (i.e. cloud or virtual private servers) via virtual machine maps and profiles. DigitalOcean droplets can be created individually or in large groups and can be provisioned and fully managed, without ever needing to be logged into. From deploying single virtual machines, to mapping and provisioning entire clouds, Salt Cloud is as scalable as you need it to be.

Prerequisites


First, consult How To Install Salt on Ubuntu 12.04 | DigitalOcean and create a Salt master/minion on your VPS.

<h3>Security Hardening</h3>

Any server accessible from the public Internet should be security hardened and your Salt master is no exception:

The Salt master communicates with the minions using an AES-encrypted ZeroMQ connection. These communications are done over TCP ports 4505 and 4506, which need to be accessible on the master only.

  • Configure a firewall and make sure to open your custom SSH port and TCP Ports 4505 & 4506;

The default firewall configuration tool for Ubuntu is ufw. To open the Salt ports, simply execute:

		sudo ufw allow [custom SSH port below 1024]/tcp
		sudo ufw allow salt
		sudo ufw enable
		sudo ufw status verbose

Either disable password logins or deploy Fail2ban & DenyHosts.

<h3>Create SSH Key Pair for DigitalOcean Control Panel</h3>

Salt Cloud uses public key encryption to secure the communication between the Salt master and DigitalOcean. Therefore, create a directory on your master in which to store your SSH keys:

sudo mkdir /keys

Next, execute:

sudo ssh-keygen -t rsa

The system will respond with <code>Enter file in which to save the key (/root/.ssh/id_rsa):</code>. Copy and paste:

/keys/digital-ocean-salt-cloud

Press “Enter” on your keyboard. The system will display <b>Enter passphrase (empty for no passphrase):</b>, asking you to enter an optional password. Do not enter a passphrase; instead, tap the “Enter” key twice.

Your new public (SSH) key is now located in “keys/digital-ocean-salt-cloud.pub.” Finally, execute:

cat /keys/digital-ocean-salt-cloud.pub

Copy and paste the public key into your DigitalOcean Control Panel, as outlined in Step Three of How To Use SSH Keys with DigitalOcean Droplets (save the name of the public key in your Control Panel as <b>digital-ocean-salt-cloud.pub</b>).

Hostname and Fully Qualified Domain Name (FQDN)


Verify that your Salt master’s hostname and FQDN are properly set. See Setting the Hostname and Fully Qualified Domain Name (FQDN) on Ubuntu 12.04.

Acquire Installation Tools


Salt Stack is built with the Python programming language; so, we’ll need <b>pip</b> (a package management system used to install and manage software packages written in Python). In addition, despite being available via Python’s repositories, we’ll be downloading the Salt Cloud package from Salt Stack’s GitHub repository.

To install <code>pip</code> and <code>git</code> on your system, execute:

sudo apt-get -y install python-pip git

Install Salt Cloud Dependencies


First, execute:

sudo apt-get -y install python-m2crypto

Then, execute:

sudo pip install pyzmq PyYAML pycrypto msgpack-python jinja2 psutil salt

Next, execute:

sudo pip install apache-libcloud

Install Salt Cloud


Finally, execute:

sudo pip install git+https://github.com/saltstack/salt-cloud.git#egg=salt_cloud

Verify Salt Cloud was successfully installed by executing:

salt-cloud --version

Configure Salt Cloud


Salt Cloud runs on a module system similar to the main Salt project; and, by default, uses PyYAML syntax for its template files – but numerous other templating languages are available as well. When creating your configuration files, be sure to follow the proper formatting techniques for YAML, which involves two spaces instead of tabs.

An online YAML parser is available when troubleshooting syntax issues with YAML files.

I. Core Configuration


The core configuration of Salt Cloud is handled in the cloud configuration file. This file is comprised of global configurations that affect all cloud providers in your system. In addition, the cloud file is where the minions that are created derive their configuration.

Create the cloud configuration file by executing (obviously, you can use whichever text editor you wish; but this guide assumes that you have installed the vim text editor):

sudo vim /etc/salt/cloud

On your keyboard, tap on the <b>i</b> key; use the arrow keys to navigate the text area; and copy & paste or create your cloud file so that it resembles the example below (replacing <code>master.yourdomain.tld</code> with the FQDN of your Salt master):

provider: do
# Set the location of the Salt master
minion:
  master: master.yourdomain.tld

To save and exit, tap the <b>Esc</b> key on your keyboard, followed by these keystrokes: 1) : 2) w 3) q 4) Enter.

For further customization, additional miscellaneous options, that can be passed in the core configuration file are available.

Cloud Provider Modules


Next, create two new directories:

sudo mkdir /etc/salt/{cloud.profiles.d,cloud.providers.d}

These new directories will hold the DigitalOcean-specific YAML configuration files.

II. DigitalOcean Cloud Provider Configuration


The DigitalOcean cloud provider configuration is used to control access to your DigitalOcean account. Create the DigitalOcean cloud provider configuration file by executing:

sudo vim /etc/salt/cloud.providers.d/digital_ocean.conf

Tap on the <b>i</b> key; use the arrow keys on your keyboard to navigate the text area; and copy & paste or create your digital_ocean.conf file so that it resembles the example below:

do:
  provider: digital_ocean
  # Digital Ocean account keys
  client_key: YourClientIDCopiedFromControlPanel
  api_key: YourAPIKeyCopiedFromControlPanel
  ssh_key_name: digital-ocean-salt-cloud.pub
  # Directory & file name on your Salt master
  ssh_key_file: /keys/digital-ocean-salt-cloud

To save and exit, tap the <b>Esc</b> key on your keyboard, followed by these keystrokes: 1) : 2) w 3) q 4) Enter.

DigitalOcean API


Using Salt Cloud with DigitalOcean requires a <b>client_key</b> and an <b>api_key</b>. These can be found in the DigitalOcean Control Panel, under the “API Access” tab.

Now, create an API key for your account by following the steps outlined in How To Use the DigitalOcean API. You will need to copy and paste both your DigitalOcean <code>Client ID</code> and <code>API Key</code> in the cloud provider configuration file, described above.

Interacting with the API


After you configure the DigitalOcean provider in <code>/etc/salt/cloud.providers.d/digital_ocean.conf</code>, you gain access to the following commands:

sudo salt-cloud --list-images do
sudo salt-cloud --list-sizes do
sudo salt-cloud --list-locations do
sudo salt-cloud --help

The output from these commands are important because it provides the variables needed to build our virtual server profiles.

III. DigitalOcean Cloud Profile


Create the DigitalOcean cloud profiles for your server fleet by executing:

sudo vim /etc/salt/cloud.profiles.d/digital_ocean.conf

Now, on your keyboard tap on the <b>i</b> key; use the arrow keys to navigate the text area; and copy & paste or create your digital_ocean.conf file so that it resembles the example below:

# Official distro images available for Arch, CentOS, Debian, Fedora, Ubuntu

ubuntu_512MB_ny2:
  provider: do
  image: Ubuntu 12.04.4 x64
  size: 512MB
#  script: Optional Deploy Script Argument
  location: New York 2
  private_networking: True

ubuntu_1GB_ny2:
  provider: do
  image: Ubuntu 12.04.4 x64
  size: 1GB
#  script: Optional Deploy Script Argument
  location: New York 2
  private_networking: True

ubuntu_2GB_ny2:
  provider: do
  image: Ubuntu 12.04.4 x64
  size: 2GB
#  script: Optional Deploy Script Argument
  location: New York 2
  private_networking: True

# Create additional profiles, if you wish
#[profile_alias_of_your_choosing]:
#  provider: do
#  image: [from salt-cloud --list-images do]
#  size: [from salt-cloud --list-sizes do]
#  script: [optional deployment script e.g. Ubuntu, Fedora, python-bootstrap, etc.]
#  location: [from salt-cloud --list-locations do]
#  private_networking: [True or False: currently only available in NY2 region]

To save and exit, tap the <b>Esc</b> key on your keyboard, followed by these keystrokes: 1) : 2) w 3) q 4) Enter.

In addition to the examples provided, Salt Cloud can accommodate multiple configuration files, which allows for more extensible configuration and plays nicely with various configuration management tools, as well as version control systems.

OS Support for Cloud VMs


Salt Cloud works primarily by executing a script on the newly-provisioned droplets as soon as they become available. By default, the script that is used is the salt-bootstrap script, unless a different deploy script is declared in the cloud profile. The preferred method (as of Salt Cloud v0.8.9) is currently to use the default salt-bootstrap script. If the salt-bootstrap script does not meet your needs, you may pass Deploy Script Arguments or write your own.

Advanced Topic


A number of options exist when creating your VPS that are beyond the scope of this article. After you feel comfortable with the foundational principles outlined in this tutorial, you may want to learn about creating a more complex setup with a map file. The map file allows for a number of virtual machines to be created and associated with specific profiles.

Provision a New Cloud Server!


To create a new cloud server, execute (replacing <b>hostname</b> with any hostname of your choice):

sudo salt-cloud --profile ubuntu_512MB_ny2 hostname

If all goes well, you should have a newly-provisioned server, bootstrapped with Salt minion (the new minion’s SSH keys will automatically be added to the Salt master). If you would like to provision multiple virtual machines from the same profile, you can do so with a single command, e.g.

sudo salt-cloud -p ubuntu_1GB_ny2 hostname1 hostname2 hostname3

(Note that <code>–profile</code> and <code>-p</code> are interchangable.)

Minion Configuration


To configure your new fleet of cloud servers, consult: How To Create Your First Salt Formula | DigitalOcean.

Destroy a Minion


There are various options that can be passed when executing a Salt Cloud command. For example, to destroy a particular minion simply execute:

sudo salt-cloud -d hostname

Additional Resources


As always, if you need help with the steps outlined in this How-To, look to the DigitalOcean Community for assistance by posing your question(s) below.

<div class=“author”>Submitted by: <a href=“https://plus.google.com/107285164064863645881?rel=author”>Pablo Carranza</a></div>

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


About the authors
Default avatar
Pablo Carranza

author

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
10 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

After running install-salt-cloud.sh and configure-digital-ocean.sh, I was able to successfully perform salt-cloud --list-sizes digitalocean today. Maybe someone else will find this solution for v2 API access helpful :)

Source: https://github.com/billyjf/digital-ocean-jenkins-dev-environment

apt-get install salt-cloud worked for me. had troubles with pip/git

Great guide, one note. I found that I need python-dev in order to install psutil.

Will this get an update anytime soon on how to use APIv2? I’m currently still using client_key and api-key. While the bearer should be used now on the API.

Some images also just disappeared form the --list-images. eg I can’t find a debian-7-0-x64 anymore. Even when I try to use the id “6372526” it says couldn’t find image. the x32 version with id “6372528” works though. Anyone knows a reason for this, maybe even a solution?

The Salt Cloud repo is now deprecated and it has been merged into the salt repo. You can read about it here https://github.com/saltstack/salt-cloud

Forgot to say that the initial provision comes across correctly as the hostname used in the provision and salt master accepts the keys and stores the keys as the provisioned hostname.

I’ve tested this with CentOS 6.5 and this works great. Unfortunately, Fedora 20 doesn’t seem to set the proper hostname on the salt-cloud provision. So, every prior call to the salt highstate comes across as “localhost.localdomain” instead of the intended hostname. This seems to be a digital ocean problem since I didn’t have this with EC2.

Andrew SB
DigitalOcean Employee
DigitalOcean Employee badge
May 7, 2014

@Kevin: Thanks! I’ve update the article to reflect that.

When creating the digital_ocean.conf file, it has the image “Ubuntu 12.04 x64” as the preferred choice. This image no longer appears to exist, and I got an error while trying to provision it.

Run “salt-cloud --list-images do” and you will see that the image has been updated to “Ubuntu 12.04.4 x64”, so adjust your digital_ocean.conf file accordingly.

Cheers for the great tutorial - my Droplet is up and running in no time flat.

Let’s say that I have my own image (i.e snaphot) and it has disabled ssh for root user. Then If I want to create a droplet using that image, salt-cloud its gonna fail due to it’s trying to connect through ssh using root.

In this case, the droplet will be created, but salt-minion it’s not installed. Maybe in this case the option would be to create a custom deploy script instead to use the default one, right?

Thanks!

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel