Question

Need guidance on using DigitalOcean API to auto-provision Droplets, with custom domain and SSL cert for each

I’m in the process of building a platform that will make use of DigitalOcean’s APIs to auto-deploy and provision web apps required by our users. The API is well documented and lots of great Community content, but I’ve now hit a few roadblocks and hoping for some pointers from the Community to help resolve them

For reference, we’re primarily using DO’s Python library (python-digitalocean) for any DO API interactions, and can use Fabric (Python SSH) to access the Droplet once created to semi-automated other actions that need completing.

Here’s my end goal, to be achieved purely via API interactions:

  1. Create Droplet using User Data to pre-configure with required packages
  2. Set env.var (either during creation, or by adding a cmd into the User Data that fetches an env.var file from remote URL, secured w/access token in URL)
  3. Create Domain Record and link it with the Droplet
  4. Issue “Let’s Encrypt” SSL cert for the domain record linked to Droplet
  5. Fetch Git repo and install the Node application contained within it
  6. Run Node application

This is the basic config of each Droplet:

  "region": "nyc1",
  "size": "s-2vcpu-2gb",
  "image": "ubuntu-22-10-x64",

Here’s where I’m hitting a brick wall and appreciate any suggestions:

Issue 1: User Data tasks are not being triggered

I’ve tried with a bash script and cloud-config via the API and Dashboard, even stripping the actions back to a simple update and install task.

To verify this, once the Droplet has been created, I connect to the Console and use node -v to confirm whether the Node install task has been run, and node is not found meaning the task has failed or not been run at all.

Here’s the super basic User Data I’m using to try and debug this:

User Data: bash Version

#!/bin/bash

sudo apt update
sudo apt-get install nodejs

User Data: CloudConfig Version

#cloud-config
package_update: true
packages:
	- nodejs

To standardise the way we access Droplet’s from our platform I’d like to assign a domain to each Droplet created. We’ll follow a domain naming pattern that means each ‘Project’ and the Droplet’s within it, exist under a subdomain.

For example: {dropletId}.cluster-1.domain.net

I’ve already setup a wildcard *.cluster-1.domain.net and added DigitalOcean’s nameservers to this, meaning any new subdomain added in DO will automatically work as DO’s nameservers are already handling the traffic.

When I create a new subdomain in the DO Dashboard, I can link that domain directly to a Droplet. As I understand it, this means if the Droplet IP changes DO will ensure all traffic to that domain is re-routed to the new IP.

So firstly, is this correct or when you add a domain and link it to a Droplet via the dashboard, is this just taking the ‘current’ IP of the Droplet and generating the DNS record for simplicity - or will it be dynamic?

If the latter, and it’s dynamically linked, how is it possible to recreate this via the API? I can only see the option to “Create a New Domain” which has a required ip_address param, and no option to set a Droplet (by ID, for example).

Is it therefore possible to link a Domain to a Droplet by ID when using the API, or will the domain need to be linked to an IP address directly, and any Droplet IP update would need to be handled by use to ensure it remains linked with the domain name?

Issue 3: Install and configure new SSL cert for each Droplet

This isn’t really an ‘issue’ so much as being curious about the best way to do it… For each new Droplet we need to install and configure an SSL certificate. Initial thought was to use “Let’s Encrypt” as its free, and to do this via SSH using Fabric (Python lib) once everything has been setup.

However, if there was a company I could buy a wildcard certificate from that isn’t too expensive and would be quicker, easier and more reliable to install onto Droplet’s without needing much config, I’d certainly consider that.

Preference, of course, would be if DigitalOcean had a place to buy one and also then offered some platform-friendly way to install it.

Open to recommendations on this one 🙏


Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

For anyone coming across this post, I’ve made some progress which is outlined below.

Issue 1: User Data tasks are not being triggered

I found and resolved two issues with my user_data script.

**Problem / Resolution 1: ** The first was as simple as incorrect indentation of the apt packages I was installed. I used a tab indent size before each package that I wanted to install, when in-fact it should be just a single space.

Incorrect cloud-config indentation

#cloud-config
# Default: true
package_update: false

# Upgrade the instance on first boot
# Default: false
package_upgrade: true

# Reboot after package install/update if necessary
# Default: false
package_reboot_if_required: true

# Install packages using apt
packages:
	 - nodejs
	 - npm

Correct cloud-config indentation

#cloud-config
# Default: true
package_update: false

# Upgrade the instance on first boot
# Default: false
package_upgrade: true

# Reboot after package install/update if necessary
# Default: false
package_reboot_if_required: true

# Install packages using apt
packages:
packages:
 - nodejs
 - npm

**Problem / Resolution 2: ** I’d got complacent with using a Mac that has homebrew installed as a package manager and was trying to install node instead of nodejs. My basic script didn’t handle errors very well, so as soon as there was an error trying to install node (as that package name doesn’t exist), the rest of the user_data script failed. The solution was to just change the package name to nodejs.

Not solved this one yet. Still looking for suggestions.

Issue 3: Install and configure new SSL cert for each Droplet

I’ve ‘nearly’ solved this one by manually installing an SSL certificate with certbot and then transposing the manual process into the cloud-config file.

The manual process involved following the instructions for “How To Secure Nginx with Let’s Encrypt on Ubuntu 20.04”, which required that I first installed Nginx by following the instructions on “How To Install Nginx on Ubuntu 20.04”. Once I’d installed and configured Nginx and the certbot process was quite simple.

Here’s the latest version of my user_data written as cloud-config. Everything in here works except for the main certbot command which still fails (and means any commands after it are not executed):

- sudo certbot --nginx -d 'yourdomain.com' --non-interactive --agree-tos -m 'hello@yourdomain.com'

If you’re using the user_data below, be sure to remove the this certbot command to see everything else working - then join me in the task of debugging why it won’t work :-)

#cloud-config
# Default: true
package_update: false

# Upgrade the instance on first boot
# Default: false
package_upgrade: true

# Reboot after package install/update if necessary
# Default: false
package_reboot_if_required: true

# For 'apt' specific config, see cloud-config-apt.txt
packages:
 - nodejs
 - npm
 - nginx
 - certbot
 - python3-certbot-nginx
 - gconf-service

# Install Nginx 
# Reference 1: https://learn.microsoft.com/en-us/azure/virtual-machines/linux/tutorial-automate-vm-deployment
# Reference 2: https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-20-04#step-5-%E2%80%93-setting-up-server-blocks-(recommended)
write_files:
  - owner: www-data:www-data
    path: /etc/nginx/sites-available/default
    content: |
      server {
        listen 80;
        listen [::]:80;
        root /home/project_name/myapp/;
        server_name yourdomain.com www.yourdomain.com;
        location / {
          proxy_pass http://localhost:8080;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection keep-alive;
          proxy_set_header Host $host;
          proxy_cache_bypass $http_upgrade;
        }
      }
  - owner: project_name_user:project_name_user
    path: /home/project_name/myapp/index.js
    content: |
      var express = require('express')
      var app = express()
      var os = require('os');
      app.get('/', function (req, res) {
        res.send('Hello World from host ' + os.hostname() + '!')
      })
      app.listen(8080, function () {
        console.log('Hello world app listening on port 8080!')
      })

# Configure Nginx and then install SSL certificate using certbot
# Reference 1: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04
# Reference 2: Run certbot without interaction https://stackoverflow.com/questions/49172841/how-to-install-certbot-lets-encrypt-without-interaction
runcmd:
 - service nginx restart
 - sudo ufw allow 'Nginx HTTP'
 - sudo ln -s /etc/nginx/sites-available/default /home/project_name/myapp/
 - sudo certbot --nginx -d 'yourdomain.com' --non-interactive --agree-tos -m 'hello@yourdomain.com'
 - cd "/home/project_name/myapp"
 - npm init
 - npm install express -y
 - nodejs index.js

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

card icon
Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Sign up
card icon
Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We’d like to help.

Learn more
card icon
Become a contributor

You get paid; we donate to tech nonprofits.

Learn more
Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow – whether you’re running one virtual machine or ten thousand.

Learn more ->
DigitalOcean Cloud Control Panel
Get started for free

Sign up and get $200 in credit for your first 60 days with DigitalOcean.

This promotional offer applies to new account only.

© 2023 DigitalOcean, LLC.