Question

cloud-init: change order of module execution

Hi folks,

since it’s not documented anywhere, (https://www.digitalocean.com/community/tutorials/an-introduction-to-cloud-config-scripting / http://cloudinit.readthedocs.io/en/latest/topics/modules.html) and I got to find out the hard way that write_files runs after runcmd, i was wondering if it’s possible to change the order in which cloud-init modules are executed?

The reasoning is that i will frequently need to place a bunch of files in place (config files, keys, etc) before being able to successfully run a command.

Subscribe
Share

Submit an answer
You can type!ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Hi @me554b5ddf77bd4849acbdb294

I’ve just tried to spin up two droplets (FreeBSD and Ubuntu) with this:

#cloud-config
write_files:
  - path: /root/test.txt
    content: |
      This is the write_files and should be line 1 of 2
runcmd:
 - echo "This is the runcmd and should be line 2 of 2" >> /root/test.txt

And this file /root/test.txt looks like this after droplet creation on Ubuntu (correct):

This is the write_files and should be line 1 of 2
This is the runcmd and should be line 2 of 2

But on FreeBSD it looks like this (wrong):

This is the runcmd and should be line 2 of 2

That’s because write-files is not included in stage cloud_init_modules on FreeBSD.

Can you create a ticket to Support via the control panel, where you link to this thread. It seems like a bug, but I’m not sure if the FreeBSD cloud-init is just limited a bit.

FreeBSD /usr/local/etc/cloud/cloud.cfg

# Written by the DigitalOcean provisioning process
syslog_fix_perms: root:wheel

datasource_list: ['ConfigDrive', 'DigitalOcean', None]

disable_root: true

# Create the default user
users:
 - default

# This will cause the set+update hostname module to not operate (if true)
preserve_hostname: false

# The modules that run in the 'init' stage
cloud_init_modules:
 - seed_random
 - bootcmd
 - set_hostname
 - update_hostname
 - users-groups
 - ssh

# The modules that run in the 'config' stage
cloud_config_modules:
 - ssh-import-id
 - locale
 - runcmd

# The modules that run in the 'final' stage
cloud_final_modules:
 - rightscale_userdata
 - scripts-per-once
 - scripts-per-boot
 - scripts-per-instance
 - scripts-user
 - ssh-authkey-fingerprints
 - phone-home
 - final-message
 - power-state-change

# Do not consume vendor-data
vendor_data:
   enabled: false

system_info:
   distro: freebsd
   default_user:
     name: freebsd
     lock_passwd: false
     gecos: FreeBSD
     groups: [wheel]
     sudo: ["ALL=(ALL) NOPASSWD:ALL"]
     shell: /bin/sh

Ubuntu /etc/cloud/cloud.cfg

# The top level settings are used as module
# and system configuration.

# A set of users which may be applied and/or used by various modules
# when a 'default' entry is found it will reference the 'default_user'
# from the distro configuration specified below
users:
   - default

# If this is set, 'root' will not be able to ssh in and they 
# will get a message to login instead as the above $user (ubuntu)
disable_root: true

# This will cause the set+update hostname module to not operate (if true)
preserve_hostname: false

# Example datasource config
# datasource: 
#    Ec2: 
#      metadata_urls: [ 'blah.com' ]
#      timeout: 5 # (defaults to 50 seconds)
#      max_wait: 10 # (defaults to 120 seconds)

# The modules that run in the 'init' stage
cloud_init_modules:
 - migrator
 - seed_random
 - bootcmd
 - write-files
 - growpart
 - resizefs
 - set_hostname
 - update_hostname
 - update_etc_hosts
 - ca-certs
 - rsyslog
 - users-groups
 - ssh

# The modules that run in the 'config' stage
cloud_config_modules:
# Emit the cloud config ready event
# this can be used by upstart jobs for 'start on cloud-config'.
 - emit_upstart
 - disk_setup
 - mounts
 - ssh-import-id
 - locale
 - set-passwords
 - grub-dpkg
 - apt-pipelining
 - apt-configure
 - package-update-upgrade-install
 - landscape
 - timezone
 - puppet
 - chef
 - salt-minion
 - mcollective
 - disable-ec2-metadata
 - runcmd
 - byobu

# The modules that run in the 'final' stage
cloud_final_modules:
 - rightscale_userdata
 - scripts-vendor
 - scripts-per-once
 - scripts-per-boot
 - scripts-per-instance
 - scripts-user
 - ssh-authkey-fingerprints
 - keys-to-console
 - phone-home
 - final-message
 - power-state-change

# System and/or distro specific settings
# (not accessible to handlers/transforms)
system_info:
   # This will affect which distro class gets used
   distro: ubuntu
   # Default user name + that default users groups (if added/used)
   default_user:
     name: ubuntu
     lock_passwd: True
     gecos: Ubuntu
     groups: [adm, audio, cdrom, dialout, dip, floppy, netdev, plugdev, sudo, video]
     sudo: ["ALL=(ALL) NOPASSWD:ALL"]
     shell: /bin/bash
   # Other config here will be given to the distro class and/or path classes
   paths:
      cloud_dir: /var/lib/cloud/
      templates_dir: /etc/cloud/templates/
      upstart_dir: /etc/init/
   package_mirrors:
     - arches: [i386, amd64]
       failsafe:
         primary: http://archive.ubuntu.com/ubuntu
         security: http://security.ubuntu.com/ubuntu
       search:
         primary:
           - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
           - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
           - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
         security: []
     - arches: [armhf, armel, default]
       failsafe:
         primary: http://ports.ubuntu.com/ubuntu-ports
         security: http://ports.ubuntu.com/ubuntu-ports
   ssh_svcname: ssh

Ran into this as well on ubuntu 18.04 :(

On other clouds (AWS) this works as expected (write_files before runcmd) for standard ubuntu 18.04

You can do something like this:

runcmd:
  - |
    set -x
    (
      while [ ! -f /path/to/my-script.sh ]; do
        sleep 1
      done
      /path/to/my-script.sh
    ) &

This should work regardless of whether write_files or runcmd happens first, but note it will cause the runcmd block to exit before the command actually finishes.

For me this is fine, my-script.sh also needs to run after cloud-init finishes all of its other processing, which is achieved as follows:

# Wait for cloudinit to finish
while [ ! -f /var/lib/cloud/instance/boot-finished ]; do
    sleep 1
done