How to update CoreOS cloud-config?

February 9, 2015 13.8k views

Once you've booted a new CoreOS host on Digital Ocean, how can you update the cloud-config?

  • The cloud-config that is passed to a droplet on creation can not be updated. It is generally used to provide a script to be run on first boot.

    Could you tell us a bit more about your use case? I'm interested to know why you might want to update it.

    The DigitalOcean metadata service includes a field called "user-data", which can be used to specify a script that will be run as your server is brought online. The CloudInit program, which runs these scripts, can process a special script type called "cloud-config". In this guide, we'll explore how to create cloud-config files and the best ways to leverage their power.
  • i am interested in this too. i use the cloud-config to start the coreos joined to my fleet.

    I just created a coreos instance, how do i get it my could-config? I see that the root cloud-config (i assume) is in /usr/share/oem/cloud-config.yml. It fires up this:

    /usr/bin/coreos-cloudinit --oem=digitalocean

    It would be nice to change that line to do something like:

    /usr/bin/coreos-cloudinit --from-url=""

    how do you customize the coreos image? I tried doing this:
    I created my own cloud-config.yml file. Then, ran the program:
    core@d2 ~ $ /usr/bin/coreos-cloudinit -from-file=cloud-config
    Checking availability of "local-file"
    Fetching user-data from datasource of type "local-file"
    Fetching meta-data from datasource of type "local-file"
    2015/02/18 20:52:25 Parsing user-data as cloud-config
    Processing cloud-config from user-data
    2015/02/18 20:52:25 Set hostname to d2
    2015/02/18 20:52:25 Authorized SSH keys for core user
    2015/02/18 20:52:25 Writing drop-in unit "20-cloudinit.conf" to filesystem
    2015/02/18 20:52:25 Writing file to "/run/systemd/system/etcd.service.d/20-cloudinit.conf"
    Failed to apply cloud-config: mkdir /run/systemd/system/etcd.service.d: permission denied

    Didn't work as core user.


  • I think is pretty common change the cloud-config since sometime the initial configuration ins't enough anymore. As @lgfausak said the coreos provides a command for that coreos-cloudinit. I also couldn't execute it with the core user. But with the root user that did work. Example:

  • I haven't found a way to enable iptables except through cloud-config (as described here:

    So the rules-save file gets rewritten at every boot. What to do if you want to change the rules? Your stuck.

    But perhaps there is a better way to configure and run iptables?

  • @asb This might well be CoreOS related, though, but consider the use case when you need to just add new meta data to the machine in a cluster. Is there a way to do it without destroying it? coreos-cloudinit works just once in my experience.

  • +1, I'd like to be able to properly update ssh keys and have them persist over reboots, seems like the way to do this is set them in the cloud-config, but then if there's a change to a ssh key I have to rebuild the whole server? Am I missing something?

  • I also had the need to change the data controlled by cloud-init and issues with the values being reverted after reboot. However, I do think we should rather treat CoreOS VPS as cattle, not pets and rebuilding a server should be trivial in that sense.

  • Any update to this? @asb I've had to rebuild my cluster 3 times so far because I've needed to make changes to cloud-config. Once because I started my cluster out has http, and only found out later that in order to use https you have to set it in the cloud-config, an then another time so I could set environment variables for etcdctl so I don't have to set them every time I login, or create a .bashrc / .profile every time I create a new user... As I understand it, AWS has the ability to modify the cloud-config file and this ability may be a deal breaker for us when it comes to choosing between DO and another platform...

3 Answers

You can find the cloud-config in :

  • /var/lib/coreos-install/user_data for baremetal
  • /var/lib/coreos-vagrant/vagrant-user-data for vagrant

update it, reboot, if no error, the new configuration should be done.
I don't know if it's the good way but it seems to work.

@bgrayburn you can update ssh keys without doing anything crazy:

I have a script that I use to push them out across my fleet:

$ cat
#!/bin/bash -x

# Usage:
#   cat public.key | keyname

if [ -z $name ]; then
  echo "Provide a name for the injected SSH key"
  exit 1

shift 1


for machine in $(fleetctl -strict-host-key-checking=false $@ list-machines --no-legend --full | awk '{ print $1;}'); do
  fleetctl  -strict-host-key-checking=false $@ ssh $machine "echo '${pubkey}' | update-ssh-keys -a $name -n"

so to run you could do something like:




I have found a solution for updating cloud-config.yml!

The file /usr/share/oem/cloud-config.yml is writable by root user, so you can change it.
I first tried to change it to read yml from file (--from-file), but it was not working.
I discovered in the source of coreos-cloudinit, that it gets the user-data from DO's metadata API, but also get network information as well, and coreos-cloudinit sets up the network. So if you just change the line in the cloud-config.yml, the network won't work.
I also discovered that coreos-cloudinit 1st read the userdata and then read the network info. The "--oem=digitalocean" param is just a shortcut for the "--from-digitalocean-metadata --convert-netconf digitalocean" parameters.

So the solution is to change the API to localhost and simulate it this way:

until /bin/curl -s -i -o /tmp/v1; do sleep 1; done \
&& until /bin/curl -s -o /tmp/v1.json; do sleep 1; done \
&& ((cat /tmp/v1 | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) \
&& /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean \
&& rm /tmp/v1 && rm /tmp/v1.json
  • Download the needed answers from metadata service to /tmp. The "untils" are very important, because the metadata API is not accessible just a little bit later, so we need to wait for it.
  • Start netcats on localhost port 80 to simulate HTTP webserver, we will have 3 requests, so start 3 netcats in order
  • 1st http request is checking metadata api:
  • 2nd request is the new cloud-config.yml from /home/core/cloud-config.yml
  • 3rd request is a JSON with the needed network data (it also contains the user-data, but not used, so we don't care):
  • So, netcats are waiting for requests one-by-one, we can start coreos-cloudinit with localhost as metadata URL, it will get our new cloud-config.yml
  • After successfull initialization we can delete the downloaded answers from /tmp

The full cloud-config is this:


      - name:
        runtime: yes
        content: |

      - name: systemd-networkd.service
        command: restart
      - name: oem-cloudinit.service
        command: restart
        runtime: yes
        content: |
          Description=Cloudinit from DigitalOcean metadata

          ExecStart=/bin/sh -c 'until /bin/curl -s -i -o /tmp/v1; do sleep 1; done && until /bin/curl -s -o /tmp/v1.json; do sleep 1; done && ((cat /tmp/v1 | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) && /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean && rm /tmp/v1 && rm /tmp/v1.json'
      id: digitalocean
      name: DigitalOcean
      version-id: 0.0.4

It is a hacky way, but works very good. You only need to upload the new config into /home/core/cloud-config.yml and all your changes are alive after reboot.

Have another answer? Share your knowledge.