How to update CoreOS cloud-config?

  • Posted February 9, 2015

Once you’ve booted a new CoreOS host on Digital Ocean, how can you update the cloud-config?


+1, I’d like to be able to properly update ssh keys and have them persist over reboots, seems like the way to do this is set them in the cloud-config, but then if there’s a change to a ssh key I have to rebuild the whole server? Am I missing something?

i am interested in this too. i use the cloud-config to start the coreos joined to my fleet.

I just created a coreos instance, how do i get it my could-config? I see that the root cloud-config (i assume) is in /usr/share/oem/cloud-config.yml. It fires up this:

/usr/bin/coreos-cloudinit --oem=digitalocean

It would be nice to change that line to do something like:

/usr/bin/coreos-cloudinit --from-url=“

how do you customize the coreos image? I tried doing this: I created my own cloud-config.yml file. Then, ran the program: core@d2 ~ $ /usr/bin/coreos-cloudinit -from-file=cloud-config Checking availability of “local-file” Fetching user-data from datasource of type “local-file” Fetching meta-data from datasource of type “local-file” 2015/02/18 20:52:25 Parsing user-data as cloud-config Processing cloud-config from user-data 2015/02/18 20:52:25 Set hostname to d2 2015/02/18 20:52:25 Authorized SSH keys for core user 2015/02/18 20:52:25 Writing drop-in unit “20-cloudinit.conf” to filesystem 2015/02/18 20:52:25 Writing file to “/run/systemd/system/etcd.service.d/20-cloudinit.conf” Failed to apply cloud-config: mkdir /run/systemd/system/etcd.service.d: permission denied

Didn’t work as core user.


Use cases for updating cloud-config without reprovisioning the host:

  • Update etcd2 and fleet client urls when SSL is added later
  • Add additional units
  • Put iptables-restore config into cloud-config and change it as cluster services evolve
  • Adjust SSH keys

Workaround is to reprovision host with new metadata. This is tedious and risky, and has to be done fairly often, and writing python scripts to do it all automatically has taken quite a bit of effort.

Any update to this? @asb I’ve had to rebuild my cluster 3 times so far because I’ve needed to make changes to cloud-config. Once because I started my cluster out has http, and only found out later that in order to use https you have to set it in the cloud-config, an then another time so I could set environment variables for etcdctl so I don’t have to set them every time I login, or create a .bashrc / .profile every time I create a new user… As I understand it, AWS has the ability to modify the cloud-config file and this ability may be a deal breaker for us when it comes to choosing between DO and another platform…

I also had the need to change the data controlled by cloud-init and issues with the values being reverted after reboot. However, I do think we should rather treat CoreOS VPS as cattle, not pets and rebuilding a server should be trivial in that sense.

@asb This might well be CoreOS related, though, but consider the use case when you need to just add new meta data to the machine in a cluster. Is there a way to do it without destroying it? coreos-cloudinit works just once in my experience.

I haven’t found a way to enable iptables except through cloud-config (as described here:

So the rules-save file gets rewritten at every boot. What to do if you want to change the rules? Your stuck.

But perhaps there is a better way to configure and run iptables?

I think is pretty common change the cloud-config since sometime the initial configuration ins’t enough anymore. As @lgfausak said the coreos provides a command for that coreos-cloudinit. I also couldn’t execute it with the core user. But with the root user that did work. Example:

The cloud-config that is passed to a droplet on creation can not be updated. It is generally used to provide a script to be run on first boot.

Could you tell us a bit more about your use case? I’m interested to know why you might want to update it.

Submit an answer
You can type!ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

You can find the cloud-config in :

  • /var/lib/coreos-install/user_data for baremetal
  • /var/lib/coreos-vagrant/vagrant-user-data for vagrant

update it, reboot, if no error, the new configuration should be done. I don’t know if it’s the good way but it seems to work.

Has anyone able to update the user-data without using @Wallner solution?

I have found a solution for updating cloud-config.yml!

The file /usr/share/oem/cloud-config.yml is writable by root user, so you can change it. I first tried to change it to read yml from file (–from-file), but it was not working. I discovered in the source of coreos-cloudinit, that it gets the user-data from DO’s metadata API, but also get network information as well, and coreos-cloudinit sets up the network. So if you just change the line in the cloud-config.yml, the network won’t work. I also discovered that coreos-cloudinit 1st read the userdata and then read the network info. The “–oem=digitalocean” param is just a shortcut for the “–from-digitalocean-metadata --convert-netconf digitalocean” parameters.

So the solution is to change the API to localhost and simulate it this way:

until /bin/curl -s -i -o /tmp/v1; do sleep 1; done \
&& until /bin/curl -s -o /tmp/v1.json; do sleep 1; done \
&& ((cat /tmp/v1 | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) \
&& /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean \
&& rm /tmp/v1 && rm /tmp/v1.json
  • Download the needed answers from metadata service to /tmp. The “untils” are very important, because the metadata API is not accessible just a little bit later, so we need to wait for it.
  • Start netcats on localhost port 80 to simulate HTTP webserver, we will have 3 requests, so start 3 netcats in order
  • 1st http request is checking metadata api:
  • 2nd request is the new cloud-config.yml from /home/core/cloud-config.yml
  • 3rd request is a JSON with the needed network data (it also contains the user-data, but not used, so we don’t care):
  • So, netcats are waiting for requests one-by-one, we can start coreos-cloudinit with localhost as metadata URL, it will get our new cloud-config.yml
  • After successfull initialization we can delete the downloaded answers from /tmp

The full cloud-config is this:


      - name:
        runtime: yes
        content: |

      - name: systemd-networkd.service
        command: restart
      - name: oem-cloudinit.service
        command: restart
        runtime: yes
        content: |
          Description=Cloudinit from DigitalOcean metadata

          ExecStart=/bin/sh -c 'until /bin/curl -s -i -o /tmp/v1; do sleep 1; done && until /bin/curl -s -o /tmp/v1.json; do sleep 1; done && ((cat /tmp/v1 | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) && /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean && rm /tmp/v1 && rm /tmp/v1.json'
      id: digitalocean
      name: DigitalOcean
      version-id: 0.0.4

It is a hacky way, but works very good. You only need to upload the new config into /home/core/cloud-config.yml and all your changes are alive after reboot.

@bgrayburn you can update ssh keys without doing anything crazy:

I have a script that I use to push them out across my fleet:

$ cat
#!/bin/bash -x

# Usage:
#   cat public.key | keyname

if [ -z $name ]; then
  echo "Provide a name for the injected SSH key"
  exit 1

shift 1


for machine in $(fleetctl -strict-host-key-checking=false $@ list-machines --no-legend --full | awk '{ print $1;}'); do
  fleetctl  -strict-host-key-checking=false $@ ssh $machine "echo '${pubkey}' | update-ssh-keys -a $name -n"

so to run you could do something like: