Once you’ve booted a new CoreOS host on Digital Ocean, how can you update the cloud-config?
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
×@bgrayburn you can update ssh keys without doing anything crazy: https://github.com/coreos/init/blob/master/bin/update-ssh-keys
I have a script that I use to push them out across my fleet:
$ cat fleetctl-inject-ssh.sh
#!/bin/bash -x
# Usage:
# cat public.key | fleetctl-inject-ssh.sh keyname
name=$1
if [ -z $name ]; then
echo "Provide a name for the injected SSH key"
exit 1
fi
shift 1
pubkey=$(cat)
for machine in $(fleetctl -strict-host-key-checking=false $@ list-machines --no-legend --full | awk '{ print $1;}'); do
fleetctl -strict-host-key-checking=false $@ ssh $machine "echo '${pubkey}' | update-ssh-keys -a $name -n"
done
so to run you could do something like:
echo 'THE_CONTENTS_OF_PUBKEY' | ./fleetctl-inject-ssh.sh <SOME_INTELLIGENT_NAME_FOR_KEY>
or
cat <PUBKEY> | ./fleetctl-inject-ssh.sh <SOME_INTELLIGENT_NAME_FOR_KEY>
I have found a solution for updating cloud-config.yml!
The file /usr/share/oem/cloud-config.yml is writable by root user, so you can change it.
I first tried to change it to read yml from file (–from-file), but it was not working.
I discovered in the source of coreos-cloudinit, that it gets the user-data from DO’s metadata API, but also get network information as well, and coreos-cloudinit sets up the network. So if you just change the line in the cloud-config.yml, the network won’t work.
I also discovered that coreos-cloudinit 1st read the userdata and then read the network info. The “–oem=digitalocean” param is just a shortcut for the “–from-digitalocean-metadata http://169.254.169.254/ –convert-netconf digitalocean” parameters.
So the solution is to change the API to localhost and simulate it this way:
until /bin/curl -s -i -o /tmp/v1 http://169.254.169.254/metadata/v1/; do sleep 1; done \
&& until /bin/curl -s -o /tmp/v1.json http://169.254.169.254/metadata/v1.json; do sleep 1; done \
&& ((cat /tmp/v1 | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 \
&& echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) \
&& /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean \
&& rm /tmp/v1 && rm /tmp/v1.json
The full cloud-config is this:
#cloud-config
coreos:
units:
- name: 99-public-interface.network
runtime: yes
content: |
[Match]
Path=*:03.0
[Network]
IPv4LL=true
- name: systemd-networkd.service
command: restart
- name: oem-cloudinit.service
command: restart
runtime: yes
content: |
[Unit]
Description=Cloudinit from DigitalOcean metadata
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'until /bin/curl -s -i -o /tmp/v1 http://169.254.169.254/metadata/v1/; do sleep 1; done && until /bin/curl -s -o /tmp/v1.json http://169.254.169.254/metadata/v1.json; do sleep 1; done && ((cat /tmp/v1 | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) && /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ --convert-netconf digitalocean && rm /tmp/v1 && rm /tmp/v1.json'
oem:
id: digitalocean
name: DigitalOcean
version-id: 0.0.4
home-url: https://www.digitalocean.com/
bug-report-url: https://github.com/coreos/bugs/issues
It is a hacky way, but works very good. You only need to upload the new config into /home/core/cloud-config.yml and all your changes are alive after reboot.
My solution is working with newer CoreOS versions, though with a little modifications.
The new oem-cloudinit.service is in /etc/systemd/system/oem-cloudinit.service file, which is writable. You just need to change to this:
[Unit]
Description=Cloudinit from DigitalOcean metadata
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'until /bin/curl -s -i -o /tmp/v1 http://169.254.169.254/metadata/v1/; do sleep 1; done && until /bin/curl -s -o /tmp/v1.json http://169.254.169.254/metadata/v1.json; do sleep 1; done && ((cat /tmp/v1 | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /home/core/cloud-config.yml | ncat -lp 80 && echo -e "HTTP/1.1 200 OK\r\n\r" | cat - /tmp/v1.json | ncat -lp 80) &>/dev/null &) && /usr/bin/coreos-cloudinit --from-digitalocean-metadata http://localhost/ && rm /tmp/v1 && rm /tmp/v1.json'
[Install]
WantedBy=multi-user.target
It still use the /home/core/cloud-config.yml as the new user-data.
The cloud-config that is passed to a droplet on creation can not be updated. It is generally used to provide a script to be run on first boot.
Could you tell us a bit more about your use case? I’m interested to know why you might want to update it.
i am interested in this too. i use the cloud-config to start the coreos joined to my fleet.
I just created a coreos instance, how do i get it my could-config? I see that the root cloud-config (i assume) is in /usr/share/oem/cloud-config.yml. It fires up this:
/usr/bin/coreos-cloudinit –oem=digitalocean
It would be nice to change that line to do something like:
/usr/bin/coreos-cloudinit –from-url=“http://my.com/my-cloud-config?hostname”
how do you customize the coreos image? I tried doing this:
I created my own cloud-config.yml file. Then, ran the program:
core@d2 ~ $ /usr/bin/coreos-cloudinit -from-file=cloud-config
Checking availability of “local-file”
Fetching user-data from datasource of type “local-file”
Fetching meta-data from datasource of type “local-file”
2015/02/18 20:52:25 Parsing user-data as cloud-config
Processing cloud-config from user-data
2015/02/18 20:52:25 Set hostname to d2
2015/02/18 20:52:25 Authorized SSH keys for core user
2015/02/18 20:52:25 Writing drop-in unit “20-cloudinit.conf” to filesystem
2015/02/18 20:52:25 Writing file to “/run/systemd/system/etcd.service.d/20-cloudinit.conf”
Failed to apply cloud-config: mkdir /run/systemd/system/etcd.service.d: permission denied
Didn’t work as core user.
-g
I think is pretty common change the cloud-config since sometime the initial configuration ins’t enough anymore. As @lgfausak said the coreos provides a command for that
coreos-cloudinit. I also couldn’t execute it with the core user. But with the root user that did work. Example: https://gist.github.com/maxcnunes/b93a118914caa9ea87eaI haven’t found a way to enable iptables except through cloud-config (as described here: http://www.jimmycuadra.com/posts/securing-coreos-with-iptables.)
So the rules-save file gets rewritten at every boot. What to do if you want to change the rules? Your stuck.
But perhaps there is a better way to configure and run iptables?
@asb This might well be CoreOS related, though, but consider the use case when you need to just add new meta data to the machine in a cluster. Is there a way to do it without destroying it? coreos-cloudinit works just once in my experience.
+1, I’d like to be able to properly update ssh keys and have them persist over reboots, seems like the way to do this is set them in the cloud-config, but then if there’s a change to a ssh key I have to rebuild the whole server? Am I missing something?
I also had the need to change the data controlled by cloud-init and issues with the values being reverted after reboot. However, I do think we should rather treat CoreOS VPS as cattle, not pets and rebuilding a server should be trivial in that sense.
+1
Any update to this? @asb I’ve had to rebuild my cluster 3 times so far because I’ve needed to make changes to cloud-config. Once because I started my cluster out has http, and only found out later that in order to use https you have to set it in the cloud-config, an then another time so I could set environment variables for etcdctl so I don’t have to set them every time I login, or create a .bashrc / .profile every time I create a new user… As I understand it, AWS has the ability to modify the cloud-config file and this ability may be a deal breaker for us when it comes to choosing between DO and another platform…
Use cases for updating
cloud-configwithout reprovisioning the host:etcd2andfleetclient urls when SSL is added lateriptables-restoreconfig intocloud-configand change it as cluster services evolveWorkaround is to reprovision host with new metadata. This is tedious and risky, and has to be done fairly often, and writing python scripts to do it all automatically has taken quite a bit of effort.