cloud-config on cents not working (at all)

January 5, 2015 5.6k views

I am trying to create a new CentOS7 (in AMS2) droplet with the following User-Data.
Unfortunately none of the provided User-Data are applied.

Please advise.


  - name: someuser
    groups: wheel
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    shell: /bin/bash
      - ssh-dss 

  - sed -i -e '/^PermitRootLogin/s/^.*$/PermitRootLogin yes/' /etc/ssh/sshd_config
  - sed -i -e '$aAllowUsers someuser' /etc/ssh/sshd_config
  - service sshd restart
9 Answers


Thanks for bringing this to our attention. I noticed your comment on the "Additional Recommended Steps for New CentOS 7 Servers" article and took a look. The users module seems to be failing on CentOS, but unfortunately the error log (/var/log/cloud-init.log) isn't very helpful:

Jan  5 11:28:49 centos7 cloud-init: 2015-01-05 11:28:49,223 -[WARNING]: Failed to create user someuser
Jan  5 11:28:49 centos7 cloud-init: 2015-01-05 11:28:49,227 -[WARNING]: Running users-groups (<module 'cloudinit.config.cc_users_groups' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_users_groups.pyc'>) failed

Though the problem seems to be limited to that module. For instance, this very simple cloud-config file works as expected:

  - content: |
      This is a test file
    path: /root/test
   - touch /root/foo

I'm going to bring this up with our engineering team and dig a little deeper.

After setting up the bare recommended configuration for a new server, there are often some additional steps that are highly recommended in most cases. In this guide, we'll continue the initial configuration by tackling some recommended, but optional procedures.
  • thanks, for taking it to the support - the sample you provided is far away from what I try to achieve (the above was just a short extract).

    I am just wondering that there are not much more requests as the users module is quite essential for bootstrapping a system.

    Therefore again: Getting this working is very important - at least to me :)

    Update: It seems to work fine on AMS3, however, I assume DO should fix that on AMS2 anyway... (No config management there? ;) )


Any updates on this?
I tested some things with CentOS on AMS3 and experienced issues loading packages (which causes cloud-init to stop running the rest of the script).

This is quite annoying as it should be a default feature as per DO which isnt working due to some config issues on the DO site causing massive delay on my site, forcing me switching to EC2 or linode for further testing purposes.

Can you please trigger support as you mentioned you do / did and get them higher attention here?

I think if DO advertises features, they should make sure they work and keep them working.

  • Hey! I brought this up with the engineers responsible for our metadata service and cloud-init implementation. They were not able to reproduce the issue, and I am not able to anymore. I've successfully used with following cloud-config file to create servers in both AMS2 and AMS3.

      - name: asb
        groups: wheel
        sudo: ['ALL=(ALL) NOPASSWD:ALL']
        shell: /bin/bash
          - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGibGRgqmPeCydQhrUWthMbaeryIoFsALE+MATT17h8Vv0TU9dnygsDE5Knu/iA6TgxvlqI6a+OVuOpWadr20T16ZInxo11/0DCtfoqIdAHDy74x0waTK3mZPlQevUxbuqPYwWY2B4hqpb3+byeWYtG4oUk/yTjnKxg5Mz6UiKixw+C63s79K9Q7EYLO2ljRHL8WbyUdjlYiIMC/gdbhNUWh9nG0eKKLfU6YoL4LkoCMImDeg2xqdv20xgkhU+F3PhqYtMe047S7JtoMhem3Z7F94HjszJvI9hTIqCKoZfxsnNYnOyWAGN8ySc6IRPnbb91xWheimwMq+yVeUoO5xP asb@asb-do
      - sed -i -e '/^PermitRootLogin/s/^.*$/PermitRootLogin yes/' /etc/ssh/sshd_config
      - sed -i -e '$aAllowUsers asb' /etc/ssh/sshd_config
      - service sshd restart

    So this does not infact seem to be an issue with our cloud-init implementation. I totally realize that is not a very satisfying answer. Without more details, it is hard to know what might be causing the issue loading packages. At first blush, it sounds like there might be some sort of networking issue. Could you please open a support ticket so that the team can take a look at the specifics of your situation (the physical hypervisor that your failed deploys were on, etc...).

  • well... thanks for looking into it, however, looking at the error log(s), I do see still some (apt?!) issues here (while creating a droplet AMS2 and CentOS7).

    Just to list some of those errors:

    2015-01-07 11:11:19,518 -[WARNING]: Unable to change the ownership of /var/log/cloud-init.log to user syslog, group adm
    2015-01-07 11:11:20,304 -[WARNING]: Could not find module named cc_ubuntu_init_switch
    2015-01-07 11:11:21,501 -[WARNING]: Running locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_locale.pyc'>) failed
    2015-01-07 11:11:21,535 -[WARNING]: Failed to run debconf-set-selections for grub-dpkg
    2015-01-07 11:11:21,557 -[WARNING]: Running apt-configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_apt_configure.pyc'>) failed

    It gets worse, when you try to add some packages:

    Cloud-init v. 0.7.5 running 'modules:config' at Wed, 07 Jan 2015 16:27:00 +0000. Up 6.84 seconds.
    2015-01-07 11:27:00,867 -[WARNING]: Running locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python2.7/site-packages/cloudinit/config/$
    2015-01-07 11:27:00,897 -[WARNING]: Failed to run debconf-set-selections for grub-dpkg
    2015-01-07 11:27:00,917 -[WARNING]: Running apt-configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python2.7/site-packages/clo$
    2015-01-07 11:27:00,934 -[WARNING]: Package update failed
    2015-01-07 11:27:00,951 -[WARNING]: Failed to install packages: ['ntp', 'httpd', 'mariadb-server']
    2015-01-07 11:27:00,951 -[WARNING]: 2 failed with exceptions, re-raising the last one
    2015-01-07 11:27:00,952 -[WARNING]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/u$
    Cloud-init v. 0.7.5 running 'modules:final' at Wed, 07 Jan 2015 16:27:01 +0000. Up 7.42 seconds.

    For reproduction on your site, you can use the following script, modifying SomeUser and your own public Key.
    For splitting tests, run a test with "packages" modul (red section) and one without it).

    # log all cloud-init process output (info & errors) to a logfile
    output: {all: ">> /var/log/cloud-init-output.log"}
    # final_message written to log when cloud-init processes are finished
    final_message: "System boot (via cloud-init) is COMPLETE, after $UPTIME seconds. Finished at $TIMESTAMP"
    #creating new user 
      - name: SomeUser
        groups: wheel
        sudo: ['ALL=(ALL) NOPASSWD:ALL']
        shell: /bin/bash
          - ssh-dss AAAABBBBCCCCCDDD...
      - ntp
      - httpd
      - mariadb-server
      #Enable EPEL Repo
      - yum -y install epel-release
      - rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
      #Enforce SHH Protocol 2
      - sed -i -e 's/#Protocol 2/Protocol 2/g' /etc/ssh/sshd_config
      - sed -i -e 's/#LoginGraceTime 2m/LoginGraceTime 2m/g' /etc/ssh/sshd_config
      - sed -i -e 's/#PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
      #Use key-based authentication, whenever possible, instead of basic authentication (username + password) to log on to your server remotely.
      - sed -i -e 's/PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config
      - sed -i -e 's/#RSAAuthentication yes/RSAAuthentication yes/g' /etc/ssh/sshd_config
      - sed -i -e 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/g' /etc/ssh/sshd_config
      - sed -i -e '$aAllowUsers SomeUser' /etc/ssh/sshd_config
      - service sshd restart
      - systemctl start firewalld
      - firewall-cmd --permanent --add-service=ssh
      - firewall-cmd --reload
      - systemctl enable firewalld

As this issue affects anyone using cloudconfig / User Data on CentOS, someone maybe wants to write a heads up on the forum / blog to let the affected customers know and helping them to avoid lots of time for troubleshooting issues that they won´t be able to fix by themselve?

Note from support regarding ticket #493247:
Yes, I believe this issue is stemming from the fact that we're serving modules to CentOS that are Ubuntu/Debian specific via our vendor-data. We'll need to deploy a patch to production in order to resolve this.

We're working on that, and we'll update you when we have something ready for you to look at.

UPDATE from support:
we modified our vendor-data that's served to CentOS to serve a module list identical to the one that ships with CentOS, and while the apt errors have vanished - we're still seeing errors related to the package installation:

2015-01-07 19:24:47,805 -[WARNING]: Package update failed
2015-01-07 19:24:47,828 -[WARNING]: Failed to install packages: ['ntp', 'httpd', 'mariadb-server']
2015-01-07 19:24:47,828 -[WARNING]: 2 failed with exceptions, re-raising the last one
2015-01-07 19:24:47,828 -[WARNING]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_package_update_upgrade_install.pyc'>) failed

We'll need to continue to experiment with cloud-init to clarify why it's unable to complete this installation, unfortunately.

  • My problem is similar.

    Only that the cloud-config works 33% of the time... (I see the same ubuntu warning)

Bump. Any easy/known fix for this?

After a recent reboot, I find nothing logging to syslog, messages, secure. They were written to at the time of boot, but not since (now 3 days later). btmp, wtmp, and lastlog are updating, other apps logging outside of /var/log are writing still.

Have another answer? Share your knowledge.