Do I need cloud-init?

May 16, 2017 6.3k views
Linux Basics Ubuntu
adminc6bc084b9ac49e0d2801b
By:
adminc6bc084b9ac49e0d2801b

Every time I update, I see the following:

$ apt list --upgradable
Listing... Done
cloud-init/zesty 0.7.9-90-g61eb03fe-0ubuntu1 all [upgradable from: 0.7.9-0ubuntu1~16.10.1]

and when upgrading

The following packages have been kept back:
  cloud-init

What is the package for and why is it held back?

DigitalOcean’s FAQ contains an entry on CloudInit:

What is CloudInit?

CloudInit is a process enabled on recent DigitalOcean images that is able to pull down and process information from metadata. When the Droplet boots for the first time, the CloudInit program executes the script it finds in the user-data field, providing users the opportunity to automate the initial configuration of their servers.

but I am not much wiser. Apparently, it is used during the initial boot but is it safe to remove it afterwards?

Log In to Comment
5 Answers
jtittle1 May 16, 2017
Accepted Answer

@adminc6bc084b9ac49e0d2801b

Last I recall, DigitalOcean uses it for the imaging and snapshot setups. If you're not going to use the snapshot / backup service, then it's safe to remove.

Removing the package shouldn't remove any other packages or core dependencies. When testing it on Ubuntu 16.04, only that one package is removed.

...

As for the purpose of the package, you can read over the basics here:

https://help.ubuntu.com/community/CloudInit

The above link provides more detailed information on what it can do as well as a few use cases for it.

...

That said, it's odd that dist-upgrade would leave the package hanging. I tested this one two Droplets and neither have any packages remaining or hanging behind, kernel or otherwise.

I normally run the full suite on initial deployment:

apt update \
&& apt -y full-upgrade \
&& apt -y upgrade \
&& apt -y autoremove

Or if you prefer to use apt-get:

apt-get update \
&& apt-get -y dist-upgrade \
&& apt-get -y upgrade \
&& apt-get -y autoremove

16.04/16.10 now use apt which I find more useful since it shortens commands (even though I mostly use bash scripts).

Reply
  • asb MOD May 16, 2017

    @adminc6bc084b9ac49e0d2801b I just wanted to jump in and add that @jtittle is absolutely correct about snapshots and backups. Without cloud-init installed, you will likely be unable to create a new Droplet based off of one. Beyond user-data, cloud-init also handles a number of configuration bits, most importantly networking. It ingests information from our metadata API and uses that to configure your network interfaces. It may be possible to login through the web console and configure it manually, but it would certainly be a pain.

    Another important thing cloud-init does is help with resizing the file system dynamically. It is configured to do so using growpart and resizefs on reboots. Again, you don't strictly need it. If you resize the Droplet's storage without cloud-init, your disk space will be correctly increased. Though you will need to resize your file system manually to use the available space.

    An Introduction to Droplet Metadata
    by Mitchell Anicas
    Update CloudInit link and conclusion Metadata is a service provided to DigitalOcean droplets that allows a droplet to access data about itself, i.e. its metadata. Examples of available droplet metadata include user-provided user data, droplet ID, data center region,...
jtittle1 May 16, 2017

@adminc6bc084b9ac49e0d2801b

When you first deploy a Droplet, there's a checkbox labeled "User Data" which allows you to pass a bash script to the service, which will then run during the initial deployment. This allows you to do an initial server configuration without having to login to the terminal. It's more useful if you're using the API as a part of automated deployments, though it can be a quick way to simply get things setup as needed from the control panel as well.

...

Unless you absolutely need to remove it, it's safe to leave it as-is. It's not really taking up much disk space or utilizing large amounts of CPU or RAM (if any at all).

The reason it's kept back is similar to how kernel updates are kept back unless you run:

apt-get dist-upgrade

The standard upgrade command:

apt-get upgrade

... won't upgrade core packages that may change functionality. For example, if you were running PHP 7.0 and PHP 7.1 was in the repositories as well, the base upgrade command won't upgrade 7.0 to 7.1 or even 8.0 (when it's released) as those are major version releases. It would, however, upgrade 7.0.x to 7.0.1, 7.0.2, etc. The same for other software.

Generally, unless you 100% confident you're not going to run in to issues with a new kernel, newer versions of software, etc -- you don't want to dist-upgrade. While it's generally safe, it does allow major version releases to be installed.

Reply
adminc6bc084b9ac49e0d2801b May 16, 2017

@jtittle

Unless you absolutely need to remove it, it's safe to leave it as-is. It's not really taking up much disk space or utilizing large amounts of CPU or RAM (if any at all).

For various reasons, I prefer to have installed only the packages that are actually used. I understand how the package is employed for the initial deployment, I would like to know if it does anything once the system is set up.

…

Actually the “packages have been kept back” is an output of dist-upgrade. I always run it since I like to live on the edge ;-)

But this particular peculiarity of apt is of secondary interest to me. Mainly, I would like to make sense of the package itself.

Reply
adminc6bc084b9ac49e0d2801b May 16, 2017

@jtittle https://help.ubuntu.com/community/CloudInit again seems to only talk about the initial boot. It reads to me like the tool is designed for setting up (short-lived) machines, I do not see any mention of snapshots. But I will take your word for it and keep the package. I make a snapshot before each upgrade.

…

Regarding the hold package, it is not a first time I have seen that. I think I resolved it on my desktop by re-installing the package. I will probably try to do the same if no-one can explain the reason for the holding.

~# apt update
Hit:1 http://ppa.launchpad.net/certbot/certbot/ubuntu zesty InRelease
Hit:2 http://security.ubuntu.com/ubuntu zesty-security InRelease
Hit:3 http://ams2.mirrors.digitalocean.com/ubuntu zesty InRelease
Hit:4 http://ams2.mirrors.digitalocean.com/ubuntu zesty-updates InRelease
Get:5 http://ams2.mirrors.digitalocean.com/ubuntu zesty-backports InRelease [89.2 kB]
Fetched 89.2 kB in 1s (75.2 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
~# apt -y full-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  cloud-init
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
~# apt show cloud-init -a
Package: cloud-init
Version: 0.7.9-90-g61eb03fe-0ubuntu1
Priority: extra
Section: admin
Origin: Ubuntu
Maintainer: Scott Moser <smoser@ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 1,491 kB
Depends: cloud-guest-utils | cloud-utils, ifupdown (>= 0.6.10ubuntu5), procps, python3 (>= 3.2), python3-requests (>= 0.8.2), python3-serial, debconf (>= 0.5) | debconf-2.0, init-system-helpers (>= 1.18~), python3-configobj, python3-jinja2, python3-jsonpatch, python3-oauthlib, python3-prettytable, python3-six, python3-yaml, python3:any (>= 3.3.2-2~)
Recommends: eatmydata, gdisk, software-properties-common
Task: ubuntu-core, cloud-image, ubuntu-core
Supported: 9m
Download-Size: 304 kB
APT-Sources: http://mirrors.digitalocean.com/ubuntu zesty/main amd64 Packages
Description: Init scripts for cloud instances
 Cloud instances need special scripts to run during initialisation
 to retrieve and install ssh keys and to let the user run various scripts.

Package: cloud-init
Version: 0.7.9-0ubuntu1~16.10.1
Status: hold ok installed
Priority: extra
Section: admin
Maintainer: Scott Moser <smoser@ubuntu.com>
Installed-Size: 1,399 kB
Depends: cloud-guest-utils | cloud-utils, ifupdown (>= 0.6.10ubuntu5), procps, python3 (>= 3.2), python3-requests (>= 0.8.2), python3-serial, debconf (>= 0.5) | debconf-2.0, init-system-helpers (>= 1.18~), python3-configobj, python3-jinja2, python3-jsonpatch, python3-oauthlib, python3-prettytable, python3-six, python3-yaml, python3:any (>= 3.3.2-2~)
Recommends: eatmydata, gdisk, software-properties-common
Download-Size: unknown
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Init scripts for cloud instances
 Cloud instances need special scripts to run during initialisation
 to retrieve and install ssh keys and to let the user run various scripts.
Reply
  • jtittle1 May 17, 2017

    @adminc6bc084b9ac49e0d2801b

    cloud-init is more so beneficial if you're using an API. From the DigitalOcean control panel, it has some utility, though when the API comes in to play and you begin looking at automation, it can be a huge time saver.

    For example, you pass in a bash script that handles everything for you -- from upgrades to the initial setup -- and that gets executed as expected (though sometimes it does take some tests).

    You can pass that bash script through during Droplet creation when using the API instead of having to access the dashboard / control panel.

    ....

    As for the package being held back, the only thing I can think of right off would be if it's marked to be held back.

    Typically, you'd do this by running:

    apt-mark hold ....

    To remove a hold, you'd use:

    apt-mark unhold ....

    In such a case where a hold is placed, it'll be held back when either upgrade or dist-upgrade is ran.

    • wimar September 27, 2017

      @jtittle1 Thanks for the tip. apt-mark unhold cloud-init did the trick for me. Anyone any idea how the package ended up being marked as hold?

adminc6bc084b9ac49e0d2801b May 17, 2017

@jtittle
I understand the benefits of provisioning but at the moment setting the server manually is enough. If my requirements change, I will likely go the whole way and use NixOS + NixOps.

…

I definitely did not hold the package manually. Maybe it got marked as hold during upgrade to zesty? Either way, I reinstalled it and it is no longer hold.

Reply
  • jtittle1 May 17, 2017

    @adminc6bc084b9ac49e0d2801b

    I'd say anything is possible when it comes to upgrading between versions. It definitely wouldn't be the first time something strange has happened :-). While most upgrades are flawless, some seem to yield unexpected results from time to time.

    As for NixOS, I was actually looking at that the other day. I've not gotten around to really diving in just yet, but I may have to take a closer look.

Have another answer? Share your knowledge.

Related Questions