Tutorial

How To Limit Resources Using cgroups on CentOS 6

Published on June 10, 2015
How To Limit Resources Using cgroups on CentOS 6

Status: Deprecated

This article covers a version of CentOS that is no longer supported. If you are currently operating a server running CentOS 6, we highly recommend upgrading or migrating to a supported version of CentOS.

Reason: CentOS 6 reached end of life (EOL) on November 30th, 2020 and no longer receives security patches or updates. For this reason, this guide is no longer maintained.

See Instead: This guide might still be useful as a reference, but may not work on other CentOS releases. If available, we strongly recommend using a guide written for the version of CentOS you are using.

Introduction

Control groups, or cgroups, is a kernel feature introduced in CentOS 6 to provide a new way of limiting access to system resources for processes. You can create your own cgroups, monitor the cgroups you configure, deny cgroups access to certain resources, and even reconfigure your cgroups dynamically on a running system.

In this tutorial, we will see how to limit CPU, memory, and disk i/o for processes. To achieve this, we will first create some control groups, add processes to them, and see how they perform.

Prerequisites

Before you get started with this tutorial, you should have a non-root user with sudo setup on your CentOS 6 Droplet. To setup a user of this type, follow our Initial Server Setup with CentOS 6 tutorial. All commands will be run as this user.

Step 1 — Installation

In this section, we will be installing the packages required for cgroups to function.

Control groups and the subsystems to which they relate can be manipulated using shell commands and utilities. However, the easiest way to work with cgroups is to install the libcgroup package. The libcgroup package provides cgroups-related command line utilities, configuration files, and man pages. This package is not installed by default on a CentOS 6 server. To install it, run the following command:

  1. sudo yum install libcgroup

Step 2 — Starting the Service

The cgconfig(control group config) service is used to create cgroups and manage subsystems. It can be configured to start up at boot time and reestablish your predefined cgroups, thus making them persistent across reboots. The cgconfig service is not started by default on CentOS 6, so let us start it:

  1. sudo service cgconfig start

Starting the cgconfig service creates a virtual filesystem mounted at /cgroup with all the subsystems. Let us verify this:

  1. sudo ls /cgroup

This command should show the following subsystems:

blkio  cpu  cpuacct  cpuset  devices  freezer  memory  net_cls

You could also run the `lscgroup’ command to verify:

  1. sudo lscgroup

You will see the subsystems in a slightly different layout:

cpuset:/
cpu:/
cpuacct:/
memory:/
devices:/
freezer:/
net_cls:/
blkio:/

System Resources

The system resources are known as subsystems, and each subsystem has several parameters to which we could assign values. CentOS 6 provides ten cgroup subsystems:

  • blkio — this subsystem sets limits on input/output access to and from block devices such as physical drives (disk, solid state, USB, etc.).
  • cpu — this subsystem sets limits on the available CPU time
  • cpuacct — this subsystem generates automatic reports on CPU resources used by tasks in a cgroup
  • cpuset — this subsystem assigns individual CPUs (on a multicore system) and memory nodes to tasks in a cgroup
  • devices — this subsystem allows or denies access to devices by tasks in a cgroup
  • freezer — this subsystem suspends or resumes tasks in a cgroup
  • memory — this subsystem sets limits on memory use by tasks in a cgroup and generates automatic reports on memory resources used by those tasks
  • net_cls — this subsystem tags network packets with a class identifier (classid) that allows the Linux traffic controller (tc) to identify packets originating from a particular cgroup task
  • net_prio — this subsystem provides a way to dynamically set the priority of network traffic per network interface
  • ns — this is the namespace subsystem

Step 3 — Configuration

In this section, we will create example cgroups and set some resource limits for those cgroups. The cgroup configuration file is /etc/cgconfig.conf. Depending on the contents of the configuration file, cgconfig can create hierarchies, mount necessary file systems, create cgroups, and set subsystem parameters (resource limits) for each cgroup.

A hierarchy is a set of cgroups arranged in a tree, such that every task in the system is in exactly one of the cgroups in the hierarchy. In a default CentOS 6 configuration, each subsystem is put into its own hierarchy.

Let us first create a few cgroups named limitcpu, limitmem, limitio, and browsers. The /etc/cgconfig.conf file contains two major types of entries — mount and group. Lines that start with group create cgroups and set subsystem parameters. Edit the file /etc/cgconfig.conf and add the following cgroup entries at the bottom:

/etc/cgconfig.conf
group limitcpu{
        cpu {
                cpu.shares = 400;
        }
}

group limitmem{
        memory {
                memory.limit_in_bytes = 512m;
        }
}

group limitio{
        blkio {
                blkio.throttle.read_bps_device = "252:0         2097152";
        }
}

group browsers{
        cpu {
                cpu.shares = 200;
        }
        memory {
                memory.limit_in_bytes = 128m;
        }
}
  • In the limitcpu cgroup, we are limiting the cpu shares available to processes in this cgroup to 400. cpu.shares specifies the relative share of CPU time available to the tasks in the cgroup.
  • In the limitmem cgroup, we are limiting memory available to the cgroup processes to 512MB.
  • In the limitio cgroup, we are limiting the disk read throughput to 2MiB/s. Here we are limiting read I/O to the primary disk, /dev/vda, with major:minor number 252:0 and 2MiB/s is converted to bytes per second (2x1024x1024=2097152).
  • In the browsers cgroup, we are limiting cpu shares to 200 and available memory to 128MB.

We need to restart the cgconfig service for the changes in the /etc/cgconfig.conf file to take effect:

  1. sudo service cgconfig restart

Let us enable cgconfig to start on system boot. When you enable the service with chkconfig, it reads the cgroup configuration file /etc/cgconfig.conf at boot time. cgroups are recreated from session to session and remain persistent.

  1. sudo chkconfig cgconfig on

Next, verify the cgroups we configured are showing up correctly:

  1. lscgroup

If everything went well, you should see:

cpuset:/
cpu:/
cpu:/browsers
cpu:/limitcpu
cpuacct:/
memory:/
memory:/browsers
memory:/limitmem
devices:/
freezer:/
net_cls:/
blkio:/
blkio:/limitio

Our next goal is to add the processes (tasks) for which we wish to limit resources to the cgroups we created earlier.

Cgred (control group rules engine daemon) is a service that moves tasks into cgroups according to parameters set in the /etc/cgrules.conf file. Entries in the /etc/cgrules.conf file can take one of the two forms:

user subsystems control_group

or

user:command subsystems control_group

user refers to a username or a groupname prefixed with the “@” character. subsystems refer to a comma-separated list of subsystem names. control_group represents a path to the cgroup, and command stands for a process name or a full command path of a process. Entries in the /etc/cgrules.conf file can include the following extra notations:

  • @ — indicates a group instead of an individual user. For example, @admin indicates all users in the admin group.
  • * — represents “all”. For example, * in the user field represents all users.
  • % — represents an item the same as the item in the line above.

Now let us add the programs/processes we wish to limit. Edit /etc/cgrules.conf and add the following at the bottom:

/etc/cgrules.conf
*:firefox       cpu,memory      browsers/
*:hdparm        blkio   limitio/
sammy   blkio   limitio/
@admin:memhog  memory  limitmem/
*:cpuhog        cpu     limitcpu/

In the above lines, we are setting the following rules:

  • firefox processes run by any user will be automatically added to the browsers cgroup and limited in cpu and memory subsystems.
  • hdparm processes run by any user will be added to the limitio cgroup and will be limited in blkio subsystem according to the parameter values specified in that cgroup.
  • All processes run by user sammy will be added to the limitio cgroup and limited in blkio subsystem.
  • memhog processes run by anyone in the admin group will be added to the cgroup limitmem and limited in memory subsystem.
  • cpuhog processes run by any user will be added to the cgroup limitcpu and limited in cpu subsystem.

We need to start the cgred service for the cgrules configuration changes to take effect, do this using the command:

  1. sudo service cgred start

We also need to make sure the cgred service is enabled to start on system boot so that our rules persist across reboots:

  1. sudo chkconfig cgred on

Note: For services that support sysconfig, you could add the variable CGROUP_DAEMON="subsystem:control_group" in /etc/sysconfig/servicename instead of editing cgrules.conf file. For example, for a service like httpd, you could add CGROUP_DAEMON="blkio:/limitio" to the file /etc/sysconfig/httpd.conf to add the httpd processes to the limitio cgroup.

Step 4 — Testing

In this step, we will verify that the disk read throughput limit of 2MiB/s is enforced correctly according to the rule we added in cgrules.conf. To do this, we will install and run the hdparm tool. The hdparm tool can set and view hardware parameters of hard disk drives, measure read and write speeds, etc. Let us install hdparm using:

  1. sudo yum install hdparm

Now, let us run a command to measure the read speed of your hard disk /dev/vda:

  1. sudo hdparm --direct -t /dev/vda

You should see the following output:

/dev/vda:
 Timing O_DIRECT disk reads:   6 MB in  3.00 seconds =   2.00 MB/sec

The output shows a disk read throughput of 2MB/s. If you stop both the cgconfig and cgred services and run the hdparm command above once again, you can see the original/default read speed from when cgroup rules were not implemented.

Conclusion

This tutorial introduces only some of the basic things you could do with cgroups. It is also possible to create sub-cgroups, count and report the amount of resources consumed by a cgroup, suspend a group of processes using the freezer subsystem, and more.

Check out the following resources for more details:

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about our products

About the authors

Default avatar
Tammy Fox

editor


Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
2 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Hi, I have a doubt. In limitcpu cpu.shares is set to 400. approximately 40% of total CPU. So any processes assigned with this group, gets only 40% even when there are no other processes using CPU. 2nd question What happens to processes that are not included in this cgrules.conf. Are they automatically added or CPU is not controlled.

cat /etc/cgconfig.conf mount { cpuset = /cgroup/cpuset; cpu = /cgroup/cpu; cpuacct = /cgroup/cpuacct; memory = /cgroup/memory; devices = /cgroup/devices; freezer = /cgroup/freezer; net_cls = /cgroup/net_cls; blkio = /cgroup/blkio; } group limitcpu{ cpu { cpu.shares = 400; } } group limitmem{ memory { memory.limit_in_bytes = 512m; } } group limitio{ blkio { blkio.throttle.read_bps_device = “252:0 2097152”; } } group browsers{ cpu { cpu.shares = 200; } memory { memory.limit_in_bytes = 128m; } } When i restart it shows the bellow error [root@sysops Desktop]# service cgconfig restart Stopping cgconfig service: [ OK ] Starting cgconfig service: /sbin/cgconfigparser; error loading /etc/cgconfig.conf: Failed to remove a non-empty group Failed to parse /etc/cgconfig.conf or /etc/cgconfig.d [FAILED] [root@sysops Desktop]#

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Featured on Community

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more