Logrotate is a system utility that manages the automatic rotation and compression of log files. If log files were not rotated, compressed, and periodically pruned, they could eventually consume all available disk space on a system.
Logrotate is installed by default on Ubuntu 20.04, and is set up to handle the log rotation needs of all installed packages, including rsyslog
, the default system log processor.
In this article, we will explore the default Logrotate configuration, then configure log rotation for a fictional custom application.
This tutorial assumes you have an Ubuntu 20.04 server, with a non-root sudo-enabled user, as described in Initial Server Setup with Ubuntu 20.04.
Logrotate is available on many other Linux distributions as well, but the default configuration may be quite different. Other sections of this tutorial will still apply as long as your version of Logrotate is similar to Ubuntu 20.04’s. Follow Step 1 to determine your Logrotate version.
Log into your server as your sudo-enabled user to begin.
Logrotate is installed by default on Ubuntu. However, if you need to install it, run the following commands to update your package list and retrieve the package:
- sudo apt update
- sudo apt install logrotate
If you’re using a non-Ubuntu server, first make sure Logrotate is installed by asking for its version information:
- logrotate --version
Outputlogrotate 3.14.0
Default mail command: /usr/bin/mail
Default compress command: /bin/gzip
Default uncompress command: /bin/gunzip
Default compress extension: .gz
Default state file path: /var/lib/logrotate/status
ACL support: yes
SELinux support: yes
If Logrotate is installed but the version number is significantly different, you may have issues with some of the configuration options that are explored in this tutorial. Refer to the documentation for your specific version of Logrotate by reading its manual (man
) page:
- man logrotate
You can also consult the online version of the Logrotate documentation. Next we’ll look at Logrotate’s default configuration structure on Ubuntu.
Logrotate’s configuration information can generally be found in two places on Ubuntu:
/etc/logrotate.conf
: this file contains some default settings and sets up rotation for a few logs that are not owned by any system packages. It also uses an include
statement to pull in configuration from any file in the /etc/logrotate.d
directory./etc/logrotate.d/
: this is where any packages you install that need help with log rotation will place their Logrotate configuration. On a standard install you should already have files here for core system tools like apt
, dpkg
, rsyslog
and so on.By default, logrotate.conf
will configure weekly log rotations, with log files owned by the root user and the syslog group, with four log files being retained at a time(rotate 4
), and new empty log files being created after the current one is rotated (create
).
Let’s take a look at a package’s Logrotate configuration file in /etc/logrotate.d
. cat
the file for the apt
package utility:
- cat /etc/logrotate.d/apt
Output/var/log/apt/term.log {
rotate 12
monthly
compress
missingok
notifempty
}
/var/log/apt/history.log {
rotate 12
monthly
compress
missingok
notifempty
}
This file contains configuration blocks for two different log files in the /var/log/apt/
directory: term.log
and history.log
. They both have the same options. Any options not set in these configuration blocks will inherit the default values or those set in /etc/logrotate.conf
. Any setting in a logrotate file will override logrotate’s default values, which are configured in /etc/logrotate.conf
. The options set for the apt
logs are:
rotate 12
: keep twelve old log files. This overrides the rotate 4
default.monthly
: rotate once a month. This overrides the weekly
default.compress
: compress the rotated files. this uses gzip
by default and results in files ending in .gz
. The compression command can be changed using the compresscmd
option.missingok
: don’t write an error message if the log file is missing.notifempty
: don’t rotate the log file if it is empty.These configuration files also inherit the default create
behaviour, which instructs Logrotate to create new logs after rotation. This could be overridden with nocreate
, although that would effectively disable most of the other functionality.
There are many more configuration options available. You can read about all of them by typing man logrotate
on the command line to bring up Logrotate’s manual page.
Next, we’ll set up a configuration file to handle logs for a fictional service called your-app
.
To manage log files using logrotate for applications outside of the pre-packaged and pre-configured system services, we have two options:
/etc/logrotate.d/
. This will be run daily as the root user along with all the other standard Logrotate jobs.hourly
configuration in /etc/logrotate.d/
would be ineffective, because the system’s Logrotate setup only runs once a day).Let’s walk through these two options with some example setups.
/etc/logrotate.d/
We want to configure log rotation for a fictional web server that puts an access.log
and error.log
into /var/log/your-app/
. It runs as the www-data
user and group.
To add a configuration for the your-app
log files to /etc/logrotate.d/
, first open up a new file in the /etc/logrotate.d
directory using nano
or your preferred editor:
- sudo nano /etc/logrotate.d/your-app
Add the following lines to your new configuration file:
/var/log/your-app/*.log {
daily
missingok
rotate 14
compress
notifempty
create 0640 www-data www-data
sharedscripts
postrotate
systemctl reload your-app
endscript
}
Some of the new configuration directives in this file are:
create 0640 www-data www-data
: this creates a new empty log file after rotation, with the specified permissions (0640
), owner (www-data
), and group (also www-data
).sharedscripts
: this flag means that any scripts added to the configuration are run only once per run, instead of for each file rotated. Since the path /var/log/your-app/*.log
includes a wildcard *
, this configuration would match any number of log files in the your-app
directory. Without the sharedscripts
option, the script specified in postrotate
would run each time logrotate processes a log file without this option.postrotate
to endscript
: this block contains a script to run after the log file is rotated. In this case we’re reloading our example app. This is sometimes necessary to get your application to switch over to the newly created log file.
Note that postrotate
runs before logs are compressed. Compression could take a long time, and your software should switch to the new log file immediately. For tasks that need to run after logs are compressed, use the lastaction
block instead.To save and quit nano
, press Ctrl+X
, and when prompted, Y
and then Enter
. You can test the config file by doing a dry run:
- sudo logrotate /etc/logrotate.conf --debug
This command calls logrotate
, points it to the standard configuration file, and turns on debug mode.
Information will print out about which log files Logrotate is handling and what it would have done to them. If all looks well, you’re done. The standard Logrotate job will run once a day and include your new configuration.
Next, we’ll try a setup that doesn’t use Ubuntu’s default configuration at all.
In this example we have an app running as our user sammy
, generating logs that are stored in /home/sammy/logs/
. We want to rotate these logs hourly, so we need to set this up outside of the /etc/logrotate.d
structure provided by Ubuntu.
First, we’ll create a configuration file in our home directory. Open it in a text editor:
- nano /home/sammy/logrotate.conf
Then paste in the following configuration:
/home/sammy/logs/*.log {
hourly
missingok
rotate 24
compress
create
}
Save and close the file. We’ve encountered all these options in previous steps, but let’s summarize: this configuration will rotate the files hourly, compressing and keeping twenty-four old logs and creating a new log file to replace the rotated one.
You’ll need to customize the configuration to suit your application, but this is a good start.
To test that the configuration works, let’s make a log file. First cd
to your user’s home directory using the cd ~
command. Then create a directory for the logs using the mkdir
command. Finally create an empty file in the logs
directory using the touch
command. Run the following commands in order to complete these steps:
- cd ~
- mkdir logs
- touch logs/access.log
Now that we have a blank log file in the right spot, let’s run the logrotate
command.
Because the logs are owned by sammy
we don’t need to use sudo
. However, we do need to specify a state file. This file records what logrotate
found and any actions that it took the last time it ran, so that it knows what to do the next time it runs. This state tracking is handled for us when using the default /etc/logrotate.conf
configuration. The state file is stored in /var/lib/logrotate/status
. Since we are not using the default configuration, we’ll need to configure the state file location manually.
We’ll have Logrotate put the state file right in our home directory for this example. It can go anywhere that’s accessible and convenient. Run the following command to use the /home/sammy/logrotate.conf
that you created and to record the state that logrotate encounters:
logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state --verbose
Outputreading config file /home/sammy/logrotate.conf
Handling 1 logs
rotating pattern: /home/sammy/logs/*.log hourly (24 rotations)
empty log files are rotated, old logs are removed
considering log /home/sammy/logs/access.log
log does not need rotating
The --verbose
flag will print out detailed information about what Logrotate is doing. In this case it did not rotate anything. This is Logrotate’s first time encountering this log file and it is zero hours old so shouldn’t be rotated.
If we examine at the state file using the cat
utility, we’ll note that Logrotate recorded some information about the run:
- cat /home/sammy/logrotate-state
Outputlogrotate state -- version 2
"/home/sammy/logs/access.log" 2021-12-3-19:0:0
Logrotate noted the logs that it saw and when it last considered them for rotation. If we run this same command one hour later, the log will be rotated as expected.
If you want to force Logrotate to rotate the log file when it otherwise would not have, use the --force
flag:
- logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state --verbose --force
This is useful when testing postrotate
and other scripts.
Finally, we need to set up a cron job to run Logrotate every hour. Open your user’s crontab:
- crontab -e
This will open a text file. If it is your first time using cron, you may be prompted to choose a default text editor. If you do not have a preference, we recommend nano
for new users. There may be some comments already in the file that explain the cron syntax. Move the cursor down to a new blank line at the end of the file and add the following:
crontab14 * * * * /usr/sbin/logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state
This task will run on the 14th minute of every hour, every day. It runs nearly the same logrotate
command we ran previously, though we expanded logrotate
to its full path of /usr/sbin/logrotate
to be safe. It’s good practice to use full paths when writing cron jobs. To learn more about cron, you can review our other tutorials.
Save the file and exit. This will install the crontab and our task will run on the specified schedule.
If we revisit our log directory in about an hour we should find the rotated and compressed log file access.log.1.gz
(or .2.gz
if you ran Logrotate with the --force
flag).
In this tutorial we verified our Logrotate version, explored the default Ubuntu Logrotate configuration, and set up two different types of custom configurations. To learn more about the command line and configuration options available for Logrotate, you can read its manual page by running man logrotate
in your terminal or by visiting the online documentation.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Sign up for Infrastructure as a Newsletter.
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.