If you have experience installing software from source code on your Linux server, you have probably come across the
make utility. This tool is primarily used to compile and build programs. It allows the source code’s author to lay out the steps required to build that specific project.
Although make was created to automate software compilation, the tool was engineered in a flexible enough fashion that it can be used to automate almost any task that you can accomplish from the command line. In this guide, we will discuss how you can repurpose make to automate repetitive tasks that happen in sequence.
A Linux environment of any kind will work for this tutorial. Package install instructions are provided for both Ubuntu/Debian Linux and for Red Hat/Rocky Linux.
Most Linux distributions allow you to install a compiler with a single command, but do not provide one by default.
On Ubuntu, you can install a package called
build-essential that will provide all the packages needed for a modern, well-supported compiler environment. Update your package sources and install the package with
- sudo apt update
- sudo apt install build-essential
On Rocky Linux or other Red Hat derivatives, you can install a group of packages called Development Tools to provide the same compiler functionality. Install the packages with
- dnf groups mark install "Development Tools"
- dnf groupinstall "Development Tools"
You can verify that a compiler is available by checking for the existence of the
make command on your system. In order to do that, use the
- which make
Now you have the tools that will allow you to take advantage of make in its usual capacity as well.
The primary way that the make command receives instructions is through the use of a
Makefiles are directory specific, meaning that make will search in the directory where it was called to find these files. We should put the Makefile in the root of whatever the task that we are going to be performing, or where it makes most sense to call the scripts we will write.
Inside the Makefile, we follow a specific format. Make uses the concepts of targets, sources, and commands in this way:
target: source command
The alignment and format of this is very important. We will discuss the format and meaning of each of these components here:
The target is a user-specified name to refer to a group of commands. Think of it as similar to a function in a programming language.
A target is aligned on the left-hand column, is a continuous word (no spaces), and ends with a colon (:).
When calling make, we can specify a target by typing:
- make target_name
Make will then check the Makefile and execute the commands associated with that target.
Sources are references to files or other targets. They represent prerequisites or dependencies for the target they are associated with.
For instance, you could have a section of your Makefile that looks like this:
target1: target2 target1_command target2: target2_command
In this example, we could call target1 like this:
- make target1
Make would then go to the Makefile and search for the
target1 target. It would then check to see if there are any sources specified.
It would find the
target2 source dependency and jump temporarily to that target.
From there, it would check if target2 had any sources listed. It does not, so it would proceed to execute
target2_command. At this point, make would reach the end of the
target2 command list and pass control back to the
target1 target. It would then execute
target1_command and exit.
Sources can be either files or targets themselves. Make uses file timestamps to see if a file has been changed since its last invocation. If a change to a source file has been made, that target is re-run. Otherwise, it marks the dependency as fulfilled and continues to the next source, or the commands if that was the only source.
The general idea is that by adding sources, we can build a sequential set of dependencies that must be executed before the current target. You can specify more than one source, separated by spaces, after any target. You can begin to see how you can specify elaborate sequences of tasks.
What gives the make command such flexibility is that the command portion of the syntax is very open-ended. You can specify any command to run under the target. You can add as many commands as needed.
Commands are specified on the line after the target declaration. They are indented by one tab character. Some versions of make are flexible about the way that you indent the command section, but in general, you should stick with a single tab to ensure that make will recognize your intent.
Make considers every indented line under the target definition to be a separate command. You can add as many indented lines and commands as you would like. Make will go through them one at a time.
There are a few things that we can place before commands to tell make to handle them differently:
-: A dash before a command tells make to not abort if an error is encountered. For instance, this could be useful if you want to execute a command on a file if it is present, and do nothing if it is not.
@: If you lead a command with the
@ symbol, the command call itself will not be printed to standard output. This is mainly used just to clean up the output that
Some additional features can help you create more complex rule chains in your Makefile.
Make recognizes variables (or macros), which behave as placeholders for substitution in your makefile. It is best to declare these at the top of your file.
The name of each variable is completely capitalized. Following the name, an equal sign assigns the name to the value on the right side. For instance, if we wanted to define an installation directory as
/usr/bin, we could add
INSTALLDIR=/usr/bin at the top of the file.
Later in the file, we can reference this location by using the
Another useful thing that we can do is allow commands to span multiple lines.
We can use any command or shell functionality within the command section. This includes escaping newline characters by ending the line with
target: source command1 arg1 arg2 arg3 arg4 \ arg5 arg6
This becomes more important if you take advantage of some of the shell’s more programmatic functionality, like if-then statements:
target: source if [ "condition_1" == "condition_2" ];\ then\ command to execute;\ another command;\ else\ alternative command;\ fi
This will execute this block as if it were a one line command. In fact, we could have written this as one line, but it improves readability considerably breaking it down like this.
If you are going to escape end of line characters, be certain to not have any extra spaces or tabs after the
\, or else you will get an error.
An additional feature that you can use for file processing is file suffixes. These are general rules that provide a way of processing files based on their extension.
For instance, if you want to process all .jpg files in a directory and convert them into .png files using the ImageMagick suite, we could have something like this in our Makefile:
.SUFFIXES: .jpg .png .jpg.png: @echo converting $< to $@ convert $< $@
There are a few things we need to look at here.
The first part is the
.SUFFIXES: declaration. This tells
make about all of the suffixes we will use in file suffixes. Some suffixes that are used often in compiling source code, like
.o files are included by default and do not need to be labeled in this declaration.
The next part is the declaration of the actual suffix rule. This takes the form of
This is not an actual target, but it will match any call for files with the second extension and build them out of the file in the first extension.
In our case, we can call
make like this to build a file called
file.png if there is a
file.jpg in our directory:
- make file.png
make will find the png file in the
.SUFFIXES declaration and see the rule for creating
.png files. It will then look for the target file with the
.png replaced by
.jpg in the directory. It will then execute the commands that follow.
The suffix rules use some variables that we have not been introduced to yet. These help make substitute different information based on what part of the process it is currently in:
$?: This variable contains the list of dependencies for the current target that are more recent than the target. These would be the targets that must be re-done before executing the commands under this target.
$@: This variable is the name of the current target. This allows us to reference the file you are trying to make, even though this rule was matched through a pattern.
$<: This is the name of the current dependency. In the case of suffix rules, this is the name of the file that is used to create the target. In our example, this would contain
$*: This file is the name of the current dependency with the matched extension stripped off. Consider this an intermediate stage between the target and source files.
We will create a Makefile that will do some image manipulations and then upload the files to our file server, so that our website can then display them.
If you would like to follow along, before you begin, ensure that you have the ImageMagick packages installed. These are command line tools for manipulating images and we will make use of them in our script.
On Ubuntu or Debian, update your package sources and install with
- sudo apt-get update
- sudo apt-get install imagemagick
On Red Hat or Rocky, you’ll need to add the
epel-release repo to get extra packages like this one, then install the package using
- dnf install epel-release
- dnf install ImageMagick
In your current directory, create a file called
- nano Makefile
Within this file, we will start implementing our conversion targets.
Our server has been set up to serve .png images exclusively. Because of this, we need to convert any .jpg files to .png before uploading.
As we learned above, a suffix rule is a great way of doing this. We will start out with the
.SUFFIX declaration that will list the formats we are converting between:
.SUFFIXES: .jpg .png.
Afterwards, we can make a rule that will change .jpg files into .png files. We can do this using the
convert command from the ImageMagick suite. The convert command syntax is
convert from_file to_file.
To implement this command, we need the suffix rule that specifies the format we’re starting with and the format we’re ending with:
.SUFFIXES: .jpg .png .jpg.png: ## This is the suffix rule declaration
Now that we have the rule that will match, we need to implement the actual convert step.
Because we don’t know exactly what filename will be matched here, we need to use the variables that we learned about. Specifically, we need to reference
$< as the original file, and
$@ as the file we are converting to. If we combine this with what we know about the convert command, we get this rule:
.SUFFIXES: .jpg .png .jpg.png: convert $< $@
Let’s add some functionality so that we can be told explicitly what is happening with an echo statement. We will include the
@ symbol before the new command and the command we already had in order to silence the actual command from being printed when it is executed:
.SUFFIXES: .jpg .png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
At this point, we should save and close the file so that we can test it.
Get a jpg file into the current directory. If you don’t have a file on hand, you can download one from the DigitalOcean website by using
- wget https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.png
- mv DO_Powered_by_Badge_blue.png badge.jpg
You can test whether your make file is working thus far by asking it to create a
- make badge.png
Outputconverting badge.jpg to badge.png using ImageMagick... conversion to badge.png successful!
Make will go to the Makefile, see the .png in the
.SUFFIXES declaration and then go to the suffix rule that matches. It then runs the commands listed.
At this point, make can create a .png file if we explicitly tell it that we want that file.
It would be better if it just created a list of .jpg files in the current directory and then converted those. We can do this by creating a variable that holds all of our files to be converted.
The best way to do this is with the wildcard directive like
We could just specify a target with a bash wildcard like
JPG_FILES=*.jpg, but this has a shortcoming. If there are no .jpg files, this actually tries to run the conversion commands on a file called
*.jpg, which will fail.
The wildcard syntax we mentioned above compiles a list of .jpg files in the current directory, and if none exist, it doesn’t set the variable to anything.
While we are doing this, we should try to handle a slight variation in .jpg files that is common. These image files are often seen with the .jpeg extension instead of .jpg. To handle these in an automated way, we can change their name in our program to .jpg files.
Instead of the above lines, we will use these two:
JPEG=$(wildcard *.jpg *.jpeg) ## Has .jpeg and .jpg files JPG=$(JPEG:.jpeg=.jpg) ## Only has .jpg files
The first line compiles a list of .jpg and .jpeg files in the current directory and stores them in a variable called
The second line references this variable and does a name translation to convert the names in the
JPEG variable that end with .jpeg into names that end with .jpg. This is done with the syntax
At the end of these two lines, we will have a new variable called
JPG which contains only .jpg filenames. Some of these files may not actually exist on the system, because they are actually .jpeg files (no actual renaming took place). This is okay because we are only using this list to make a new list of .png files we want to create:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png)
Now, we have a list of files we want to request in the variable
PNG. This list contains only .png filenames, because we did another name conversion. Now, every file that was a .jpg or .jpeg file in this directory has been used to compile a list of .png files we want to create.
We also need to update the
.SUFFIXES declaration and the suffixes rule to reflect that we are now handling .jpeg files:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
As you can see, we have added the .jpeg to the suffixes list and also included another suffix match for our rule.
We have quite a lot in our Makefile right now, but we don’t have any normal targets yet. Let’s fix that so that we can pass our
PNG list to our suffix rule:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png convert: $(PNG) .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
All this new target does is list the .png filenames that we gathered as a requirement. Make then sees if there’s a way that it can acquire the .png files and uses the suffix rule to do so.
Now, we can use this command to convert all of our .jpg and .jpeg files to .png files:
- make convert
Let’s add another target. Another task that is generally done when uploading images to a server is to resize them. Having your images the correct size will save your users from having to resize images on the fly when they request them.
An ImageMagick command called
mogrify can resize images in the way that we need. Let’s say the area where our images will be displayed on our site is 500px wide. We can convert for this area with the command:
- mogrify -resize 500\> file.png
This will resize any images larger than 500px wide to fit this area, but will not touch smaller images. This is what we want. As a target, we can add this rule:
resize: $(PNG) @echo resizing file... @mogrify -resize 648\> $(PNG) @echo resizing is complete!
We can add this to our file like this:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png convert: $(PNG) resize: $(PNG) @echo resizing file... @mogrify -resize 648\> $(PNG) @echo resizing is complete! .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
Now, we can string these two targets together as dependencies of another target:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png webify: convert resize convert: $(PNG) resize: $(PNG) @echo resizing file... @mogrify -resize 648\> $(PNG) @echo resizing is complete! .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
You may notice that resize implicitly will run the same commands as convert. We are going to specify them both though in case that is not always the case. The convert could in the future contain more elaborate processing.
The webify target now converts and resizes images.
Now that we have our images ready for the web, we can create a target to upload them to the static images directory on our server. We can do this by passing our list of converted files to
Our target will look something like this:
upload: webify scp $(PNG) root@ip_address:/path/to/static/images
This will upload all of our files to the remote server. Our file now looks like this:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png upload: webify scp $(PNG) root@ip_address:/path/to/static/images webify: convert resize convert: $(PNG) resize: $(PNG) @echo resizing file... @mogrify -resize 648\> $(PNG) @echo resizing is complete! .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
Let’s add a cleaning option to get rid of all of the local .png files after they have been uploaded to the remote server:
clean: rm *.png
Now, we can add another target at the top, that calls this one after we upload our files to the remote server. This will be the most complete target, and the one that we want to be default.
To specify this, we will put it as the first target available. This will be used as the default. We will call it
all by convention:
JPEG=$(wildcard *.jpg *.jpeg) JPG=$(JPEG:.jpeg=.jpg) PNG=$(JPG:.jpg=.png) .SUFFIXES: .jpg .jpeg .png all: upload clean upload: webify scp $(PNG) root@ip_address:/path/to/static/images webify: convert resize convert: $(PNG) resize: $(PNG) @echo resizing file... @mogrify -resize 648\> $(PNG) @echo resizing is complete! clean: rm *.png .jpeg.png .jpg.png: @echo converting $< to $@ using ImageMagick... @convert $< $@ @echo conversion to $@ successful!
With these last touches, if you enter the directory with the Makefile and .jpg or .jpeg files, you can call make without any arguments to process your files, send them to your server, and then delete the .png files you uploaded.
As you can see, it is possible to string together tasks and also to cherry pick a process up to a certain point. For instance, if you only want to convert your files and need to host them on a different server, you can just use the
At this point, you should have a good idea of how to use Makefiles in general. More specifically, you should know how to use make as a tool for automating most kinds of procedures.
While in some cases it may work better to write a script, Makefiles are a way of setting up a structured, hierarchical relationship between processes. Learning how to leverage this tool can help make repetitive tasks more manageable.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.