There are three main commands, and a common log location that you can use to get started troubleshooting HAProxy errors. Generally when you are troubleshooting HAProxy, you will use these commands in the order indicated here, and then examine the log file for specific diagnostic data.
The commands and log that you will commonly use to troubleshoot HAProxy across most Linux distributions are:
systemctl
- Used to control and interact with Linux services via the systemd
service manager.journalctl
- Used to query and view the logs that are generated by systemd
.haproxy
- When troubleshooting, this command is used to check HAProxy’s configuration./var/log/haproxy.log
- This file contains log entries from HAProxy itself detailing TCP and HTTP traffic that is being handled by the server.These commands, how to use them, and HAProxy’s logs where you can find additional information about errors are described in further detail in the following sections.
systemctl
Commands for HAProxyTo troubleshoot common HAProxy errors using the systemd
service manager, the first step is to inspect the state of the HAProxy processes on your system. The following systemctl
commands will query systemd
for the state of HAProxy’s processes on most Linux distributions.
- sudo systemctl status haproxy.service -l --no-pager
The -l
flag will ensure that output is not truncated or ellipsized. The --no-pager
flag will make sure that output will go directly to your terminal without requiring any interaction on your part to view it. If you omit the --no-pager
flag you will be able to scroll through the output using arrow keys, or the page up and down keys. To quit from the pager use the q
key. You should receive output like this:
Output● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/lib/systemd/system/haproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-08-20 19:30:11 UTC; 5s ago
Docs: man:haproxy(1)
file:/usr/share/doc/haproxy/configuration.txt.gz
Process: 487 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=0/SUCCESS)
Main PID: 488 (haproxy)
Tasks: 2 (limit: 2344)
. . .
Aug 19 21:31:46 d6cdd0c71489 systemd[1]: Started HAProxy Load Balancer.
Your output may be slightly different depending on which Linux distribution you are using, but in any case, make a note of the Active
line in the output. If your HAProxy server does not show active (running)
as highlighted in the example output but you expect it should, there may be an error. Typically if there is a problem, you will have a line like the following in your output (note the highlighted failed
portion):
Example Error Output Active: failed (Result: exit-code) since Thu 2020-08-20 19:32:26 UTC; 6s ago
If there is a problem with your HAProxy process or configuration you can troubleshoot it further using the journalctl
command.
journalctl
Commands for HAProxyTo inspect the systemd
logs for HAProxy, you can use the journalctl
command. The systemd
logs for HAProxy will usually indicate whether there is a problem with starting or managing the HAProxy process.
These logs are separate from HAProxy’s request and error logs. journalctl
displays logs from systemd
that describe the HAProxy service itself, from startup to shutdown, along with any process errors that may be encountered along the way.
- sudo journalctl -u haproxy.service --since today --no-pager
The --since today
flag will limit the output of the command to log entries beginning at 00:00:00 of the current day only. Using this option will help restrict the volume of log entries that you need to examine when checking for errors. You should receive output like the following (there may be a few extra lines between the Starting
and Started
lines depending on your Linux distribution):
OutputAug 20 19:37:08 d6cdd0c71489 systemd[1]: Starting HAProxy Load Balancer...
. . .
Aug 20 19:37:08 d6cdd0c71489 systemd[1]: Started HAProxy Load Balancer.
If there is an error, you will have a line in the output that is similar to the following, with the main difference between Linux distributions being the highlighted yourhostname
portion:
Example Error OutputAug 20 19:32:25 yourhostname systemd[1]: Failed to start HAProxy Load Balancer.
If your HAProxy server has errors in the journalctl
logs like the previous example, then the next step to troubleshoot possible issues is investigating HAProxy’s configuration using the haproxy
command line tool.
haproxy
To troubleshoot HAProxy configuration issues, use the haproxy -c
command. The tool will parse your HAProxy files and detect any errors or missing settings before attempting to start the server.
Run the command like this on Ubuntu, Debian, CentOS, and Fedora based distributions. Be sure to change the path to the configuration file if you are using a different filename or location:
- sudo haproxy -c -f /etc/haproxy/haproxy.cfg
A working HAProxy configuration will result in output like the following:
OutputConfiguration file is valid
If there is an error in your HAProxy configuration, like a typo or misplaced directive, haproxy -c
will detect it and attempt to notify you about the problem.
For example, attempting to use the bind
directive in haproxy.cfg
in the wrong location will result in messages like the following:
Example Error Output[ALERT] 232/194354 (199) : parsing [/etc/haproxy/haproxy.cfg:13] : unknown keyword 'bind' in 'global' section
[ALERT] 232/194354 (199) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT] 232/194354 (199) : Fatal errors found in configuration.
In this example the bind
directive is misplaced inside a global
configuration section, so HAProxy generates the unknown keyword
error. The message also includes a line number 13
, so that you can edit the file and fix or remove the erroneous line without having to search through the file.
Learning how to use haproxy -c
to detect and fix errors is useful when you are troubleshooting an existing error, or before you reload HAProxy with an edited configuration that may contain errors.
HAProxy log files are a very helpful resource for troubleshooting. Generally, any error that you receive in a browser or other HTTP client will have a corresponding entry in HAProxy’s logs. Sometimes HAProxy will also output errors related to configuration and other debugging information to its log files.
On Ubuntu and Debian based Linux distributions, the haproxy
package includes scripts that configure log output in /var/log/haproxy.log
.
On CentOS, Fedora, and other RedHat-derived Linux distributions, haproxy
does not output to a log file by default. To log HAProxy output logs to /var/log/haproxy.log
, follow this quickstart tutorial, How To Configure HAProxy Logging with Rsyslog on CentOS 8.
When you are troubleshooting HAProxy using its log file, examine /var/log/haproxy.log
for errors using a tool like tail
or less
. For example, to view the last two lines of the log using tail
, run the following command:
- sudo tail -n 2 /var/log/haproxy.log
An example error will resemble something like the following lines, regardless of which Linux distribution you are using to run your HAProxy server:
Log ExamplesAug 20 19:36:21 d6cdd0c71489 haproxy[19202]: [ALERT] 258/134605 (19202) : Proxy 'app', server 'app1' [/etc/haproxy/haproxy.cfg:88] verify is enabled by default but no CA file specified. If you're running on a LAN where you're certain to trust the server's certificate, please set an explicit 'verify none' statement on the 'server' line, or use 'ssl-server-verify none' in the global section to disable server-side verifications by default.
Aug 20 19:36:22 d6cdd0c71489 haproxy[4451]: 203.0.113.1:54428 [20/Aug/2020:19:36:22.288] main app/<NOSRV> 0/-1/-1/-1/1 503 212 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"
These example lines are just for illustration purposes. If you are diagnosing errors with your HAProxy server, chances are the lines in your logs will have different contents than these. Some lines will include success responses and other non-critical diagnostic entries.
Regardless of your Linux distribution, the format of the lines in your HAProxy logs will include any HTTP status codes that are returned to clients, along with requesting IPs and the status of backend servers.
Once you have an idea of what might be causing problems with your HAProxy server you can continue researching and troubleshooting the issue. The HTTP status code and text description are especially useful, since they give you explicit and specific terms that you can use to narrow down the range of possible causes of a problem.
Troubleshooting HAProxy errors can range from diagnosing errors with the service itself, to locating misconfigured options for modules, or to examining customized access control rules in detail. This introduction to diagnosing issues with HAProxy explained how to use a number of utilities to help narrow down the possible causes of errors. Usually, you will use these utilities in the same order, although you can always skip some, or start directly with examining logs if you have a general idea of what the problem might be.
However, as a general sequence for troubleshooting, it helps to be methodical and use these tools in the order described. Start troubleshooting with systemctl
to examine the state of the HAProxy server. If you need more information, examine the systemd
logs for HAProxy using the journalctl
command. If the issue is still not apparent after checking journalctl
, testing HAProxy’s configuration using haproxy -c -f /etc/haproxy/haproxy.cfg
is the next step. Finally, for in-depth troubleshooting, examining HAProxy’s log files will usually indicate a specific error, with helpful diagnostic messages and error codes.
The rest of the tutorials in this series will examine some common errors that you may encounter when using HAProxy in more detail.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This tutorial series explains how to troubleshoot and fix some of the most common errors that you may encounter when using the HAProxy TCP and HTTP proxy server.
Each tutorial in this series includes descriptions of common HAProxy configuration, network, filesystem, or permission errors. The series begins with an overview of the commands and log files that you can use to troubleshoot HAProxy. Subsequent tutorials examine specific errors in detail.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Sign up for Infrastructure as a Newsletter.
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.