We hope you find this tutorial helpful. In addition to guides like this one, we provide simple cloud infrastructure for developers. Learn more →

How To Set Up a Local OSSEC Installation on Debian 8

Posted May 14, 2015 8.6k views Security Debian


OSSEC is an open-source, host-based intrusion detection system (HIDS) that performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, time-based alerting, and active response. It's the application to install on your server if you want to keep an eye on what's happening inside it.

OSSEC can be installed to monitor just the server it's installed on, which is a local installation in OSSEC's parlance, or be installed as a server to monitor one or more agents. In this tutorial, you'll learn how to install OSSEC to monitor the Debian 8 server it is installed on, that is, a local OSSEC installation.


To follow this tutorial, you will need:

  • A Debian 8 Droplet with a sudo non-root user and SSH keys, which you can set up by following this tutorial.

Step 1 — Installing Required Packages

In this step, we'll install the packages required for OSSEC. First, update the package database.

  • sudo apt-get update

Then install available updates.

  • sudo apt-get -y upgrade

Finally, install OSSEC's dependencies (build-essential and inotify-toops) and ntp, which is a Network Time Protocol service.

  • sudo apt-get install build-essential inotify-tools ntp

Finally, enable the NTP service. This helps the server keep accurate time automatically.

  • sudo systemctl start ntp

Step 2 — Enabling A Firewall

A fresh installation of a Debian 8 server has no active firewall application. In this step, we'll learn how to enable the IPTables firewall application and make sure that runtime rules persist after a reboot.

The easiest method of accomplishing that is to install the iptables-persistent package using:

  • sudo apt-get install -y iptables-persistent

After authenticating, you'll be prompted to save the IPv4 and IPv6 firewall rules to separate files. Press ENTER at both prompts to accept the default locations, which are /etc/iptables/rules.v4 and /etc/iptables/rules.v6.

By default, there are no rules in those files, so we'll have to create them to keep the server protected and the open SSH connection up. We're only interested in the IPv4 rules, so we'll only modify the rules.v4 rules file.

Open the rules.v4 rules file using nano or your favorite text editor.

  • sudo nano /etc/iptables/rules.v4

The full contents of that file look like this:

Original /etc/iptables/rules.v4
# Generated by iptables-save v1.4.21 on Sat May  9 01:27:00 2015
:INPUT ACCEPT [5722:416593]
:OUTPUT ACCEPT [4372:503060]
# Completed on Sat May  9 01:27:00 2015

The following default ruleset will be enough to keep the server protected and the SSH connection up, so copy and paste it just between the :OUTPUT ACCEPT [4372:503060] and COMMIT lines. These rules were taken from the official Debian documentation; you can see what each rule does by the inline comments.

#  Allow all loopback traffic. Drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT

#  Accept all established inbound connections

#  Allow all outbound traffic

#  Uncomment the next two lines to allow HTTP and HTTPS connections
#-A INPUT -p tcp --dport 80 -j ACCEPT
#-A INPUT -p tcp --dport 443 -j ACCEPT

#  Allow SSH connections. If you changed your SSH port, do same here.
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

#  Allow ping
-A INPUT -p icmp --icmp-type echo-request -j ACCEPT

#  Log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

#  Drop all other inbound - default deny unless explicitly allowed policy

Save and close the file. Then, to apply the new ruleset, restart iptables-persistent.

  • sudo systemctl restart netfilter-persistent

You can now verify that the rules are in place with this command.

  • sudo iptables -L

Your output will look like this:

iptables -L output
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere     
REJECT     all  --  anywhere             loopback/8           reject-with icmp-port-unreachable

. . .         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere    

Step 3 — Downloading and Verifying OSSEC

OSSEC is delivered as a compressed tarball. In this step, you'll download it and its checksum file, which is used to verify that the tarball has not been tampered with. You can check the project's website for the latest version. At the time of this writing, OSSEC 2.8.1 is the latest stable release.

To download the tarball, type:

  • wget -U ossec

Then download the checksum file using

  • wget -U ossec

After downloading both files, verify the md5sum of the compressed tarball.

  • md5sum -c ossec-hids-2.8.1-checksum.txt

The output should be:

md5sum output
ossec-hids-2.8.1.tar.gz: OK
md5sum: WARNING: 1 line is improperly formatted

Follow that by verifying the SHA1 checksum.

  • sha1sum -c ossec-hids-2.8.1-checksum.txt

Its output should be:

sha1sum output
ossec-hids-2.8.1.tar.gz: OK
sha1sum: WARNING: 1 line is improperly formatted

In each case, ignore the WARNING line. The OK line is what confirms that the file is good.

Step 4 — Installing OSSEC

In this step, we will install OSSEC. To begin, first untar it.

  • tar xf ossec-hids-2.8.1.tar.gz

It will be unpacked into a directory called ossec-hids-2.8.1. Change into that directory.

  • cd ossec-hids-2.8.1

Note: There is a bug in OSSEC that was introduced in version 2.8.1. The bug causes it to overwrite the contents of the /etc/hosts.deny file. There is already a permanent fix in version 2.9, which should be released soon.

To fix the /etc/hosts.deny bug, open the file in the /var/ossec/active-response directory after extracting OSSEC from the downloaded tarball.

  • nano active-response/

Towards the end of the file, look for two lines in the code as below that begin with TMP_FILE =, underneath the # Deleting from hosts.deny comment. Edit both lines to remove the spaces on either side of the = sign so the code block looks like this.

Modified code block
# Deleting from hosts.deny
elif [ "x${ACTION}" = "xdelete" ]; then
   TMP_FILE=`mktemp /var/ossec/ossec-hosts.XXXXXXXXXX`
   if [ "X${TMP_FILE}" = "X" ]; then
     # Cheap fake tmpfile, but should be harder then no random data
     TMP_FILE="/var/ossec/ossec-hosts.`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -1 `"

Save and close the file. This concludes the bug fix.

Next, begin the installation.

  • sudo ./

Throughout the installation process, you'll be prompted to provide some input. You'll first be prompted to select the installation language, which, by default, is English (en). Press ENTER if that's your preferred language. Otherwise, type in the 2 letters from the list of supported languages first. Then, press ENTER again to start the installation.

Question 1 will ask What kind of installation do you want (server, agent, local, hybrid or help)?. Type local and press ENTER.

For all of the following questions, press ENTER to accept the default, but note that question 3.1 will prompt you for your email address. Enter it, and the installer will use it to find the corresponding SMTP server automatically.

If installation is successful, the last few lines of the post-installation output should read:

OSSEC successful installation output
- Configuration finished properly.

 - To start OSSEC HIDS:
                /var/ossec/bin/ossec-control start

 - To stop OSSEC HIDS:
                /var/ossec/bin/ossec-control stop

 - The configuration can be viewed or modified at /var/ossec/etc/ossec.conf

. . .

Step 4 — Customizing OSSEC's Email Settings

Here we are going to verify that the email credentials specified in the previous step and the one that OSSEC auto-configured are correct.

The email settings are in OSSEC's main configuration file - ossec.conf, which is in the `/var/ossec/etc directory. To access and modify any OSSEC file, you first need to switch to the root user.

  • sudo su

Now that you're root, change into the directory where OSSEC's configuration file is.

  • cd /var/ossec/etc

Then make a backup copy of the configuration file.

  • cp ossec.conf ossec.conf.00

Open the original file using the nano text editor or your preferred text editor.

  • nano ossec.conf

The email settings are at the top of the file, which looks like this.


< email_to> is the email you gave during installation. Alerts will be sent to that email address, and < smtp_server> is the SMTP server auto-discovered by the installation script. You don't have to change these values.

< email_from> is the email address that OSSEC's alerts would appear to be coming from. By default, it is created based on OSSEC's mail user account and the hostname of the server. You should change this to a valid email address to reduce the odds of your emails being tagged as spam by your email provider's SMTP server. Note that < email_to> and < email_from> can be the same if the receiving SMTP server does not have a strict spam policy.

After modifying the email settings, save and close the file. Then start OSSEC.

  • /var/ossec/bin/ossec-control start

Check your inbox for an email that says that OSSEC has started. If you receive an email from your OSSEC installation, then you know that future alerts will also reach your inbox. If you don't, check your spam folder.

Step 6 — Adding Alerts

By default, OSSEC will issue alerts on file modifications and other activities on the server, but it will not alert on new file additions and also will not alert in real-time — only after the scheduled system scan, which is 79200 seconds (or 22 hours) by default. In this section, we will modify OSSEC so that it can alert on file additions and in real-time.

First, open ossec.conf.

  • nano ossec.conf

Scroll down to the < syscheck> section. Just under the <frequency> tag, add < alert\_new\_files>yes< /alert\_new\_files >.

    <!-- Frequency that syscheck is executed - default to every 22 hours -->

While you still have ossec.conf open, take a look at the list of system directories that OSSEC monitors, which is just under the last line you modified. Add report_changes="yes" realtime="yes" to both directory tags.

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>

Aside from the default list of directories that OSSEC has been configured to monitor, you may also add any that you wish to monitor. For example, you may add your home directory. To do that, add this new line right under other directory lines, substituting in your username.

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/home/<^sammy</directories><^>

Now save and close ossec.conf.

The next file to modify is in the /var/ossec/rules directory, so change to that directory.

  • cd /var/ossec/rules

The /var/ossec/rules directory contains many XML files, including ossec_rules.xml, which contains OSSEC's default rule definitions, and local_rules.xml, which is where you can add custom rules.

In ossec_rules.xml, the rule that fires when a file is added to a monitored directory is rule 554. By default, OSSEC does not send out alerts when that rule is triggered, so the task here is to change that behavior. Here's what rule 554 looks like by default:

Rule 554 from /var/ossec/rules/ossec_rules.xml
<rule id="554" level="0">
<description>File added to the system.</description>

OSSEC does not send out an alert if a rule is set to level 0, so we will copy that rule to localrules.xml and modify it to trigger an alert. To do that, open `localrules.xml`.

  • nano local_rules.xml

Add the following at the end of the file, before the line with the </group> tag.

<rule id="554" level="7" overwrite="yes">
<description>File added to the system.</description>

</group> <!-- SYSLOG,LOCAL -->

<!-- EOF -->

Save and close the file, then restart OSSEC to apply the changes.

  • /var/ossec/bin/ossec-control restart

You should now be receiving alerts whenever a file or added, modified, or deleted. Note that OSSEC does not alert on file additions in real-time, only after a full system scan.

Step 6 (Optional) — Stopping IPTables Denied Alerts

In this step, which is optional but highly recommended, we will configure OSSEC to not alert on IPTables denied messages.

At the beginning of this tutorial, we enabled the IPTables firewall. After installing OSSEC, it will alert on rule 1002, which is triggered when IPTables denies an attacker and logs the incidence to syslog. While it's good to know when an attacker has been foiled, such alerts can number in the hundreds every day and clog up your inbox.

To remove these alerts, we need to customize rule 1002. That rule is found in /var/ossec/rules/syslog_rules.xml, and looks like this:

Rule 1002 in /var/ossec/rules/syslog_rules.xml
<rule id="1002" level="2">
    <description>Unknown problem somewhere in the system.</description>

Open syslog_rules.xml for editing.

  • nano /var/ossec/rules/syslog_rules.xml

Find the BAD_WORDS variable, which is defined at the top of that file and contains a number of keywords. It looks like this:

BAD_WORDS definition in in /var/ossec/rules/syslog_rules.xml
<var name="BAD_WORDS">core_dumped|failure|error|attack|bad |illegal |denied|refused|unauthorized|fatal|failed|Segmentation Fault|Corrupted</var>

Below the BAD_WORDS definition, copy and paste this new variable, IGNORED_WORD. The variable contains just one keyword.

<var name="BAD_WORDS">core_dumped|failure|error|attack|bad |illegal |denied|refused|unauthorized|fatal|failed|Segmentation Fault|Corrupted</var>
<var name="IGNORED_WORD">denied</var>

Then, we'll use the new IGNORED_WORD variable in a customized version of rule 1002, which we'll call rule 100031. The complete rule is shown below. Copy and paste it to the bottom of the file, before the line with the group tag.

<rule id="100031" level="0">
     <description>Ignored IPTables deny messages.</description>


<!-- EOF -->

Save and close the file. With the variable and customized rule in place, restart OSSEC.

  • /var/ossec/bin/ossec-control restart

With that, OSSEC should stop sending alerts of IPTables denied messages.


That's all it takes to install and configure a local OSSEC on a Debian 8 server. There is a lot of further customization available, which you can explore in the projects official documentation.

Creative Commons License