How To Set Up a Local OSSEC Installation on Debian 8

Introduction

OSSEC is an open-source, host-based intrusion detection system (HIDS) that performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, time-based alerting, and active response. It’s the application to install on your server if you want to keep an eye on what’s happening inside it.

OSSEC can be installed to monitor just the server it’s installed on, which is a local installation in OSSEC’s parlance, or be installed as a server to monitor one or more agents. In this tutorial, you’ll learn how to install OSSEC to monitor the Debian 8 server it is installed on, that is, a local OSSEC installation.

Prerequisites

To follow this tutorial, you will need:

• A Debian 8 Droplet with a sudo non-root user and SSH keys, which you can set up by following this tutorial.

Step 1 — Installing Required Packages

In this step, we’ll install the packages required for OSSEC. First, update the package database.

sudo apt-get update

Then install available updates.

sudo apt-get -y upgrade

Finally, install OSSEC’s dependencies (build-essential and inotify-toops) and ntp, which is a Network Time Protocol service.

sudo apt-get install build-essential inotify-tools ntp

Finally, enable the NTP service. This helps the server keep accurate time automatically.

sudo systemctl start ntp

Step 2 — Enabling A Firewall

A fresh installation of a Debian 8 server has no active firewall application. In this step, we’ll learn how to enable the IPTables firewall application and make sure that runtime rules persist after a reboot.

The easiest method of accomplishing that is to install the iptables-persistent package using:

sudo apt-get install -y iptables-persistent

After authenticating, you’ll be prompted to save the IPv4 and IPv6 firewall rules to separate files. Press ENTER at both prompts to accept the default locations, which are /etc/iptables/rules.v4 and /etc/iptables/rules.v6.

By default, there are no rules in those files, so we’ll have to create them to keep the server protected and the open SSH connection up. We’re only interested in the IPv4 rules, so we’ll only modify the rules.v4 rules file.

Open the rules.v4 rules file using nano or your favorite text editor.

sudo nano /etc/iptables/rules.v4

The full contents of that file look like this:

# Generated by iptables-save v1.4.21 on Sat May  9 01:27:00 2015
*filter
:INPUT ACCEPT [5722:416593]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [4372:503060]
COMMIT
# Completed on Sat May  9 01:27:00 2015

The following default ruleset will be enough to keep the server protected and the SSH connection up, so copy and paste it just between the :OUTPUT ACCEPT [4372:503060] and COMMIT lines. These rules were taken from the official Debian documentation; you can see what each rule does by the inline comments.

#  Allow all loopback traffic. Drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 -j REJECT

#  Accept all established inbound connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

#  Allow all outbound traffic
-A OUTPUT -j ACCEPT

#  Uncomment the next two lines to allow HTTP and HTTPS connections
#-A INPUT -p tcp --dport 80 -j ACCEPT
#-A INPUT -p tcp --dport 443 -j ACCEPT

#  Allow SSH connections. If you changed your SSH port, do same here.
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

#  Allow ping
-A INPUT -p icmp --icmp-type echo-request -j ACCEPT

#  Log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

#  Drop all other inbound - default deny unless explicitly allowed policy
-A INPUT -j DROP
-A FORWARD -j DROP

Save and close the file. Then, to apply the new ruleset, restart iptables-persistent.

sudo systemctl restart netfilter-persistent

You can now verify that the rules are in place with this command.

sudo iptables -L

Your output will look like this:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere     
REJECT     all  --  anywhere             loopback/8           reject-with icmp-port-unreachable

. . .         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere    

Step 3 — Downloading and Verifying OSSEC

OSSEC is delivered as a compressed tarball. In this step, you’ll download it and its checksum file, which is used to verify that the tarball has not been tampered with. You can check the project’s website for the latest version. At the time of this writing, OSSEC 2.8.1 is the latest stable release.

To download the tarball, type:

wget -U ossec http://www.ossec.net/files/ossec-hids-2.8.1.tar.gz

Then download the checksum file using

wget -U ossec http://www.ossec.net/files/ossec-hids-2.8.1-checksum.txt

After downloading both files, verify the md5sum of the compressed tarball.

md5sum -c ossec-hids-2.8.1-checksum.txt

The output should be:

ossec-hids-2.8.1.tar.gz: OK
md5sum: WARNING: 1 line is improperly formatted

Follow that by verifying the SHA1 checksum.

sha1sum -c ossec-hids-2.8.1-checksum.txt

Its output should be:

ossec-hids-2.8.1.tar.gz: OK
sha1sum: WARNING: 1 line is improperly formatted

In each case, ignore the WARNING line. The OK line is what confirms that the file is good.

Step 4 — Installing OSSEC

In this step, we will install OSSEC. To begin, first untar it.

tar xf ossec-hids-2.8.1.tar.gz

It will be unpacked into a directory called ossec-hids-2.8.1. Change into that directory.

cd ossec-hids-2.8.1

To fix the /etc/hosts.deny bug, open the host-deny.sh file in the /var/ossec/active-response directory after extracting OSSEC from the downloaded tarball.

nano active-response/host-deny.sh

Towards the end of the file, look for two lines in the code as below that begin with TMP_FILE = , underneath the # Deleting from hosts.deny comment. Edit both lines to remove the spaces on either side of the = sign so the code block looks like this.

# Deleting from hosts.deny
elif [ "x${ACTION}" = "xdelete" ]; then
   lock;
   TMP_FILE=`mktemp /var/ossec/ossec-hosts.XXXXXXXXXX`
   if [ "X${TMP_FILE}" = "X" ]; then
     # Cheap fake tmpfile, but should be harder then no random data
     TMP_FILE="/var/ossec/ossec-hosts.`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -1 `"
   fi

Save and close the file. This concludes the bug fix.

Next, begin the installation.

sudo ./install.sh

Throughout the installation process, you’ll be prompted to provide some input. You’ll first be prompted to select the installation language, which, by default, is English (en). Press ENTER if that’s your preferred language. Otherwise, type in the 2 letters from the list of supported languages first. Then, press ENTER again to start the installation.

Question 1 will ask What kind of installation do you want (server, agent, local, hybrid or help)?. Type local and press ENTER.

For all of the following questions, press ENTER to accept the default, but note that question 3.1 will prompt you for your email address. Enter it, and the installer will use it to find the corresponding SMTP server automatically.

If installation is successful, the last few lines of the post-installation output should read:

- Configuration finished properly.

 - To start OSSEC HIDS:
                /var/ossec/bin/ossec-control start

 - To stop OSSEC HIDS:
                /var/ossec/bin/ossec-control stop

 - The configuration can be viewed or modified at /var/ossec/etc/ossec.conf

. . .

Step 4 — Customizing OSSEC’s Email Settings

Here we are going to verify that the email credentials specified in the previous step and the one that OSSEC auto-configured are correct.

The email settings are in OSSEC’s main configuration file - ossec.conf, which is in the ``/var/ossec/etc` directory. To access and modify any OSSEC file, you first need to switch to the root user.

sudo su

Now that you’re root, change into the directory where OSSEC’s configuration file is.

cd /var/ossec/etc

Then make a backup copy of the configuration file.

cp ossec.conf ossec.conf.00

Open the original file using the nano text editor or your preferred text editor.

nano ossec.conf

The email settings are at the top of the file, which looks like this.

<global>
    <email_notification>yes</email_notification>
    <email_to>sammy@example.com</email_to>
    <smtp_server>mail.example.com.</smtp_server>
    <email_from>sammy@example.com</email_from>
</global>

< email_to> is the email you gave during installation. Alerts will be sent to that email address, and < smtp_server> is the SMTP server auto-discovered by the installation script. You don’t have to change these values.

< email_from> is the email address that OSSEC’s alerts would appear to be coming from. By default, it is created based on OSSEC’s mail user account and the hostname of the server. You should change this to a valid email address to reduce the odds of your emails being tagged as spam by your email provider’s SMTP server. Note that < email_to> and < email_from> can be the same if the receiving SMTP server does not have a strict spam policy.

After modifying the email settings, save and close the file. Then start OSSEC.

/var/ossec/bin/ossec-control start

Check your inbox for an email that says that OSSEC has started. If you receive an email from your OSSEC installation, then you know that future alerts will also reach your inbox. If you don’t, check your spam folder.

Step 6 — Adding Alerts

By default, OSSEC will issue alerts on file modifications and other activities on the server, but it will not alert on new file additions and also will not alert in real-time — only after the scheduled system scan, which is 79200 seconds (or 22 hours) by default. In this section, we will modify OSSEC so that it can alert on file additions and in real-time.

First, open ossec.conf.

nano ossec.conf

Scroll down to the < syscheck> section. Just under the <frequency> tag, add < alert\_new\_files>yes< /alert\_new\_files >.

<syscheck>
    <!-- Frequency that syscheck is executed - default to every 22 hours -->
    <frequency>79200</frequency>
    <alert_new_files>yes</alert_new_files>

While you still have ossec.conf open, take a look at the list of system directories that OSSEC monitors, which is just under the last line you modified. Add report_changes="yes" realtime="yes" to both directory tags.

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>

Aside from the default list of directories that OSSEC has been configured to monitor, you may also add any that you wish to monitor. For example, you may add your home directory. To do that, add this new line right under other directory lines, substituting in your username.

<!-- Directories to check  (perform all possible verifications) -->
<directories report_changes="yes" realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/bin,/sbin</directories>
<directories report_changes="yes" realtime="yes" check_all="yes">/home/<^sammy</directories><^>

Now save and close ossec.conf.

The next file to modify is in the /var/ossec/rules directory, so change to that directory.

cd /var/ossec/rules

The /var/ossec/rules directory contains many XML files, including ossec_rules.xml, which contains OSSEC’s default rule definitions, and local_rules.xml, which is where you can add custom rules.

In ossec_rules.xml, the rule that fires when a file is added to a monitored directory is rule 554. By default, OSSEC does not send out alerts when that rule is triggered, so the task here is to change that behavior. Here’s what rule 554 looks like by default:

<rule id="554" level="0">
<category>ossec</category>
<decoded_as>syscheck_new_entry</decoded_as>
<description>File added to the system.</description>
<group>syscheck,</group>
</rule>

OSSEC does not send out an alert if a rule is set to level 0, so we will copy that rule to local_rules.xml and modify it to trigger an alert. To do that, open local_rules.xml.

nano local_rules.xml

Add the following at the end of the file, before the line with the </group> tag.

<rule id="554" level="7" overwrite="yes">
<category>ossec</category>
<decoded_as>syscheck_new_entry</decoded_as>
<description>File added to the system.</description>
<group>syscheck,</group>
</rule>

</group> <!-- SYSLOG,LOCAL -->


<!-- EOF -->

Save and close the file, then restart OSSEC to apply the changes.

/var/ossec/bin/ossec-control restart

You should now be receiving alerts whenever a file or added, modified, or deleted. Note that OSSEC does not alert on file additions in real-time, only after a full system scan.

Step 6 (Optional) — Stopping IPTables Denied Alerts

In this step, which is optional but highly recommended, we will configure OSSEC to not alert on IPTables denied messages.

At the beginning of this tutorial, we enabled the IPTables firewall. After installing OSSEC, it will alert on rule 1002, which is triggered when IPTables denies an attacker and logs the incidence to syslog. While it’s good to know when an attacker has been foiled, such alerts can number in the hundreds every day and clog up your inbox.

To remove these alerts, we need to customize rule 1002. That rule is found in /var/ossec/rules/syslog_rules.xml, and looks like this:

<rule id="1002" level="2">
    <match>$BAD_WORDS</match>
    <options>alert_by_email</options>
    <description>Unknown problem somewhere in the system.</description>
</rule>

Open syslog_rules.xml for editing.

nano /var/ossec/rules/syslog_rules.xml

Find the BAD_WORDS variable, which is defined at the top of that file and contains a number of keywords. It looks like this:

<var name="BAD_WORDS">core_dumped|failure|error|attack|bad |illegal |denied|refused|unauthorized|fatal|failed|Segmentation Fault|Corrupted</var>

Below the BAD_WORDS definition, copy and paste this new variable, IGNORED_WORD. The variable contains just one keyword.

<var name="BAD_WORDS">core_dumped|failure|error|attack|bad |illegal |denied|refused|unauthorized|fatal|failed|Segmentation Fault|Corrupted</var>
<var name="IGNORED_WORD">denied</var>

Then, we’ll use the new IGNORED_WORD variable in a customized version of rule 1002, which we’ll call rule 100031. The complete rule is shown below. Copy and paste it to the bottom of the file, before the line with the group tag.

<rule id="100031" level="0">
     <if_sid>1002</if_sid>
     <match>$IGNORED_WORD</match>
     <description>Ignored IPTables deny messages.</description>
</rule>

</group>


<!-- EOF -->

Save and close the file. With the variable and customized rule in place, restart OSSEC.

/var/ossec/bin/ossec-control restart

With that, OSSEC should stop sending alerts of IPTables denied messages.

Conclusion

That’s all it takes to install and configure a local OSSEC on a Debian 8 server. There is a lot of further customization available, which you can explore in the projects official documentation.