Question

How to import syslog into Suricata and Elastic Stack SIEM

Hello,

I followed the tutorial (https://www.digitalocean.com/community/tutorials/how-to-build-a-siem-with-suricata-and-elastic-stack-on-ubuntu-20-04). Thank you by the way for this very complete tutorial.

However, I can not find how to import log files into the SIEM coming from an external source (I have a Windows machine sending logs with nxlog, collected by rsyslog on the Ubuntu server).

I have tried configuring filebeat.yml (enabled the filebeat.inputs, and set the paths to /var/log/*/*.log) but this does not send any of the log files in this folder to the SIEM.

Could you please indicate to me what I am doing wrong? Is the import of log files in Suricata instead?

I would very much appreciate any help, I am a bit lost on this topic.

Thank you,

PS : please excuse me if I chose the wrong topic.


Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Bobby Iliev
Site Moderator
Site Moderator badge
July 10, 2023

Hi there,

There are a few things that I could suggest starting with in order to troubleshoot the problem further. The first step in resolving this issue is to make sure that the logs are actually being received by rsyslog. You can confirm this by checking the log files on your Ubuntu server, i.e., files under /var/log/.

Once you have confirmed that the logs are reaching your Ubuntu server, you can focus on getting Filebeat to ingest these logs and forward them to Elasticsearch. Here are the steps you should follow:

  1. Configure Filebeat to ingest the logs: The filebeat.inputs section of your filebeat.yml file should look something like this:

    filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /var/log/*.log
    

    This configuration tells Filebeat to read all log files directly under the /var/log/ directory. If your logs are in a subdirectory, you’ll need to adjust the path accordingly.

  2. Ensure Filebeat is outputting to Elasticsearch: Make sure that Filebeat is configured to send its output to your Elasticsearch instance. The output.elasticsearch section of your filebeat.yml file should look something like this:

    output.elasticsearch:
      hosts: ["<your-elasticsearch-host>:9200"]
    

    Replace <your-elasticsearch-host> with the hostname or IP address of your Elasticsearch instance.

  3. Restart Filebeat: After making any changes to filebeat.yml, you’ll need to restart Filebeat to apply them. You can do this with the following command:

    sudo systemctl restart filebeat
    
  4. Check Filebeat’s logs: If Filebeat still isn’t sending your logs to Elasticsearch, you should check Filebeat’s own logs for any error messages. You can do this with the following command:

    sudo journalctl -u filebeat
    

Best,

Bobby

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

card icon
Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Sign up
card icon
Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We’d like to help.

Learn more
card icon
Become a contributor

You get paid; we donate to tech nonprofits.

Learn more
Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow – whether you’re running one virtual machine or ten thousand.

Learn more ->
DigitalOcean Cloud Control Panel
Get started for free

Enter your email to get $200 in credit for your first 60 days with DigitalOcean.

New accounts only. By submitting your email you agree to our Privacy Policy.

© 2023 DigitalOcean, LLC.