David76
By:
David76

How to config Logstash to parse external logs and send them to ElasticSearch?

March 6, 2015 1.4k views
Logging Security

I need ElasticSearch+Logstash to collect all my servers logins+logouts.

I use evtsys in Windows Servers and rsyslog in Unix to send the login logs to my Red Hat server (5000 port).

If I captured the 5000 port, I can see the logins:

tcpdump -s0 -A -i eno1 port 5000

Mar  5 15:30:02 SERVER1 sshd[19659]: Accepted password for root from 10.XXX.XXX.XXX port 59784 ssh2
Mar  3 08:58:57 SERVER2 Security-Auditing: 4624: AUDIT_SUCCESS Se inici.. sesi..n correctamente en una cuenta. 
Mar  5 15:28:11 SERVER1 sshd[16963]: pam_unix(sshd:session): session closed for user root
...

I have installed Logstash 1.4.2. and ElasticSearch 1.4.4 (last versions).

I can't make the connection between Logstash and ElasticSearch (0 assigned shards, 0 indexes created...).

I have tried with this /etc/logstash/conf.d/logstash.conf:

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

Suggestions? Thanks in advance!

Be the first one to answer this question.