multiple customers single dashboard

Heya,

It does sound possible for sure!

Let’s give a certain review of the situation and how to approach it:

• Log Centralization:

  • First, ensure that the Suricata logs from each server (or customer) are centralized in a single Elasticsearch instance. This can be achieved using tools like Filebeat, Logstash, or Suricata’s built-in logging capabilities.
  • Filebeat is commonly used as an agent to ship Suricata logs to Elasticsearch.

• Tag Logs with Customer-Specific Information: To filter logs by customer, you’ll need to tag the logs based on the server or customer. This is done by adding metadata to each log entry, either using Filebeat or Logstash.

  • Filebeat: You can add a custom field in the Filebeat configuration file (filebeat.yml), which will tag logs based on the customer’s identity.

filebeat.inputs:
- type: log
  paths:
    - /var/log/suricata/eve.json
  fields:
    customer_id: "customer_1"

• You can configure this for each server to ensure logs are tagged with the respective customer ID.

• Logstash: If you are using Logstash, you can use filters to add metadata like customer IDs when processing logs:

filter {
  if [host] == "customer1_server_ip" {
    mutate { add_field => { "customer_id" => "customer_1" } }
  }
}

• Index Design:

  • Single Index: You can store all logs in a single index and use fields like customer_id for filtering. This is simple and allows you to scale easily, but performance may degrade as logs grow.
  • Multiple Indices: Alternatively, create separate indices for each customer. For example, you could have suricata_customer1-*, suricata_customer2-*, etc. This is more organized but could be harder to maintain if you have many customers.

• Index Patterns in Kibana: Once the logs are ingested into Elasticsearch, you can create index patterns in Kibana to filter logs by customer.

  • Go to Kibana > Management > Index Patterns.
  • Create an index pattern that matches your logs, such as suricata-*.
  • You can then filter by customer-specific fields (like customer_id) in the Discover tab, which allows you to search, analyze, and visualize logs per customer.

• Dashboards and Visualizations:

  • In Kibana or another visualization tool, you can create saved searches, filters, and visualizations based on the customer_id field.
  • You can then build a dashboard that includes various visualizations (graphs, charts, etc.), and each visualization can filter logs by customer.
  • Kibana supports the creation of query filters or dropdown filters in dashboards so you can dynamically select and view logs per customer.

On another note, consider using an automation tool like Ansible to deploy and configure Suricata, Filebeat, or Logstash on each server with the correct customer-specific settings to streamline the setup.

With this setup, your dashboard will be able to display a comprehensive view of all logs while still offering the flexibility to filter and analyze data by customer.

Regards