Unix socket madness with Trisul and IDS alerts

Unix socket madness with Trisul and IDS alerts

One of the things Trisul can do is to merge rich traffic analytics data with traditional IDS alerts from systems like Snort and Suricata. Up until Trisul 5.5 you could connect Trisul into a single Unix Socket to which Snort or Barynard2 would write alerts. Once alerts were inside Trisul they would be merged with other types of NSM data in real time. You could then ask queries like “Show me list of flows that produced a HIGH priority alert”

Why Unix Sockets?

Unix sockets are old school, yet they are used in a suprisingly large number of systems even today for IPC (Interprocess communications). There are basically two ways you can read alerts from a logging system like Suricata.

  1. Traditional Log Tailing Observe an output log file and process any new alerts.
  2. Network messaging Just read messages off a network feed. This can be unix sockets, TCP/UDP sockets or higher abstractions like message queues.

The advantages of Unix Sockets we see are

  1. Secure – purely local, unlike TCP/UDP sockets
  2. Message based – Unix socket that use SOCK_DGRAM preserve message boundaries. You can simply read the messages as sent
  3. chmod – can be secured using traditional linux chmod style permissions. For network sockets you need to use ACLs or other mechanisms like CURVE or TLS
  4. No waldo – A most loved feature of IDS log tail is the so called waldo file. You need to know where to position yourself on the log file when you restart the system.

So there! We like Unix Sockets and try to use it even for shiny new output formats like Suricata’s EVE JSON output format.

Let us now see some code that shows you how you can use Trisul’s new LuaJIT API to interface with Unix Sockets.

Trisul 6.0 Platform

Trisul 6.0 is now positioned as a platform that leverages the power of LuaJIT to provide plug-in points into our processing pipeline. Now you can write an ‘inputfilter’ LUA script to listen to ANY type of feed that would result in an arbitrary alert.

New scripts on github

We just released four scripts on Github Trisul-Scripts

  1. suricata_eve.lua (github) – EVE – Extensible Event Format is a modern output format that uses JSON. If you havent used Suricata’s EVE output format you really should check it out.
  2. suricata_eve_unixsocket.lua – Using a bit of LuaJIT FFI read EVE from a Unix Socket
  3. snort_unixsocket.lua – Traditional “-A unsock” output format from snort
  4. barnyard2_unixsocket.lua – Unified2 binary format from Barnyard2.

The input filter framework

Input filters are LUA scripts you write that can drive trisul. You can write custom PCAP readers, alert monitors, or custom flow-like records and use that as the input. In the illustration below you can see how multiple scripts can listen on different inputs. The Trisul platform will take care of the orchestration, threading, preventing starvation etc.

Using Unix Sockets. LuaJIT FFI to the rescue

Sometimes LuaJIT feels like cheating because you can drop down to standard C at anytime and call linux APIs and use C structures. If you are already familiar with LuaJIT FFI all of this will be familiar to you. Others can read the LUA code to understand how this C-LuaJIT interface works. Once you have setup the LuaJIT “cdefs”, read the JSON message from the socket, all you have to do is to map the EVE message to a Trisul alert format.

Lets take a look at how we map the EVE format.

From eve_unixsocket.lua

The real action happens in the step_alert function. You need to map the EVE types to a LUA table that Trisul understands as per the inputfilter API

-- setup unixsocket using FFI

  -- read JSON EVE

  -- filter out non 'alert'  types

  -- map to input filter

  local ret =  {

      AlertGroupGUID='{9AFD8C08-07EB-47E0-BF05-28B4A7AE8DC9}',     -- Trisul alert group = External IDS 
      TimestampSecs = tv_sec,                                      -- Epoch based time stamps
      TimestampUsecs = tv_usec,
      SigIDKey = p.alert["signature_id"],                          -- SigIDKey is mandatory 
      SigIDLabel = p.alert["signature"],                           -- User Label for the above SigIDKey 
      SourceIP = p["src_ip"],                                      -- IP and Port pretty direct mappings
      SourcePort = p["src_port"],
      DestIP = p["dest_ip"],
      DestPort = p["dest_port"],
      Protocol = protocol_num(p["proto"]),                         -- convert TCP to 6 
      SigRev = p.alert["rev"],
      Priority = p.alert["severity"],
      ClassificationKey = p.alert["category"],
      AlertStatus=p.alert["action"],                                -- allowed/blocked like ALARM/CLEAR
      AlertDetails=p.alert["signature"]                             -- why waste a text field 'AlertDetails'?

Most of the fields are straight up one-to-one mapping. I found the AlertStatus and EVE alert.action to be quite interesting. Apparently Suricata uses that field in IPS mode to indicate whether the event resulted in a BLOCK or ALLOW. By mapping it into Trisul AlertStatus you get a neat integration that result in something like below.

Notice the ‘allowed’ tag.

Explore the API

Over the next few days we will be writing about the other new LuaJIT APIs in Trisul. One such API is the File Extraction API.

Install Trisul now, no sign up needed

Installing Trisul Network Analytics 6.0 is painless. There are no sign ups. It is as simple as apt-get install or yum install Head over to the Downloads area to get started. The APIs are included as part of the base package.

Until next time.

Free Download Trisul 6.0 !