Messages OverviewTop

A TRP message corresponds to a specific type of data you want to retrieve from Trisul. Every message sent will elicit a corresponding response or an ErrorResponse.

Protocol messages

Common messages

These are supporting messages the appear in one or more responses.

HelloTop

A handshake to get the identity of the Trisul server and to set the identity of the TRP client. It is not required that you start off a TRP Session with a HelloRequest, use this only to get server identity and set your own identity in server logs.

HelloRequest

message HelloRequest{
  required string   station_id=1;
}       

HelloResponse

message HelloResponse{
  required string       trisul_id=1;
  required string       trisul_description=2;
  required string       connection_id=3;
  required string       version_string=4;
  required Timestamp    connection_start_time=5;
  required Timestamp    connection_up_time=6;
  required AuthLevel    current_auth_level=7;   
}

CounterGroupInfoTop

Use this to retrieve information about Counter Groups such as bucket sizes, time window of available data, names, and guid. The most common usage of this message is to retrieve the bucket size and the available time window.

CounterGroupInfoRequest

message CounterGroupInfoRequest {
    optional int64         context=1[default=0];
    optional string        counter_group=2;
}

Usage

• Skip the counter_group field if you want to retrieve details for all counter groups.

CounterGroupInfoResponse

message CounterGroupInfoResponse{
    optional int64                  context=1;
    repeated CounterGroupDetails    group_details=2;
}

Samples

See cginfo on the Trisul Scripts github page.

CounterItemTop

Retrieve traffic statistics about a counter item. A counter item is identified by a counter group id (a GUID) and a key. The typical use of this message is to get time-series usage or total usage of any metered item.

CounterItemRequest

message CounterItemRequest{
    optional int64          context=1 [default=0];
    required string         counter_group=2;
    optional int64          meter=3;
    required string         key=4;
    required TimeInterval   time_interval=5;
    optional int64          volumes_only=6 [default=0];
}

CounterItemResponse

message CounterItemResponse{
    required KeyStats       stats=1;
}

Sample code

See getvolume and hourlystats on the Trisul Scripts github page.

CounterGroupTop

Retrieve toppers for a specific meter in counter group in a specified time interval.

Some examples :

1. Retrieve top hosts by total volume yesterday
2. Retrieve top applications by connections between 8AM and 6PM today

CounterGroupRequest

message CounterGroupRequest{
    optional int64          context=1 [default=0];
    required string         counter_group=2;
    optional int64          meter=3 [default=0];
    optional int64          maxitems=4 [default=10];
    optional TimeInterval   time_interval=5;
    optional Timestamp      time_instant=6;
    optional int64          flags=7;
}

CounterGroupResponse

Returns the list of keys that represent the top items. If you want to retrieve statistics for these keys you can send out individual CounterItemRequests for each of them. The field metric contains the total volume for each key.

message  CounterGroupResponse {
    required int64          context=1;
    required string         counter_group=2;
    required int64          meter=3;
    repeated KeyDetails     keys=6;
}

BulkCounterItemTop

BulkCounterItemRequest

Get traffic stats of multiple counter items at once. Use this method if you want to retrieve stats for a number of items for the same time interval and meter.

message BulkCounterItemRequest{
    optional int64          context=1[default=0];
    required string         counter_group=2;
    required int64          meter=3;
    required TimeInterval   time_interval=4;
    repeated string         keys=5;
}

BulkCounterItemResponse

    message BulkCounterItemResponse{
    repeated KeyStats       stats=1;
}

FilteredDatagramTop

Retrieve raw packets from Trisul. You can retrieve them in libpcap format ready to be opened in Wireshark or Unsniff.

Usage

The FilteredDatagram message allows you to :

1. Retrieve packets for a given flow (TCP/UDP session)
2. Retrieve relevant packets for an alert
3. Retrieve packets for a resource (such as a HTTP URI)
4. Arbitrary filter string

The filter string

Allows you to use a powerful filter to get the packets you want. The filter string in an expression in Trisul Filter Format. You can retrieve packets matching exotic metering criteria like “Get me all non-HTTP traffic from China or Ukraine”.

FilteredDatagramRequest

message FilteredDatagramRequest{
    optional int64          max_packets=1[default=0];                
    optional int64          max_bytes=2[default=0];                   
    optional CompressionType compress_type=3[default=UNCOMPRESSED];              
 
    // by trisul filter format expr
    message ByFilterExpr {
        required TimeInterval   time_interval=1;                   
        required string         filter_expression=2;
    }
    optional ByFilterExpr   filter_expression=4;
 
    // by session
    message BySession   {
      optional string         session_group=1[default="{99A78737-4B41-4387-8F31-8077DB917336}"];
      required SessionID      session_id=2;
    }
    optional BySession          session=5;
 
 
    // by alert
    message ByAlert     {
      optional string         alert_group=1[default="{9AFD8C08-07EB-47E0-BF05-28B4A7AE8DC9}"];
      required AlertID        alert_id=2;
    }
    optional ByAlert            alert=6;
 
    // by resource
    message ByResource  {
      required string         resource_group=1;
      required ResourceID     resource_id=2;
    }
    optional ByResource         resource=7;

}

Usage notes

• If both max_packets and max_bytes are specified, Trisul will stop at whatever limit is reached first

FileredDatagramResponse

message FilteredDatagramResponse{
    required PcapFormat         format=1;                                 
    required CompressionType    compress_type=2;                   
    required TimeInterval       time_interval=3;                
    required int64               num_datagrams=4;               
    required int64               num_bytes=5;      
    required string              sha1=6;
    required bytes               contents=7;

}

The contents field is a string that contains the packet data.

• If compression type is GZIP, you need to unzip the contents string
• Just save the uncompressed contents string into a file
• You can then open the file using Wireshark or Unsniff

ControlledContextTop

For Raw Packets mode only

Create a new drill down retro analysis context by controlling for a key over a time interval. You would typically :

• Send a ControlledContextRequest over a time interval (say ‘Hosts involved in ICMP’)
• Use the returned context id in other TRP commands
• Call ReleaseResource after you are done with the context.

ControlledContextRequest

message ControlledContextRequest{
    required TimeInterval       time_interval=1;          
    required string             filter_expression=2;       
} 

ControlledContextResponse

The context parameter in the response is a number that uniquely identifies the context on the server side. You can use this context value in any subsequent TRP command.

message ControlledContextResponse{
    required int64              context=1;
    optional string             context_db=2;
    required TimeInterval       time_interval =3;          
} 

SearchKeysTop

Find keys matching a given pattern. You can use this to retrieve all hosts which have ‘youtube’ in their DNS names for example.

SearchkeysRequest

message SearchKeysRequest{
    optional int64          context=1[default=0];                    
    required string         counter_group=2;        
    required string         pattern=3;        
    required int64          maxitems=4;
} 

SearchKeysResponse

message SearchKeysResponse{
    optional int64          context=1;                    
    required string         counter_group=2;        
    repeated KeyDetails     found_keys=3;
} 

SessionItemTop

Retrieve information about a single flow.

SessionItemRequest

message SessionItemRequest{
    optional int64          context=1[default=0];
    optional string         session_group=2[default="{99A78737-4B41-4387-8F31-8077DB917336}"];
    repeated string         session_keys=3;
    repeated SessionID      session_ids=4;
}

Each flow is uniquely identified by a

1. Session ID : For completed or long running flows (OR)
2. Session Key : For brand new sessions (less than 1 minute old) in progress

Note either the session_key or session_id must be specified.

SessionItemResponse

message SessionItemResponse{
    optional int64          context=1[default=0];
    required string         session_group=2;

    message Item {
        optional string         session_key=1;
        optional SessionID      session_id=2;
        optional string         user_label=3;
        required TimeInterval   time_interval=4;
        required int64          state=5;
        required int64          az_bytes=6;
        required int64          za_bytes=7;
        required KeyDetails     key1A=8;
        required KeyDetails     key2A=9;
        required KeyDetails     key1Z=10;
        required KeyDetails     key2Z=11;
    }

    repeated Item           items=3;
}

TopperSnapshotTop

Similar to CounterGroupRequest but retrieves toppers at various instants in the given time interval using a single command.

TopperSnapshotRequest

message TopperSnapshotRequest{
    optional int64          context=1;
    required string         counter_group=2;
    required int64          meter=3;
    required TimeInterval   Time=4;
    required int64          maxitems=5;
}

TopperSnapshotResponse

message TopperSnapshotResponse{
    optional int64          context=1;
    required string         counter_group=2;
    required int64          meter=3;
    required Timestamp      time=4;
    required int64          window_secs=5;
    required string         keys=6;
    required string         labels=7;
    
}

UpdateKeysTop

Used to assign a friendly name and/or description to a key.

UpdateKeysRequest

message UpdateKeyRequest{
    optional int64          context=1;
    required string         counter_group=2;
    required string         key=4;
    required string         label=5;
    optional string         description=6;
}

The response to this message is

1. OKResponse : If the update was successful
2. ErrorResponse : If an error occurred

KeySessionActivityTop

Retrieve flows matching a given key (IP or Port) over a time interval.

KeySessionActivityrequest

message KeySessionActivityRequest{
    optional int64          context=1[default=0];
    optional string         session_group=2[default="{99A78737-4B41-4387-8F31-8077DB917336}"];
    required string         key=3;
    optional string         key2=4;
    optional int64          maxitems=5[default=100];
    optional int64          volume_filter=6[default=0];
    optional int64          duration_filter=7[default=0];
    required TimeInterval   time_interval=8;
}

KeySessionActivityResponse

message KeySessionActivityResponse{
    optional int64          context=1;
    required string         session_group=2;
    repeated SessionID      sessions=3;
}

Typically, after getting the SessionIDs you will fire individual SessionItemRequests for each item in order to retrieve the details.

SessionTrackerTop

Retrieve session trackers. Session trackers are tools which enable Trisul to automatically track and store top flows matching a given criteria.

SessionTrackerRequest

message SessionTrackerRequest {
    optional int64          context=1;
    optional string         session_group=2;
    required int64          tracker_id=3 [default=1];
    optional int64          maxitems=4 [default=100];
    required TimeInterval   time_interval=5;
}

SessionTrackerResponse

message SessionTrackerResponse{
    optional int64          context=1;
    required string         session_group=2;
    repeated SessionID      sessions=3;
}

Typically, after getting the SessionIDs you will fire individual SessionItemRequests for each item in order to retrieve the details.

SessionGroupTop

Retrieve current top flows.

SessionGroupRequest

message SessionGroupRequest {
    optional int64          context=1;
    optional string         session_group=2;
    optional int64          tracker_id=3;
    optional string         key_filter=4;
    optional int64          maxitems=5 [default=100];
}

SessionGroupResponse

message SessionGroupResponse {
    optional int64          context=1;
    required string         session_group=2;
    repeated string         session_keys=3;
}

Typically, after getting the SessionIDs you will fire individual SessionItemRequests for each SessionKey in order to retrieve the flow details.

ServerStatsTop

Statistics about the Trisul server.

ServerStatsRequest

message ServerStatsRequest{
     required int64         param=1;
}

ServerStatsResponse

message ServerStatsResponse{
    required string             instance_name=1;
    required int64              connections=2;
    required int64          uptime_seconds=3;
    required double         cpu_usage_percent_trisul=4;
    required double         cpu_usage_percent_total=5;
    required double         mem_usage_trisul=6;
    required double         mem_usage_total=7;
    required double         mem_total=8;
    required int64          size_total=9;
    required double         drop_percent_cap=11;
    required double         drop_percent_trisul=12;
    required TimeInterval   time_interval=13;
}

AlertItemTop

Used to retrieve details about a list of alert ids.

AlertItemRequest

Use this request to get details about a list of alerts_ids.

message AlertItemRequest{
    optional int64          context=1[default=0];            
    required string         alert_group=2;
    repeated AlertID        alert_ids=3;
}

AlertItemResponse

Details about each alert in the request. Invalid IDs in the request will not be present in the response.

message AlertItemResponse{
    optional int64          context=1;
    required string         alert_group=2;
    message Item {
        optional int64          sensor_id=1;
        required Timestamp      time=2;
        required AlertID                alert_id=3;
        optional string         source_ip=4;
        optional string         source_port=5;
        optional string         destination_ip=6;
        optional string         destination_port=7;
        required string         sigid=8;
        required string         classification=9;
        required string         priority=10;
        required Timestamp      dispatch_time=11;
        required string         aux_message1=12;
        required string         aux_message2=13;
    }
    repeated Item           items=3;
}

AlertGroupTop

The AlertGroup Request and Response methods are used to query the alerts known to Trisul.

Trisul supports four different types of alerts

1. Threshold Crossing Alerts
2. Flow Tracker Alerts
3. Malware Blacklist Alerts (requires the Badfellas plugin)
4. Snort or Suricata Alerts (Trisul accepts these alerts from Unix socket)

Each of these categories are called “Alert Groups” and each of them is uniquely identified by a GUID. You need to specify an appropriate GUID for the alert_group parameter, see here for the GUID definitions.

AlertGroupQueryRequest

Retrieve alerts matching a query.

1. If you specify one or more of the optional parameters they will be ANDed.
2. If you dont specify any parameter all alerts will be retrieved

message AlertGroupRequest {
    optional int64          context=1[default=0];
    required string         alert_group=2;
    required TimeInterval   time_interval=3;
    optional int64          maxitems=5 [default=10];
    optional string         source_ip=6;
    optional string         source_port=7;
    optional string         destination_ip=8;
    optional string         destination_port=9;
    optional string         sigid=10;
    optional string         classification=11;
    optional string         priority=12;
    optional string         aux_message1=13;
    optional string         aux_message2=14;
}

AlertGroupResponse

Matching alert IDs. You will usually follow this up with an //AlertItemRequest// to get full details about each alert in the list.

message AlertGroupResponse {
    optional int64          context=1;
    required string         alert_group=2;
    repeated AlertID        alerts=3;
}

ResourceGroupTop

Resources are additional meta data collected by Trisul about the traffic it sees. Currently Trisul logs the following resources

1. DNS names
2. HTTP URLs
3. SSL Certificates

You use the ResourceGroupRequest to find the resources you need- followed by one or more ResourceItemRequests to retrieve details of those resources.

ResourceGroupRequest

message ResourceGroupRequest {
    optional int64          context=1[default=0];            
    required string         resource_group=2;
    required TimeInterval   time_interval=3;
    optional int64          maxitems=4 [default=10];
    optional string         source_ip=5;
    optional string         source_port=6;
    optional string         destination_ip=7;
    optional string         destination_port=8;
    optional string         uri_pattern=9;
    optional string         userlabel_pattern=10;
}

ResourceGroupResponse

Retrieves a list of ResourceIDs matching the request query.

Typically after this response you need to issue one or more ResourceItemRequest requests to query the attributes of each resource.

message ResourceGroupResponse {
    optional int64          context=1;
    required string         resource_group=2;
    repeated ResourceID     resources=3;
}

ResourceItemTop

Resource Items represent a single resource (such as HTTP URL or DNS name).

Use this request to retrieve details of each resource based on the ID.

ResourceItemRequest

message ResourceItemRequest{
    optional int64          context=1[default=0];            
    required string         resource_group=2;
    repeated ResourceID     resource_ids=3;
}

ResourceItemResponse

message ResourceItemResponse{
    optional int64          context=1;            
    required string         resource_group=2;
    message Item {
        required Timestamp      time=1;
        required ResourceID     resource_id=2;
        optional string         source_ip=3;
        optional string         source_port=4;
        optional string         destination_ip=5;
        optional string         destination_port=6;
        optional string         uri=7;
        optional string         userlabel=8;
    }
    repeated Item           items=3;
}

GrepTop

Search for text or binary patterns in reassembled TCP streams.

GrepRequest

////////////////////////////////////
// GrepRequest
message GrepRequest {
    optional int64          context=1[default=0];            
    optional string         session_group=2[default="{99A78737-4B41-4387-8F31-8077DB917336}"];
    required TimeInterval   time_interval=3;
    optional int64          maxitems=4 [default=50];
    required string         pattern=5;
}

All TCP streams are reassembled and matched with the sequence of bytes specified in // pattern //. Note that pattern is not a Regex, it must be an exact matching string.

GrepResponse

message GrepResponse  {
    optional int64          context=1;
    optional string         session_group=2[default="{99A78737-4B41-4387-8F31-8077DB917336}"];
    repeated SessionID      sessions=3;
}

Normally you would follow up a GrepResponse with a SessionItemRequest to get more information about the returned flows.

KeyLookupTop

Retrieve user labels for keys. Typical uses would be to get hostnames, alert signature names, app names, and any other names that apply to keys within a counter group.

KeyLookup – mutliple keys from a single CG

message KeyLookupRequest {
    optional int64          context=1[default=0];            
    required string         counter_group=2;
    repeated string         keys=3;
}
 

KeyLookup

message KeyLookupResponse {
    optional int64          context=1;            
    required string         counter_group=2;
    repeated KeyDetails     key_details=3;
}

Common MessagesTop

OKResponseTop

You can expect OKResponse back when some requests do not have a specific response message.

message OKResponse{
   required int64       original_command=1;
   optional string      message=2;
}

ErrorResponseTop

Any request message can get back an error response.

message ErrorResponse{
   required int64       original_command=1;
   required int64       error_code=2;
   required string      error_message=3;
}

KeyDetailsTop

Represents a key.

message KeyDetails {
    required string         key=1;
    optional string         label=2;
    optional string         description=3;
    optional int64          metric=4;
}

KeyStatsTop

Key statistics across multiple meters.

message KeyStats {
    optional int64          context=1[default=0];
    required string         counter_group=2;
    required string         key=3;
    repeated MeterValues        meters=4;
}

TimestampTop

A time instant.

message Timestamp{
  required int64            tv_sec=1;
  required int64            tv_usec=2 [default=0];
}

TimeIntervalTop

A time interval, two time instants.

message TimeInterval  {
    required Timestamp      from=1;
    required Timestamp      to=2;
}

Usage

The long way to create this is

# from and to are two "Time" objects 
tmint = TRP::Message::TimeInterval.new( 
    :from => TRP::Message::Timestamp.new(:tv_sec => from.tv_sec),
    :to => TRP::Message::Timestamp.new(:tv_sec => to.tv_sec)
)

The TrisulRP Ruby Gem has a shortcut called mk_time_interval

tmint = mk_time_interval( [from, to] )

CounterGroupDetailsTop

Details about a counter group.

message CounterGroupDetails {
    required string         guid=1;
    required string         name=2;
    optional int64      bucket_size=3;
    optional TimeInterval   time_interval=4;
}

StatsTupleTop

A timestamp + sample value tuple.

message StatsTuple {
    required Timestamp      ts=1; 
    required int64          val=2; 
}

MeterValuesTop

A time-series data for a single meter. A meter is a statistical data point such as “Total Bytes Received” “Total Connections” etc. Each counter group hosts a number of such meters. You can get a list of these meters from the web interface via (Customize → Counters → Edit Topper Policies )

message MeterValues {
    required int32          meter=1;
    repeated StatsTuple     values=2;
}

SessionIDTop

Uniquely identifies a session over all time.

message SessionID {
    required int64          slice_id=1;
    required int64          session_id=2;
}

Resource IDTop

Uniquely identifies a resource over all time.

message ResourceID {
    required int64          slice_id=1;
    required int64          resource_id=2;
}