Flow Trackers for large scale flow analytics
Can your network analytics tool answer this query interactively ?
Show me a list of top 100 flows that uploaded data to servers outside North America last week
or even the basic query
Show me the top 100 large volume flows today
This kind of query is surprisingly difficult for large datasets over long timeframes. You use Flow Trackers in Trisul for exactly this purpose. In this blog, lets see how you can create a flow tracker of your own.
How this works?
Essentially Trisul is a streaming analytics tool. It maintains a table, also known as a sketch, of top flows matching criteria your specify. This “sketch” is then flushed to disk every 5 minutes. Out of the box, Trisul has 6 flow trackers. One each for Total Volume, Upload, Download, Duration, TCP Payload Upload, and TCP Payload Download. Therefore every 5 minutes, 6 topper sketches are flushed to disk, each containing 100 top flows in that group. So when you run these long term range based queries, the sketches are used rather than the raw flows.
Example : Track top flows to servers outside America (US/CA/MX)
Let us create a flow tracker of our own. One that tracks all flows that have been marked as involving countries other than US, CA, MX.
Step 1 : First tag all flows with country codes
We first need to mark the flows with a country code. For that we use an Automatic Flow Tagger . If you’ve used them you would know that tags are nothing but text labels added to flow records based on observed metrics.
- Go to Tools > Flow Taggers > Manage
- Create a new Automatic Flow Tagger , select the Country counter group, and enter * to match all country codes.
Step 2 : Create a flow tracker based on the country tags
We now create a new flow tracker for flows that do not involve US, MX, CA country codes.
- Go to Tools > Flow Tracker > Manage
- Create a New Flow Tracker and fill out as shown below
- Note that we used negated keys to track flows to countries other than US/UK/MX
Thats all it takes. You only have to restart Trisul to start tracking these flows.
Other types
You can use hostnames, IP ranges (even really large ranges such as 100.0.0.0~200.0.0.0), Port ranges (Port-1000~Port-2000) also for flow trackers. All the TCP/IP tuples can be used as well as the tags in the above example. Trisul detects the tuple automatically from what you’ve entered.
Viewing
You may now view flow tracker activity from Tools > Flow Tracker or from Retro Tools > Flow Tracker
Enjoy this powerful functionality and we want to hear from you.
Free Download Trisul 4.0
Trisul Network Analytics 4.0 is now available – sign up and download. Enjoy!