New in Trisul 3.6 – Long Tail Hosts monitoring
We’ve got a new feature in Trisul 3.6 that is going to start paying off from day one. Its called Long Tail Web Hosts
Here is how it works.
Alexa Top 1 M and Quantcast Top 1M
Trisul Badfellas already extracts intel from roughly a dozen top quality free feeds . We’ve further enhanced it to leverage the following feeds
Based on these feeds we calculate a consolidated rank list. The Badfellas plugin monitors each HTTP Host (from HTTP Host Header and TLS Subject) and adds in two new counter groups called “Long Fat Tail” and “Long Thin Tail”
Long Fat and Long Thin counter groups
- Long Fat : Sites that are ranked but not in the top 10%
- Long Thin : Sites that arent ranked at all
In almost all our enterprise Trisul deployments, we observed that the three existing host based counter groups “Hosts” and “External Hosts” and “HTTP Hosts” were not giving us sufficient visibility into the long tail or rarely seen hosts. The usual suspects like Google,Facebook,Twitter,WindowsUpdate dominated all of the topper lists down to the 50th or 60th ranks. Thats no good. With the new Long Tail counter groups we have excellent visibility into the fringe traffic which is so critical to detect unauthorized / malware activity.
Why Long Fat ?
At first we only had a single counter group for hosts that arent ranked at all (Long Thin). We quickly found the top 1 million sites lists some obscure ones which might be worth looking into. What we have now is a nice two step view into long-tail activity.
Using the feature
Just install the latest trisul-badfellas package. There is no configuration to be done. The new BadFellas “long tail” counter groups are just like any other counter group. You can pull packets, tag flows, cross drill, generate reports.
For example : Select Retro → Counters → Long Fat Tail Hosts to show following
Cool things you can do for Network Security Monitoring
Tag flows from long tail hosts
Allows you to search for all long tail flows, download PCAPs, and investigate in detail later.
Use the Flow Tagger to create automatic tags for all flows involving long tail hosts.
PCAP storage
Are you budget constrained on very high speed links ? You can make a tradeoff. Dramatically reduce PCAP storage and RAID requirements by skipping traffic NOT in either of the long tail hosts Are you sure you want to store encrypted Gmail, Twitter, Facebook, Dropbox ? Do you know what to do with that ?
This will only store long thin tail PCAPs
Copy<DefaultMode>IGNORE</DefaultMode> <RuleChain> <Rule mode="FULL">{BF2C028A-CDDE-4074-8AF5-A007582D820E}=*</Rule> ...
Adding your own hosts
Every deployment has trusted traffic that is exchanged with servers not ranked in Alexa or Quantcast. You can create your own rankings that will be merged with the others by the following methods.
Create a file named longtail-local.csv in /usr/local/share/trisul/plugins
Add in your hosts with a rank ( comments ‘#’ and blank lines are ignored)
# My top ranked hosts
# List hosts that are important to you but not ranked
1,myblog.info
2,unleash.com
3,trisul.org
..
..Updates
Trisul updates the lists automatically, default is once per day. You can change it to once per-week by following instructions in the Badfellas plugin documentation.
Enjoy and spread the word !
Trisul Dev Team