When you install Trisul Network Analytics , you get a free command line tool called trisul_ixmgtool

This tool has a unique capability to squish PCAP files that is very handy to create fat pcap files useful for testing. This article explains how this free tool works.

While testing NSM¹⁾ platforms we look for FAT pcap files because it stresses the memory and performance of algorithms. Given a 10GB thin PCAP file with just 1 flow, and a 1GB fat PCAP file with 100K flows - you should prefer the FAT file for testing.

FAT PCAP files can be hard to obtain. You might get them from large corporate border networks for private use, but in general it is quite hard to come across these.

With the trisul_ixmgtool you can merge multiple thin PCAPs into a single fat PCAP file.

Mergecap is a command line utility from the wireshark project. It also combines multiple thin PCAP files into a single fat PCAP file. But it preserves the timestamps, hence works to fatten the output PCAP only if there is significant overlap in the time windows.

trisul_ixmgtool when run with the squish option , aligns the timestamps of the files to the lowest timestamp and then processes the merge. The following diagram illustrates the difference between mergecap and ixmgtool.

You can think of ixmgtool as combining the following three operations

1. Find the lowest timestamp from all the pcap files, and compute the deltas for each file
2. Run editcap -t delta to transform the timestamps of each file
3. Run mergecap on the transformed pcap files

To get the free ixmgtool install Trisul Probe , you will find the trisul_ixmgtool in /usr/local/bin

Usage

unpl@unpl:~$ trisul_ixmgtool 
Usage : ixmgtool [-squish|-squish10]  -r home-dir f1 f2 f3 f4 f5 ..  -out outfile 

Options

• -squish : align the timestamps to the lowest found and merge
• -squish10 : fatten by 10 TIMES by taking each TCP flow and making 10 extra duplicate flows by changing the source IP address 10 different IPs in the 10.0.0.x range

If you run without the squish options, ixmgtool is the same as mergecap.

Say you have put 10 files in a directory and you want to create a single FAT file. If you are curious, we got these files from the good folks who run WRCCDC²⁾

unpl@unpl:~/wr$ ls -lh 
total 2.5G
-rw-rw-r-- 1 unpl unpl 119M Mar 15 20:14 wrccdc.regionals.2019-03-01.111129006380000.pcap
-rw-rw-r-- 1 unpl unpl 112M Mar 15 20:14 wrccdc.regionals.2019-03-01.111133006390000.pcap
-rw-rw-r-- 1 unpl unpl 124M Mar 15 20:14 wrccdc.regionals.2019-03-01.111138006400000.pcap
-rw-rw-r-- 1 unpl unpl 125M Mar 15 20:14 wrccdc.regionals.2019-03-01.111143006410000.pcap
-rw-rw-r-- 1 unpl unpl 106M Mar 15 20:14 wrccdc.regionals.2019-03-01.111147006420000.pcap
-rw-rw-r-- 1 unpl unpl 110M Mar 15 20:14 wrccdc.regionals.2019-03-01.111151006430000.pcap
-rw-rw-r-- 1 unpl unpl 107M Mar 15 20:14 wrccdc.regionals.2019-03-01.111155006440000.pcap
-rw-rw-r-- 1 unpl unpl 105M Mar 15 20:14 wrccdc.regionals.2019-03-01.111159006450000.pcap
-rw-rw-r-- 1 unpl unpl 112M Mar 15 20:14 wrccdc.regionals.2019-03-01.111203006460000.pcap
-rw-rw-r-- 1 unpl unpl 119M Mar 15 20:14 wrccdc.regionals.2019-03-01.111206006470000.pcap
-rw-rw-r-- 1 unpl unpl 113M Mar 15 20:14 wrccdc.regionals.2019-03-01.111210006480000.pcap
-rw-rw-r-- 1 unpl unpl 118M Mar 15 20:14 wrccdc.regionals.2019-03-01.111215006490000.pcap

Running the following command

unpl@unpl:~/wr$ trisul_ixmgtool -squish -r . *.pcap -out fatone.pcap


EOF on wrccdc.regionals.2019-03-01.111203006460000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111159006450000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111147006420000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111143006410000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111210006480000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111206006470000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111151006430000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111155006440000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111129006380000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111138006400000.pcap, bye ! 
Done.

results in a fat pcap

unpl@unpl:~/wrccdc$ ls -lh fatone.pcap 
-rw------- 1 unpl unpl 1.2G Apr 13 13:29 fatone.pcap

To get a really FAT pcap you can use the -squish10 option. This creates 10 dummy flows for each flow by manipulating the source IP to 10 different IPs in the 10.0.0.x range.

unpl@unpl:~/wrccdc$ trisul_ixmgtool -squish10 -r . *.pcap -out really_fatone.pcap

5000000 Packets  15005458762 Bytes Time Fri Mar  1 19:11:31 2019-475695
EOF on wrccdc.regionals.2019-03-01.111203006460000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111159006450000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111147006420000.pcap, bye ! 
8000000 Packets  23767761206 Bytes Time Fri Mar  1 19:11:32 2019-940185
EOF on wrccdc.regionals.2019-03-01.111143006410000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111210006480000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111206006470000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111151006430000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111155006440000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111129006380000.pcap, bye ! 
EOF on wrccdc.regionals.2019-03-01.111138006400000.pcap, bye ! 

unpl@unpl:~/wrccdc$ ls -lh really_fatone.pcap 
-rw------- 1 unpl unpl 13G Apr 13 13:35 really_fatone.pcap

trisul_ixmgtool can be used to create FAT pcaps. These can be very useful for stressing NSM solutons. Using the squish options you can create a mega thick PCAP file for testing by throwing all your PCAP testing files in single directory from varying timestamps and creating a single thick one.

Hope this is useful to the NSM community.

To get the tool (it is free). Install the Trisul Probe package for your platform from the Trisul Download page