Network Security Monitoring aka NSM involves the collection of a variety of information elements by passive observation of network traffic. There are essentially two ways to run packet analytics, Live Capture vs Reading PCAP files. They may appear to be the same, but there are some crucial differences that are hidden only to surface when you try to read in PCAP files.

When you have a large PCAP file , say covering 6 hours of traffic. You have two choices, you can choose to replay the file to a NSM system using a tool like TCPReplay. But if you did that at normal rate it would take 6 hours !! If you speed it up or play it at top speed, you lose crucial information about traffic metrics.

But if you played it back at natural rate, then that is an enormous waste of CPU and time. You could be processing packets at the natural rate of 1MB/s for hours when your powerful CPU can do 800Mbps. This is why we prefer to read PCAP files, presumably we can just let our CPU rip through as fast as possible.

All packet capture tools I know are clocked by the timestamps present in the packets themselves.

In a Live traffic capture, the wall time is the clock. During low traffic periods your CPU and Memory usage goes down, but the time moves at fixed rate. Tick, tock..

When you read PCAPs, the time moves faster or slower depending on how fast your tool can inject the PCAP files.

A process-A that generates info-A can rip through the PCAP file at 100Mbps, another process-B generating info-B may only be able to process at 5Mbps. If both of these information streams hit a single backend, then we may have these problems.

1. If Event-B and Event-A that occurred at the same time in real world, arrive at the backend 40 minutes apart. Can they be stored and indexed correctly ?
2. If Event-B generates some new enrichment data about Event-A and they arrive 40 minutes late at the backend. What happens to the enrichment?

A lot of this depends on the capabilities of the database.

Some protocols need to maintain state. A good example is FTP which has a control channel which sets up a data channel. Another is SSL Session resumption, or SIP/TRP for VOIP, etc.

Your tools needs to maintain this state in order to decode the protocols correctly. A single threaded processing pipeline may not need a state synchronization but a multi threaded pipe does.

Even for PCAP file reads, we can benefit by multiple threads. Trisul uses 2 threads by default, you can also use 4, or 8 threads to speed up PCAP file reads. The problem here is the following.

• If you have TWO threads of processing , now thread A and thread B need to synchronize their state every X packets or every X seconds, etc.
• Say you synchronize every 10ms. In a real world FTP capture, there will be enough lag between the control and data channels, so you can get away with this sync rate.
• In a PCAP import, you can have 1 control packet and immediately followed by a data packet that might go to another thread. Since you are reading as fast as you can, the state doesnt sync.

There are some other less important differences, we can document in the next article.