Trisul Network Analytics
Developer Zone

User Tools

Site Tools

You are here: Home » offline » Analyzing the WRCCDC PCAP dump using TrisulNSM : Part 1 Approach
Trace:
offline:wrccdc_pcaps

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
offline:wrccdc_pcaps [2018/05/12 15:41] – [Where to start with giant PCAP dumps] veeraoffline:wrccdc_pcaps [2018/05/13 00:10] (current) – [Drilling down] veera
Line 1: Line 1:
-====== Analyzing the WRCCDC PCAP dump using TrisulNSM ======+====== Analyzing the WRCCDC PCAP dump using TrisulNSM : Part 1 Approach ======
  
 The good folks at WRCCDC ((Western Regional Collegiate Cyber Defense Competition))  were kind enough to release [[https://archive.wrccdc.org/|packet captures]] (PCAPS) of the recently concluded event. The entire corpus is roughly 1TB.  Now the question is : **What are the tools that can help you unravel the information in the PCAPs?**  The good folks at WRCCDC ((Western Regional Collegiate Cyber Defense Competition))  were kind enough to release [[https://archive.wrccdc.org/|packet captures]] (PCAPS) of the recently concluded event. The entire corpus is roughly 1TB.  Now the question is : **What are the tools that can help you unravel the information in the PCAPs?** 
  
 +This is Part-1 of a 3 Part series 
  
-In this article, we show you how you can use the TrisulNSM Docker image to perform an end-to-end analysis of the PCAP dump.+  * Part 1: Approach how to avoid getting overwhelmed by large PCAPS  
 +  * [[offline:wrccdc_pcaps_trisulnsm|Part 2: How to use the free TrisulNSM Docker Image to analyze the PCAP dump]] 
 +  * [[offline:wrccdc_pcaps_results|Part 3: Screenshots & video of possible analysis paths (using TrisulNSM)]]
  
-===== Where to start with giant PCAP dumps ===== 
  
-The strategy for dealing with giant PCAPs is similar to capturing live traffic for the very first time from big networks. You start from knowledge about the source of PCAPs and then build a baseline analysis. Then you can spread out into different analysis paths depending on what you are looking for. The tool must support this process end to end. +===== Where to start with giant PCAP dumps =====
  
 +The strategy to avoid getting overwhelmed by giant PCAPs is similar to capturing live traffic for the very first time from big networks. You start from knowledge about the organization and then build a baseline analysis. Then you can spread out into different analysis paths depending on what you are looking for. The tooling you have must support this process end to end. 
  
 In this particular case, here is What we know for sure : In this particular case, here is What we know for sure :
   - the PCAPs are published by WRCCDC   - the PCAPs are published by WRCCDC
   - these are from a cyber defense competition    - these are from a cyber defense competition 
-  - they are big. Big enough to make packet level tools like Wireshark , NetworkMiner etc impractical as first line tool +  - they are big. Big enough to make packet level tools like Wireshark , NetworkMiner etc impractical as first-line tool 
  
  
-So when given a large PCAP dump, we like to divvy up the work into two distinct areas+We like to divvy up the work into two distinct tasksets 
   - **Monitoring Tasks** : Look around at higher level trying to spot patterns, gain understanding, watch outliers. Goal here is to gain total visibility and then discover potential **Drilldown tasks**   - **Monitoring Tasks** : Look around at higher level trying to spot patterns, gain understanding, watch outliers. Goal here is to gain total visibility and then discover potential **Drilldown tasks**
   - **Drilldown Tasks** : You've already identified something that needs further investigation. You now need tooling to increase resolution on that path alone and complete the investigation. The ideal end point for this in NSM would be to drilldown to the packet level.    - **Drilldown Tasks** : You've already identified something that needs further investigation. You now need tooling to increase resolution on that path alone and complete the investigation. The ideal end point for this in NSM would be to drilldown to the packet level. 
Line 24: Line 28:
 ===== Monitoring ===== ===== Monitoring =====
    
- +We like to build a baseline understanding of the network from the PCAPs from the following four angles. In that order. 
-We like to explore the PCAPs from the following perspectives. In this order. +
  
 ==== 1: Traffic Analytics ==== ==== 1: Traffic Analytics ====
Line 31: Line 34:
 First step is to get the "lay of the land". We created TrisulNSM to excel in this area. You can try answering these traffic questions.  First step is to get the "lay of the land". We created TrisulNSM to excel in this area. You can try answering these traffic questions. 
  
-  * How many packets in the PCAP dump? Bytes? How many Flows ? +  * What is the duration of the PCAPs? How many packets in the PCAP dump? Bytes? How many Flows ? 
   * What does the overall bandwidth usage chart look like ?    * What does the overall bandwidth usage chart look like ? 
   * How much of that bandwidth went to external world, how much stayed inside?    * How much of that bandwidth went to external world, how much stayed inside? 
Line 66: Line 69:
 Steps 1-3, will give you a rather solid foundation. By this time, you should have atleast a dozen potential starting points to dig deeper. For example : //"Hey, I am seeing a dozen 500MB+ downloads , need to check what it is".//  At this point, we like to dig a little deeper into advanced counter groups that are available in TrisulNSM out of the box. These give excellent medium resolution for you to investigate. Some of the useful counter groups are : Steps 1-3, will give you a rather solid foundation. By this time, you should have atleast a dozen potential starting points to dig deeper. For example : //"Hey, I am seeing a dozen 500MB+ downloads , need to check what it is".//  At this point, we like to dig a little deeper into advanced counter groups that are available in TrisulNSM out of the box. These give excellent medium resolution for you to investigate. Some of the useful counter groups are :
  
-  * SSL/TLS  counters :  What kind of certificate activity are you seeing ? Who are the top CAs, TLS Organizations, Ciphers used?  Are there any self signed certificates going around ?  +  * **SSL/TLS  counters** :  What kind of certificate activity are you seeing ? Who are the top CAs, TLS Organizations, Ciphers used?  Are there any self signed certificates going around ?  
-  * SNI : Which are the top-K and bottom-K SNI (Server Name Indication) in TLS traffic you're seeing.  +  * **SNI** : Which are the top-K and bottom-K SNI (Server Name Indication) in TLS traffic you're seeing.  
-  * JA3 Hash : The JA3 is a TLS client hello finger print. It is a very good soft indicator that can spot common client applications like Browsers. There are some good malware prints too.  +  * **JA3 Hash** : The JA3 is a TLS client hello finger print. It is a very good soft indicator that can spot common client applications like Browsers. There are some good malware prints too.  
-  * Geo counters : Traffic and Hits by country. You can open up the Bottom-K lists and explore deeper. +  * **Geo counters** : Traffic and Hits by country. You can open up the Bottom-K lists and explore deeper. 
-  * HTTP  counters :  HTTP Error codes by time. +  * **HTTP  counters** :  HTTP Error codes by time. 
  
 Trisul gives you 20 more counter groups, but the above five are good medium resolution starting points. Trisul gives you 20 more counter groups, but the above five are good medium resolution starting points.
Line 76: Line 79:
  
  
 +===== Drilling down  =====
  
 +The relevant factor in Drilldown is whether the analyst is looking for something specific. A security oriented analyst may only want to follow certain drilldown paths from certain starting points. A more general threat-hunting analyst might want to drilldown on all possible leads. These techniques are useful
  
 +  - **Top-K/Bottom-K** : You can use these to find out heavy hitters and rare items. 
 +  - **Graph Analytics** : Expand a particular item to reveal connected items.
 +  - **Resources** : View meta data like TLS Certificates, HTTP Headers, DNS records from lower resolution items such as an HTTP Error code or a rare TLS Cipher.
 +  - **Full Text Search** : For tools based on Log analysis such as Splunk or ELK you can query for logs and aggregations. TrisulNSM is based on streaming analytics but has limited FTS (Full Text Search) capability for HTTP Headers and TLS Certificates.
 +  - **Files** : Dump executables and potential troublesome downloads. TrisulNSM can extract files of any size and show them. 
 +  - **Flows** : The second last level of drill down. From a IP or App or other items you can pull up a list of flows. 
 +  - **Packets** : The final level of drill down.  After this you should have all the information to decide if any escalation is required for action outside the NSM toolset.  
  
  
  
 +==== Next ====
  
-===== Instructions to run TrisulNSM over the PCAPs ===== +Enough of theory. [[offline:wrccdc_pcaps_trisulnsm|Part-2 of this series]] explains how you can get the TrisulNSM Docker image to run over the PCAP dump
- +
- +
- +
- +
-Download the first 8 PCAP files. Roughly 4GB into ''/opt/trisulroot5/wrccdc'' +
- +
-<code> +
-root@unpl:~# ls -lh /opt/trisulroot5/wrccdc/ +
-total 3.8G +
--rw-r--r-- 1 root root 477M May 10 12:28 wrccdc.2018-03-23.010014000000000.pcap +
--rw-r--r-- 1 root root 477M May 10 12:28 wrccdc.2018-03-23.010103000000000.pcap +
--rw-r--r-- 1 root root 477M May 10 12:28 wrccdc.2018-03-23.010232000000000.pcap +
--rw-r--r-- 1 root root 477M May 10 12:28 wrccdc.2018-03-23.010356000000000.pcap +
--rw-r--r-- 1 root root 477M May 10 12:29 wrccdc.2018-03-23.010717000000000.pcap +
--rw-r--r-- 1 root root 477M May 10 12:29 wrccdc.2018-03-23.010904000000000.pcap +
--rw-r--r-- 1 root root 477M Mar 23 20:13 wrccdc.2018-03-23.011313000000000.pcap +
--rw-r--r-- 1 root root 477M Mar 23 20:13 wrccdc.2018-03-23.011338000000000.pcap +
-root@unpl:~#  +
-</code> +
- +
- +
- +
-Run the Docker image over the pcaps +
- +
-<code> +
- +
-sudo docker run  --name=trisul1n \ +
-  --privileged=true --net=host -v /opt/trisulroot5:/trisulroot  \ +
-      -d trisulnsm/trisul6  --enable-file-extraction   \ +
-          --webserver-port 4000 --websockets-port 4003 \ +
-          --fine-resolution  \ +
-              --pcap  wrccdc +
-</code> +
- +
- +
- +
-Upon completion your ''docker logs -f trisul1n'' should show something like below. +
- +
- +
-<code> +
- +
-Finished elapsed : 328 seconds +
- +
- +
-==== SUCCESSFULLY IMPORTED FROM PCAP REPO /trisulroot/wrccdc ===== +
-==== TO VIEW DASHBOARDS ===== +
-==== 1. login to the Web Trisul interface ===== +
-==== 2. select wrccdc1 on the Login Screen ===== +
- +
-Started TrisulNSM docker image. Sleeping. +
- +
-</code> +
- +
- +
- +
-Using Trisul to analyze the PCAPs  +
- +
- +
-File extraction +
- +
-<code> +
-DOCKER:unpl:root savedfiles$ ls /tmp/savedfiles/*.exe -l +
--rw-r--r-- 1 trisul trisul   287392 May 11 12:52 /tmp/savedfiles/00_00_f91a_10.128.0.201__PsGetsid.exe +
--rw-r--r-- 1 trisul trisul   287392 May 11 12:52 /tmp/savedfiles/00_00_fb80_10.128.0.201__PsGetsid.exe +
--rw-r--r-- 1 trisul trisul 12582912 May 11 12:52 /tmp/savedfiles/00_01_dbcf_10.150.0.70__chocolate_debug.exe +
--rw-r--r-- 1 trisul trisul 42846720 May 11 12:52 /tmp/savedfiles/00_01_df63_10.150.0.70__chocolate_debug.exe +
-DOCKER:unpl:root savedfiles$  +
- +
- +
-</code> +
  
offline/wrccdc_pcaps.1526119913.txt.gz · Last modified: 2018/05/12 15:41 by veera