Differences

This shows you the differences between two versions of the page.

--- offline:defcon26ctf [2018/11/12 22:29] – [Processing the DEFCON 26 CTF PCAPS using Trisul NSM] veera
+++ offline:defcon26ctf [2018/11/12 23:00] (current) – [Port connections over time] veera
@@ Line 3: / Line 3: @@
 With the right tools, analyzing large PCAP dumps can be lots of fun.  This article is a step-by-step of using TrisulNSM to dive into the DEFCON26 CTF PCAP ((The PCAPs can be accessed from the [[https://www.defcon.org/html/links/dc-ctf.html|DEFCON 26 CTF Competition website]]))
-Given only a PCAP file, your first task as an analyst is to make sense of it from multiple angles. I typically like to start off with a statistical overview.
+Given only a large PCAP dump, your first task as an analyst is to make sense of it from multiple angles. I typically like to start off with a statistical overview.
   - What is the timeframe of the CTF?
@@ Line 13: / Line 13: @@
   - then once you have a baseline, You can follow several tracks to investigate in depth. Down to the packet level.
- [[https://trisul.org|TrisulNSM (Trisul Network Analytics)]] is the leading platform today for performing this kind of analysis over very large PCAP files.  This article describes how you can use our free docker image ''trisulnsm/trisul6'' to slurp this PCAP and then have some fun analyzing it.
+ [[https://trisul.org|TrisulNSM (Trisul Network Analytics)]] is the leading platform today for performing this kind of analysis over very large PCAP files.  This article describes how you can use our free docker image ''trisulnsm/trisul6'' to slurp this PCAP and then have some fun analyzing it.  The docker image runs a [[docker:pcap_analysis|2-pass analysis with TrisulNSM]] and then with Suricata and then presents a unified analysis.
@@ Line 22: / Line 22: @@
 So,lets get started. We assume you have a fairly decent Linux machine ready with Docker installed.  Any distro will do because we are going to be using our Docker image.
-Steps
+  - Download the [[https://media.defcon.org/DEF%20CON%2026/DEF%20CON%2026%20ctf/|DEFCON26 PCAP]] , a 5GB file into a directory.
-  - Download the DEFCON26 PCAP , a 5GB file into a directory.
   - Unrar the file and extract the inside PCAP into a filename without spaces such as ''dc26ctf.pcap''
   - Create a directory on your system where the results will be stored. Move the PCAP to that directory so the docker container can see the PCAP file.
@@ Line 33: / Line 31: @@
-Run the trisulnsm/trisul6 docker image over the PCAP
+Run the trisulnsm/trisul6 docker image over the PCAP and run the webserver over port 4000.
 <code bash>
@@ Line 45: / Line 43: @@
 </code>
-To check the logs whether the process was kicked off without issues.
+You can check the docker logs if the process kicked off successfully.
 <code bash>
@@ Line 51: / Line 49: @@
 </code>
-If there are no errors here, it means the process has been kicked off. This can take a while to complete.  Expect anywhere from 20 minutes to an hour depending on your computer's power and whether you have an SSD. To monitor the progress login to the container and check.
+If there are no errors here, it means the process has been kicked off.  Expect anywhere from 20 minutes to an hour depending on your computer's CPU/Memory/SSD configuration ((Since Trisul is a streaming analytics platform, the time taken to process a PCAP dump is linked to the time duration of the PCAP, rather than the volume of traffic)).
+To monitor the progress login to the container and do the following.
 <code bash>
@@ Line 72: / Line 72: @@
 ===== Screenshots =====
-Timeline showing volume of traffic between 10 and 100Mbps over the period of the competition. You can select any timewindow and drill down
+After the processing is complete. You can view the results from the web interface. Here are some sample leads.
+==== Retro Counters ====
-{{:offline:dc26-1.png?400|}}
+Click on //Retro > Retro Counters// to view a Timeline showing traffic bandwidth. Here we see between 10 and 100Mbps spanning a 3-day period of the competition. From here you can select any timewindow and drill down into Counters.
+{{:offline:dc26-1.png?800|}}
-Trend
-{{:offline:dc26-2.png?400|}}
+==== Trend ====
+Clicking the //Topper Trends// tab in Retro counters gives you a timeseries view of top activity of hosts, apps, VLANs.
-Top flows
-{{:offline:dc26-3.png?400|}}
+{{:offline:dc26-2.png?800|}}
-PCAP totals dashboard
+==== Top flows ====
-{{:offline:dc26-4.png?400|}}
+Click on //Dashboards > Sessions// to see top flows by volume, long lived flows, download, upload.  This is a really good place to start because in many CTF or even enterprise loads - elephant flows ((Elephant flows are large volume flows that dominate the bulk of the data transfer))  dominate the overall volume of data. Here we see a single flow from IP 10.13.37.8 pushing nearly 800MB in a 10 Min transfer.
+{{:offline:dc26-3.png?800|}}
+==== PCAP totals dashboard ====
+Open //Dashboards > Show All > PCAP Totals//
+The PCAP Totals dashboard is an excellent place to start off your analysis. On a single dashboard you can see the traffic details, number of unique host, apps, VLANS, TLS Certificates, IDS Alerts, HTTP URLS, SNI, JA3 TLS Fingerprints, and over 40 other types of metrics. You can then click on them to drill down further.
+{{:offline:dc26-4.png?800|}}
+==== Edge Graph Analytics ====
+You can click on the small blue button next to any table item and open "Edge Graph" to reveal neighboring items. Here we went from PCAP Totals > Click on HTTP Status > Then on the weird looking "Status 123"
 Exploring HTTP Status 123
-{{:offline:dc26-5.png?400|}}
+{{:offline:dc26-5.png?800|}}
+==== IDS Alerts, attacks on Drupal ====
+Select //Alerts > Show All > IDS// to show the IDS alert categories seen.  You can then click on an alert to drill down further or pull up PCAPs.
+{{:offline:dc26-6.png?800|}}
+==== Pivot to packets from anywhere ====
+Trisul lets you seamlessly pivot from any analysis point to PCAPs. You can pull down entire PCAP or use the super nifty "PCAP Headers" to only see the top of the PCAP. In the PCAP headers, we show the 'strings' seen in the PCAP header, the actual Hexdump, and a TSHARK like packet summary.
+{{:offline:dc26-7.png?800|}}
+==== Conversations of a particular hosts ====
+Click on Dashboards > Hosts > Then on any host and "Explore Flows" to bring up the Flow explorer.  In TrisulNSM, every flow is stored for instant recall.  You can also select Tools > Explore Flows > Then enter a query expression in the box to retrieve flows.
-Alerts, attacks on Drupal
+{{:offline:dc26-8.png?800|}}
-{{:offline:dc26-6.png?400|}}
+==== Port connections over time  ====
-Pivot to packets from anywhere
+The last one here is quite interesting. Go to Retro Counters > Select the entire Time interval and then select "Apps".  We find that CTF contestants attacking different ports on different days. Hmm, maybe something to look deeper into.
-{{:offline:dc26-7.png?400|}}
+{{:offline:dc26-9.png?800|}}
-Conversations of a particular hosts
+===== Conclusion=====
-{{:offline:dc26-8.png?400|}}
+Hope network analysis enthusiasts find this useful.   The docker image  bundles a [[https://trisul.org|Free License of Trisul]]. PCAP dumps upto 3 days in time can be imported.
+You can also install TrisulNSM natively on your Ubuntu or CentOS and then import the PCAPs there. The Docker image  however makes it really easy.
-Port connections over time
-{{:offline:dc26-9.png?400|}}