Differences

This shows you the differences between two versions of the page.

--- lua:quic [2018/12/12 19:51] – veera
+++ lua:quic [2019/08/14 20:25] (current) – [QUIC protocol analysis using the Trisul Scripting API] veera
@@ Line 2: / Line 2: @@
-BITMAUL
+QUIC (Quick UDP Internet Connection) is a protocol championed by Google to speed up web services by replacing the traditional TCP/HTTP network layer with a new UDP based protocol.  QUIC is almost exclusively used by Google services right now like YouTube, but there is an IETF Internet Draft on it now ((HTTP/3 Internet Draft https://quicwg.org/base-drafts/draft-ietf-quic-http.html)) . The movement is to merge HTTP
+semantics on the UDP based QUIC and call the new thing HTTP/3.  As of today the only QUIC services found in the wild are from the Google stable.
+This article describes how you can pull out key indicators from QUIC into Trisul using the [[https://www.trisul.org/docs/lua/index.html|Lua Scripting API]].
+<note>
+**UPDATES**  14-Aug-19   Updated to support QUIC version 46</note>
+<note>
+The QUIC analysis LUA scripts can be found here in the [[https://github.com/trisulnsm/bitmaul/tree/master/examples/quic|BITMAUL/examples/quic]] repo
+</note>
+===== Network Security Monitoring for QUIC =====
+In the NSM((Network Security Monitoring involves collecting multiple types of data characterizing network traffic http://www.informit.com/articles/article.aspx?p=350391 )) worldview, we would like to collect as much as possible about the QUIC sessions. This would be in addition to //Flow records// and //PCAP// we collect for all flows.
+We were seeing quite a bit of QUIC traffic to YouTube in one of our probes, so we went ahead and got the PCAPs and started analyzing them using Wireshark and the Google QUIC Crypto ((the CRYPTO protocol is documented here at https://github.com/romain-jacotin/quic/blob/master/doc/QUIC_crypto_protocol.md))  document to see what can be extracted. We found the following indicators
+  * **Connection ID** -  a 64-bit random number that would likely be globally unique identifying the QUIC connection.
+  * **SNI** - Server Name Indicator ; similar to the TLS SNI Extension
+  * **Client User Agent** : This was surprising to note that QUIC leaks this before the handshake while in HTTPS (HTTP over TLS)  this is encrypted and we resort to things like JA3 Hash to guess the User Agent.
+  * **Certificate Chain** : Similar to the X.509 certificate chain found in  TLS.
+Then it is a matter of using the open source [[https://github.com/trisulnsm/bitmaul|BITMAUL LUA Protocol Dissection library]] to do the heavy lifting. In about 150 lines of code, we have a working QUIC dissector.
+==== Explaining the scripts ====
+The scripts are on Github at [[https://github.com/trisulnsm/bitmaul/tree/master/examples/quic|BITMAUL/examples/quic|BITMAUL/examples/quic]]
+  - A new [[https://www.trisul.org/docs/lua/protocol_handler.html|protocol_handler]] for QUIC attached to UDP 443 ( ''quic-protocol.lua'' )
+  - A Trisul [[https://www.trisul.org/docs/lua/simple_counter.html|simple_counter]] script that calls for every QUIC packet ( ''quic-simplecounter.lua'' )
+  - The actual QUIC dissector , which returns a LUA table with all fields filled in ( ''quic-dissect.lua'' )
+  - A QUIC certificate decompressor using LuaJIT FFI into zlib
+The [[https://github.com/trisulnsm/bitmaul/blob/master/examples/quic/quic-dissect.lua|''quic-dissect.lua'' script]] is where the real stuff happens, the rest of the files are plumbing into the Trisul platform. Start from there.
+==== Output of the QUIC analysis ====
+The goal of all Trisul scripts is to add some piece of information into the streaming analysis. What we do in quic-simplecounter.lua is.
+  - Tag (Enrich) every flow with the label QUIC
+  - Tag every flow with the ConnectionID
+  - Tag every flow with  User Agent and  SNI (Server Name)
+  - Extract certificate chain and add to Trisul as a SSL Cert Resource (think of this as a log )
+This is how the outputs look like.
+===== Flow Tags =====
+To pull out all QUIC flows go to Tools > Explore Flows > then search for tag=QUIC
+Click to zoom the image, you can see the QUIC flows tagged with QUIC, ConnectionID, Server Name, User-Agent
+{{ :lua:quic2.png?600 |}}
+You can also search for tag=doubleclick.net to pull out QUIC flows from doubleclick.net
+===== Extract X.509 Certificate in QUIC =====
+Just as we do for all SSL flows, we pull out the certificates in QUIC from the server. Apparently QUIC also uses a 64-bit cert FLV.1 hash for well known certificate chains (like googles),but we were unable to get our Chrome browser to use them. We always got full certs.
+This took a while for me to get the certificate extraction right due to the following issues.
+  * the CRT\xff tag contains the certificate which is compressed. This is a bit unexpected because even before the authentication is done, we are running an decompress operation.
+  * you need to use a pre-shared dictionary to do the decompression, it isnt documented anywhere. I finally located the dictionary in the [[https://github.com/google/proto-quic/blob/master/src/net/quic/core/crypto/cert_compressor.cc|chromium source code cert_compressor.cc]]
+  * the certificate spans multiple UDP packets hence needs some reassembly. Put together a very naive reassembly code in quic-dissect.lua
+Go to Resources > SSL Certs > press ENTER or search //quic//
 {{ :lua:quic1.png?600 |}}
+===== Comparison to  Bro/Zeek =====
+The Trisul scripting API allows you to write in LUA rather than a mix of C/Bro language which need a compilation step. We find this is a major efficiency advantage.  Also the BITMAUL protocol analysis framework is fast , safe and covers most common protocol analysis idioms- rather than using C++.  Both are great frameworks for network analysis.  More choice to the user. x
+===== Conclusion =====
+The goal here is to show the power of the Trisul scripting API rather than a production grade QUIC analyzer.
+While the script is working fine in our test environment but putting into production would need some extra work. Particularly when QUIC is used for HD streaming, we need a more efficient way to shunt the stream after the initial handshake otherwise we enter the C->Lua interface for every UDP packet.  After the first 3-4 packets QUIC is all encrypted.
-{{:lua:quic2.png?600|}}
+Head over to the Github page for the [[https://github.com/trisulnsm/bitmaul/tree/master/examples/quic|QUIC analyzer]] to access the scripts.  You can [[https://trisul.org/download|download and install Trisul]] for free and play with this yourself, we have also included a sample PCAP.