Darktrace Blog

Perspectives on cyber defense

Beyond the hash: How unsupervised machine learning unlocks the true power of JA3

Max Heinemeyer, Director of Threat Hunting | Friday June 22, 2018

Introducing JA3

JA3 is a methodology for fingerprinting Transport Layer Security applications. It was first posted on GitHub in June 2017 and is the work of Salesforce researchers John Althouse, Jeff Atkinson, and Josh Atkins. The JA3 TLS/SSL fingerprints created can overlap between applications but are still a great Indicator of Comprise (IoC). Fingerprinting is achieved by creating a hash of 5 decimal fields of the Client Hello message that is sent in the initial stages of an TLS/SSL session.

JA3 is an interesting approach to the increasing usage of encryption in networks. There is also a clear uptick in cyber-attacks using encrypted command and control (C2) channels – such as HTTPS – for malware communication.

The benefits of JA3 for enhancing rules-and-signatures security

These near-unique fingerprints can be used to enhance traditional cyber security approaches such as whitelisting, blacklisting, and searching for IoCs.

Let’s take the following JA3 hash for example: 3e860202fc555b939e83e7a7ab518c38. According to one of the public lists that maps JA3s to applications, this JA3 hash is associated with the ‘hola_svc’ application. This is the infamous Hola VPN solution that is non-compliant in most enterprise networks. On the other hand, the following hash is associated with the popular messenger software Slack: a5aa6e939e4770e3b8ac38ce414fd0d5. Traditional cyber security tools can use these hashes like traditional signatures to search for instances of them in data sets or trying to blacklist malicious ones.

While there is some merit to this approach, it comes with all the known limitations of rules-and-signatures defenses, such as the overlaps in signatures, the inability to detect unknown threats, as well as the added complexity of having to maintain a database of known signatures.

JA3 in Darktrace

Darktrace creates JA3 hashes for every TLS/SSL connection it encounters. This is incredibly powerful in a number of ways. First, the JA3 can add invaluable context to a threat hunt. Second, Darktrace can also be queried to see if a particular JA3 was encountered in the network, thus providing actionable intelligence during incident response if JA3 IoCs are known to the incident responders.

Things become much more interesting once we apply our unsupervised machine learning to JA3: Darktrace’s AI algorithms autonomously detect which JA3s are anomalous for the network as a whole and which JA3s are unusual for specific devices.

It basically tells a cyber security expert: This JA3 (3e860202fc555b939e83e7a7ab518c38) has never been seen in the network before and it is only used by one device. It indicates that an application, which is used by nobody else on the network, is initiating TLS/SSL connections. In our experience, this is most often the case for malware or non-compliant software. At this stage, we are observing anomalous behavior.

Darktrace’s AI combines these IoCs (Unusual Network JA3, Unusual Device JA3, …) with many other weak indicators to detect the earliest signs of an emerging threat, including previously unknown threats, without using rules or hard-coded thresholds.

Catching Red-Teams and domain fronting with JA3

The following is an example where Darktrace detected a Red-Team’s C2 communication by observing anomalous JA3 behavior.

The unsupervised machine learning algorithms identified a desktop device using a JA3 that was 100% unusual for the network connecting to an external domain using a Let’s Encrypt certificate, which, along with self-signed certificates, is often abused by malicious actors. As well as the JA3, the domain was also 100% rare for the network – nobody else visited it:

It turned out that a Red-Team had registered a domain that was very similar to the victim’s legitimate domain: www.companyname[.]com (legitimate domain) vs. www.companyname[.]online (malicious domain). This was intentionally done to avoid suspicion and human analysis. Over a 7-day period in a 2,000-device environment, this was the only time that Darktrace flagged unusual behavior of this kind.

As the C2 traffic was encrypted (therefore no intrusion detection was possible on the payload) and the domain was non-suspicious (no reputation-based blacklisting worked), this C2 had remained undetected by the rest of the security stack.

Combining unsupervised machine learning with JA3 is incredibly powerful for the detection of domain fronting. Domain fronting is a popular technique to circumvent censorship and to hide C2 traffic. While some infrastructure providers take action to prevent domain fronting on their end, it is still prevalent and actively used by attackers.

The only agreed-upon method within wide parts of the cyber-security community to detect domain fronting appears to be TLS/SSL inspection. This usually involved breaking up encrypted communication to inspect the clear-text payloads. While this works, it commonly involves additional infrastructure, network restructuring and comes with privacy issues – especially in the context of GDPR.

Unsupervised machine learning makes the detection of domain fronting without having to break up encrypted traffic possible by combining unusual JA3 detection with other anomalies such as beaconing. A good start for a domain fronting threat hunt? A device beaconing to an anomalous CDN with an unusual JA3 hash.

Conclusion

JA3 is not a silver bullet to pre-empt malware compromise. As a signature-based solution, it shares the same limitations of all other defenses that rely on pre-identified threats or blacklists: having to play a constant game of catch-up with innovative attackers. However, as a novel means of identifying TLS/SSL applications, JA3 hashing can be leveraged as a powerful network behavioral indicator, an additional metric that can flag the use of unauthorized or risky software, or as a means of identifying emerging malware compromises in the initial stages of C2 communication. This is made possible through the power of unsupervised machine learning.

Blog Archive

Thursday October 4, 2018
Monday August 20, 2018
Monday July 16, 2018
Friday June 22, 2018
Wednesday May 9, 2018
Monday April 16, 2018
Wednesday March 7, 2018
Tuesday February 13, 2018
Friday February 2, 2018
Monday January 22, 2018
Friday December 8, 2017
Monday November 27, 2017
Monday October 30, 2017
Wednesday October 25, 2017
Thursday October 12, 2017
Monday October 2, 2017
Monday September 18, 2017
Monday July 31, 2017
Thursday June 29, 2017
Wednesday June 21, 2017
Wednesday May 17, 2017
Monday May 8, 2017
Wednesday April 5, 2017
Monday March 6, 2017
Monday February 13, 2017
Monday January 30, 2017
Monday January 9, 2017
Friday December 16, 2016
Monday December 5, 2016
Friday November 18, 2016
Friday November 4, 2016
Monday October 24, 2016

About the authors

Justin Fier

Justin Fier is the Director for Cyber Intelligence & Analytics at Darktrace, based in Washington D.C. Justin is one of the US’s leading cyber intelligence experts, and his insights have been widely reported in leading media outlets, including Wall Street Journal, CNN, the Washington Post, and VICELAND. With over 10 years of experience in cyber defense, Justin has supported various elements in the US intelligence community, holding mission-critical security roles with Lockheed Martin, Northrop Grumman Mission Systems and Abraxas. Justin is also a highly-skilled technical specialist, and works with Darktrace’s strategic global customers on threat analysis, defensive cyber operations, protecting IoT, and machine learning.

Dave Palmer

Dave Palmer is the Director of Technology at Darktrace, overseeing the mathematics and engineering teams and project strategies. With over ten years of experience at the forefront of government intelligence operations, Palmer has worked across UK intelligence agencies GCHQ & MI5, where he delivered mission-critical infrastructure services, including the replacement and security of entire global networks, the development of operational internet capabilities and the management of critical disaster recovery incidents. He holds a first-class degree in Computer Science and Software Engineering from the University of Birmingham.

Andrew Tsonchev

Andrew oversees Darktrace’s OT security offerings, providing cyber defense solutions for industrial environments. Andrew has worked extensively across all aspects of Darktrace's technical and commercial operations, and advises Darktrace’s strategic Fortune 500 customers on advanced threat detection, machine learning and autonomous response. Andrew has a technical background in threat analysis and research, and holds a first-class degree in physics from Oxford University and a first-class degree in philosophy from King’s College London.

Max Heinemeyer

Max is a cyber security expert with over eight years’ experience in the field specializing in network monitoring and offensive security. At Darktrace, Max works with strategic customers to help them investigate and respond to threats as well as overseeing the cyber security analyst team in the Cambridge UK headquarters. Prior to his current role, Max led the Threat and Vulnerability Management department for Hewlett-Packard in Central Europe. He was a member of the German Chaos Computer Club, working as a white hat hacker in penetration testing and red teaming engagements. Max holds a MSc from the University of Duisburg-Essen and a BSc from the Cooperative State University Stuttgart in International Business Information Systems.

EnglishFrançais日本語