Blog
/
/
February 2, 2022

Why AAA Washington Chose Autonomous Response

Learn how AAA Washington improved cybersecurity with an autonomous response. Explore the reasons and benefits behind this strategic decision.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Ron Nichols
Senior Information Security Analyst at AAA Washington (Guest Contributor)
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
02
Feb 2022

AAA Washington is best known for its emergency road service, but operates in a broader range of areas including insurance and travel. Our priorities from a security side are two-fold: making sure we are adequately prepared to defend against advanced and pertinent threats like ransomware, and protecting the sensitive data of our employees and our members.

About two years ago, we hit a fork in the road. Our information security team was conscious that we had a gap in real-time monitoring, and in particular, 24/7 response. It wasn’t that we didn’t already have tools in place, or that we weren’t shipping logs, we just didn’t have a 24/7 protocol. So if an attack were to come in at 3am, for example, we weren’t confident enough in our ability to take immediate action to contain the threat.

So we looked at two options. It was our Matrix ‘red pill or blue pill’ moment: a choice between the willingness to learn a life-changing truth by taking the red pill, or taking the blue pill and opting for the more traditional path.

For us, that blue pill – and what many recommended at the time – was the option of consulting an external 24/7 Security Operations Center. We knew this would solve our problem, but it also had a lot of drawbacks, mainly around time consumption: you have to get a service-level agreement (SLA) in place, set up SNMP traps, ship logs over to the SOC, who are then tasked with untangling those logs. You know that the SOC is then looking at AAA Washington’s environment along with hundreds of others. You’ve got to develop a relationship with the SOC technician who doesn’t know the nuances of your environment or your business logic…

So understandably there was a level of reluctance there.

And then we had the red pill, which for us, was Darktrace, offering AI technology that could learn our environment all by itself, and respond autonomously to emerging attacks. No steep learning curve, no ongoing maintenance.

We had to try it. Cloud deployments are available but even for our on-prem arrangement, the trial process was a no-brainer: we got the box, plugged it in, and we were off and going. If we didn’t like it, all we had to do was unplug it and ship it back.

The visibility Darktrace gave us was immediately apparent, and in that first week it alerted us to the fact that every other night, 1GB of outbound traffic was going to an East Coast data center from our back-up appliance. We thought we knew what was going on in our digital enterprise, but we had no idea – Darktrace providing that knowledge and filling those gaps showed us that this was heading exactly in the direction we wanted.

Autonomous Response

So full marks for visibility and anomaly detection, but what about that response capability that led us to consider Darktrace in the first place? We were keen to see what actions Antigena would recommend and assess their accuracy and severity.

Being naturally risk-averse at AAA Washington, we initially set Antigena up in human confirmation mode, meaning an operator had to give the green light before it took action. It took about two weeks for it to learn the nuances of our digital environment, and it wasn’t long before we found its actions were extremely accurate, and minimally disruptive.

It never took drastic action like quarantining a device, it simply stopped what we needed it to. It played a significant role in protecting us in the wake of some high-profile attacks, including the SUNBURST attacks and the more recent Log4shell vulnerability.

Adapting to a hybrid cloud strategy

In the two years since deploying Darktrace, we have made significant changes to our digital infrastructure – including, like so many others, migrating to the cloud. I wondered whether we would lose the visibility and protection we got from Darktrace when this happened.

But with its dedicated SaaS Modules for Microsoft 365 and others, Darktrace had this covered. It’s been able to shed a light on malicious activity occurring across our full Microsoft 365 product suite.

We can see things like unusual email forwarding rules that indicate an account takeover. With other tools, it takes six to eight clicks to find that information. The information is available, but accessing that data is a complex and convoluted process. Darktrace delivers that holy grail of having a single pane of glass view in a security tool. Having that detailed one stop view means reducing mean time to understanding, and mean time to response.

Self-Learning AI on the endpoint

And when large-scale remote working came about, Darktrace again brought visibility and Autonomous Response to cover our endpoint devices, protecting them from threats like ransomware that would go undetected from network coverage alone. The ability to stop these threats at the first hurdle, before they spread and infected other devices, was crucial for us.

It was another case of Darktrace adapting, and another reason I’m confident about working with Darktrace as a long-term partner: every time I think Darktrace is going to not be as relevant, these new developments bring us up to speed.

Keeping the show on the road

Darktrace has done exactly what we wanted to do by filling that gap we had in 24/7 response. But it has gone further by proving that time and time again, it can adapt as our digital infrastructure changes and grows, and can cover our employees wherever they work.

The technology presents us with all the information we need in a single pane of glass with the Threat Visualizer. With the Mobile App, I can get notifications of high-priority alerts and Darktrace’s autonomous actions, wherever I am. And when there’s a serious incident, there is always someone available to offer support and get me what I need to know, fast.

Taking that red pill all those months ago was one of the best decisions I’ve made as an IT security professional. Whatever challenges are down the road, I’m confident Darktrace will be there to meet them.

Hear from more Darktrace customers

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Ron Nichols
Senior Information Security Analyst at AAA Washington (Guest Contributor)

More in this series

No items found.

Blog

/

Network

/

August 8, 2025

Ivanti Under Siege: Investigating the Ivanti Endpoint Manager Mobile Vulnerabilities (CVE-2025-4427 & CVE-2025-4428)

ivanti cve exploitation edge infrastructure Default blog imageDefault blog image

Ivanti & Edge infrastructure exploitation

Edge infrastructure exploitations continue to prevail in today’s cyber threat landscape; therefore, it was no surprise that recent Ivanti Endpoint Manager Mobile (EPMM) vulnerabilities CVE-2025-4427 and CVE-2025-4428 were exploited targeting organizations in critical sectors such as healthcare, telecommunications, and finance across the globe, including across the Darktrace customer base in May 2025.

Exploiting these types of vulnerabilities remains a popular choice for threat actors seeking to enter an organization’s network to perform malicious activity such as cyber espionage, data exfiltration and ransomware detonation.

Vulnerabilities in Ivanti EPMM

Ivanti EPMM allows organizations to manage and configure enterprise mobile devices. On May 13, 2025, Ivanti published a security advisory [1] for their Ivanti Endpoint Manager Mobile (EPMM) devices addressing a medium and high severity vulnerability:

  • CVE-2025-4427, CVSS: 5.6: An authentication bypass vulnerability
  • CVE-2025-4428, CVSS: 7.2: Remote code execution vulnerability

Successfully exploiting both vulnerabilities at the same time could lead to unauthenticated remote code execution from an unauthenticated threat actor, which could allow them to control, manipulate, and compromise managed devices on a network [2].

Shortly after the disclosure of these vulnerabilities, external researchers uncovered evidence that they were being actively exploited in the wild and identified multiple indicators of compromise (IoCs) related to post-exploitation activities for these vulnerabilities [2] [3]. Research drew particular attention to the infrastructure utilized in ongoing exploitation activity, such as leveraging the two vulnerabilities to eventually deliver malware contained within ELF files from Amazon Web Services (AWS) S3 bucket endpoints and to deliver KrustyLoader malware for persistence. KrustyLoader is a Rust based malware that was discovered being downloaded in compromised Ivanti Connect Secure systems back in January 2024 when the zero-day critical vulnerabilities; CVE-2024-21887 and CVE-2023-46805 [10].

This suggests the involvement of the threat actor UNC5221, a suspected China-nexus espionage actor [3].

In addition to exploring the post-exploit tactics, techniques, and procedures (TTPs) observed for these vulnerabilities across Darktrace’s customer base, this blog will also examine the subtle changes and similarities in the exploitation of earlier Ivanti vulnerabilities—specifically Ivanti Connect Secure (CS) and Policy Secure (PS) vulnerabilities CVE-2023-46805 and CVE-2024-21887 in early 2024, as well as CVE-2025-0282 and CVE-2025-0283, which affected CS, PS, and Zero Trust Access (ZTA) in January 2025.

Darktrace Coverage

In May 2025, shortly after Ivanti disclosed vulnerabilities in their EPMM product, Darktrace’s Threat Research team identified attack patterns potentially linked to the exploitation of these vulnerabilities across multiple customer environments. The most noteworthy attack chain activity observed included exploit validation, payload delivery via AWS S3 bucket endpoints, subsequent delivery of script-based payloads, and connections to dpaste[.]com, possibly for dynamic payload retrieval. In a limited number of cases, connections were also made to an IP address associated with infrastructure linked to SAP NetWeaver vulnerability CVE-2025-31324, which has been investigated by Darktrace in an earlier case.

Exploit Validation

Darktrace observed devices within multiple customer environments making connections related to Out-of-Band Application Security Testing (OAST). These included a range of DNS requests and connections, most of which featured a user agent associated with the command-line tool cURL, directed toward associated endpoints. The hostnames of these endpoints consisted of a string of randomly generated characters followed by an OAST domain, such as 'oast[.]live', 'oast[.]pro', 'oast[.]fun', 'oast[.]site', 'oast[.]online', or 'oast[.]me'. OAST endpoints can be leveraged by malicious actors to trigger callbacks from targeted systems, such as for exploit validation. This activity, likely representing the initial phase of the attack chain observed across multiple environments, was also seen in the early stages of previous investigations into the exploitation of Ivanti vulnerabilities [4]. Darktrace also observed similar exploit validation activity during investigations conducted in January 2024 into the Ivanti CS vulnerabilities CVE-2023-46805 and CVE-2024-21887.

Payload Delivery via AWS

Devices across multiple customer environments were subsequently observed downloading malicious ELF files—often with randomly generated filenames such as 'NVGAoZDmEe'—from AWS S3 bucket endpoints like 's3[.]amazonaws[.]com'. These downloads occurred over HTTP connections, typically using wget or cURL user agents. Some of the ELF files were later identified to be KrustyLoader payloads using open-source intelligence (OSINT). External researchers have reported that the KrustyLoader malware is executed in cases of Ivanti EPMM exploitation to gain and maintain a foothold in target networks [2].

In one customer environment, after connections were made to the endpoint fconnect[.]s3[.]amazonaws[.]com, Darktrace observed the target system downloading the ELF file mnQDqysNrlg via the user agent Wget/1.14 (linux-gnu). Further investigation of the file’s SHA1 hash (1dec9191606f8fc86e4ae4fdf07f09822f8a94f2) linked it to the KrustyLoader malware [5]. In another customer environment, connections were instead made to tnegadge[.]s3[.]amazonaws[.]com using the same user agent, from which the ELF file “/dfuJ8t1uhG” was downloaded. This file was also linked to KrustyLoader through its SHA1 hash (c47abdb1651f9f6d96d34313872e68fb132f39f5) [6].

The pattern of activity observed so far closely mirrors previous exploits associated with the Ivanti vulnerabilities CVE-2023-46805 and CVE-2024-21887 [4]. As in those cases, Darktrace observed exploit validation using OAST domains and services, along with the use of AWS endpoints to deliver ELF file payloads. However, in this instance, the delivered payload was identified as KrustyLoader malware.

Later-stage script file payload delivery

In addition to the ELF file downloads, Darktrace also detected other file downloads across several customer environments, potentially representing the delivery of later-stage payloads.

The downloaded files included script files with the .sh extension, featuring randomly generated alphanumeric filenames. One such example is “4l4md4r.sh”, which was retrieved during a connection to the IP address 15.188.246[.]198 using a cURL-associated user agent. This IP address was also linked to infrastructure associated with the SAP NetWeaver remote code execution vulnerability CVE-2025-31324, which enables remote code execution on NetWeaver Visual Composer. External reporting has attributed this infrastructure to a China-nexus state actor [7][8][9].

In addition to the script file downloads, devices on some customer networks were also observed making connections to pastebin[.]com and dpaste[.]com, two sites commonly used to host or share malicious payloads or exploitation instructions [2]. Exploits, including those targeting Ivanti EPMM vulnerabilities, can dynamically fetch malicious commands from sites like dpaste[.]com, enabling threat actors to update payloads. Unlike the previously detailed activity, this behavior was not identified in any prior Darktrace investigations into Ivanti-related vulnerabilities, suggesting a potential shift in the tactics used in post-exploitation stages of Ivanti attacks.

Conclusion

Edge infrastructure vulnerabilities, such as those found in Ivanti EPMM and investigated across customer environments with Darktrace / NETWORK, have become a key tool in the arsenal of attackers in today’s threat landscape. As highlighted in this investigation, while many of the tactics employed by threat actors following successful exploitation of vulnerabilities remain the same, subtle shifts in their methods can also be seen.

These subtle and often overlooked changes enable threat actors to remain undetected within networks, highlighting the critical need for organizations to maintain continuous extended visibility, leverage anomaly based behavioral analysis, and deploy machine speed intervention across their environments.

Credit to Nahisha Nobregas (Senior Cyber Analyst) and Anna Gilbertson (Senior Cyber Analyst)

Appendices

Mid-High Confidence IoCs

(IoC – Type - Description)

-       trkbucket.s3.amazonaws[.]com – Hostname – C2 endpoint

-       trkbucket.s3.amazonaws[.]com/NVGAoZDmEe – URL – Payload

-       tnegadge.s3.amazonaws[.]com – Hostname – C2 endpoint

-       tnegadge.s3.amazonaws[.]com/dfuJ8t1uhG – URL – Payload

-       c47abdb1651f9f6d96d34313872e68fb132f39f5 - SHA1 File Hash – Payload

-       4abfaeadcd5ab5f2c3acfac6454d1176 - MD5 File Hash - Payload

-       fconnect.s3.amazonaws[.]com – Hostname – C2 endpoint

-       fconnect.s3.amazonaws[.]com/mnQDqysNrlg – URL - Payload

-       15.188.246[.]198 – IP address – C2 endpoint

-       15.188.246[.]198/4l4md4r.sh?grep – URL – Payload

-       185.193.125[.]65 – IP address – C2 endpoint

-       185.193.125[.]65/c4qDsztEW6/TIGHT_UNIVERSITY – URL – C2 endpoint

-       d8d6fe1a268374088fb6a5dc7e5cbb54 – MD5 File Hash – Payload

-       64.52.80[.]21 – IP address – C2 endpoint

-       0d8da2d1.digimg[.]store – Hostname – C2 endpoint

-       134.209.107[.]209 – IP address – C2 endpoint

Darktrace Model Detections

-       Compromise / High Priority Tunnelling to Bin Services (Enhanced Monitoring Model)

-       Compromise / Possible Tunnelling to Bin Services

-       Anomalous Server Activity / New User Agent from Internet Facing System

-       Compliance / Pastebin

-       Device / Internet Facing Device with High Priority Alert

-       Anomalous Connection / Callback on Web Facing Device

-       Anomalous File / Script from Rare External Location

-       Anomalous File / Incoming ELF File

-       Device / Suspicious Domain

-       Device / New User Agent

-       Anomalous Connection / Multiple Connections to New External TCP Port

-       Anomalous Connection / New User Agent to IP Without Hostname

-       Anomalous File / EXE from Rare External Location

-       Anomalous File / Internet Facing System File Download

-       Anomalous File / Multiple EXE from Rare External Locations

-       Compromise / Suspicious HTTP and Anomalous Activity

-       Device / Attack and Recon Tools

-       Device / Initial Attack Chain Activity

-       Device / Large Number of Model Alerts

-       Device / Large Number of Model Alerts from Critical Network Device

References

1.     https://forums.ivanti.com/s/article/Security-Advisory-Ivanti-Endpoint-Manager-Mobile-EPMM?language=en_US

2.     https://blog.eclecticiq.com/china-nexus-threat-actor-actively-exploiting-ivanti-endpoint-manager-mobile-cve-2025-4428-vulnerability

3.     https://www.wiz.io/blog/ivanti-epmm-rce-vulnerability-chain-cve-2025-4427-cve-2025-4428

4.     https://www.darktrace.com/blog/the-unknown-unknowns-post-exploitation-activities-of-ivanti-cs-ps-appliances

5.     https://www.virustotal.com/gui/file/ac91c2c777c9e8638ec1628a199e396907fbb7dcf9c430ca712ec64a6f1fcbc9/community

6.     https://www.virustotal.com/gui/file/f3e0147d359f217e2aa0a3060d166f12e68314da84a4ecb5cb205bd711c71998/community

7.     https://www.virustotal.com/gui/ip-address/15.188.246.198

8.     https://blog.eclecticiq.com/china-nexus-nation-state-actors-exploit-sap-netweaver-cve-2025-31324-to-target-critical-infrastructures

9.     https://www.darktrace.com/blog/tracking-cve-2025-31324-darktraces-detection-of-sap-netweaver-exploitation-before-and-after-disclosure

10.  https://www.synacktiv.com/en/publications/krustyloader-rust-malware-linked-to-ivanti-connectsecure-compromises

The content provided in this blog is published by Darktrace for general informational purposes only and reflects our understanding of cybersecurity topics, trends, incidents, and developments at the time of publication. While we strive to ensure accuracy and relevance, the information is provided “as is” without any representations or warranties, express or implied. Darktrace makes no guarantees regarding the completeness, accuracy, reliability, or timeliness of any information presented and expressly disclaims all warranties.

Nothing in this blog constitutes legal, technical, or professional advice, and readers should consult qualified professionals before acting on any information contained herein.

Any references to third-party organizations, technologies, threat actors, or incidents are for informational purposes only and do not imply affiliation, endorsement, or recommendation.

Darktrace, its affiliates, employees, or agents shall not be held liable for any loss, damage, or harm arising from the use of or reliance on the information in this blog.

The cybersecurity landscape evolves rapidly, and blog content may become outdated or superseded. We reserve the right to update, modify, or remove any content without notice.

Continue reading
About the author
Nahisha Nobregas
SOC Analyst

Blog

/

/

August 7, 2025

How CDR & Automated Forensics Transform Cloud Incident Response

cloud security investigation guy on computer doing workDefault blog imageDefault blog image

Introduction: Cloud investigations

In cloud security, speed, automation and clarity are everything. However, for many SOC teams, responding to incidents in the cloud is often very difficult especially when attackers move fast, infrastructure is ephemeral, and forensic skills are scarce.

In this blog we will walk through an example that shows exactly how Darktrace Cloud Detection and Response (CDR) and automated cloud forensics together, solve these challenges, automating cloud detection, and deep forensic investigation in a way that’s fast, scalable, and deeply insightful.

The Problem: Cloud incidents are hard to investigate

Security teams often face three major hurdles when investigating cloud detections:

Lack of forensic expertise: Most SOCs and security teams aren’t natively staffed with forensics specialists.

Ephemeral infrastructure: Cloud assets spin up and down quickly, leaving little time to capture evidence.

Lack of existing automation: Gathering forensic-level data often requires manual effort and leaves teams scrambling around during incidents — accessing logs, snapshots, and system states before they disappear. This process is slow and often blocked by permissions, tooling gaps, or lack of visibility.

How Darktrace augments cloud investigations

1. Darktrace’s CDR finds anomalous activity in the cloud

An alert is generated for a large outbound data transfer from an externally facing EC2 instance to a rare external endpoint. It’s anomalous, unexpected, and potentially serious.

2. AI-led investigation stitches together the incident for a SOC analyst to look into

When a security incident unfolds, Darktrace’s Cyber AI Analyst TM is the first to surface it, automatically correlating behaviors, surfacing anomalies, and presenting a cohesive incident summary. It’s fast, detailed, and invaluable.

Once the incident is created, more questions are raised.

  • How were the impacted resources compromised?
  • How did the attack unfold over time – what tools and malware were used?
  • What data was accessed and exfiltrated?

What you’ll see as a SOC analyst: The incident begins in Darktrace’s Threat Visualizer, where a Cyber AI Analyst incident has been generated automatically highlighting large anomalous data transfer to a suspicious external IP. This isn’t just another alert, it’s a high-fidelity signal backed by Darktrace’s Self-Learning AI.

Cyber AI Analyst incident created for anomalous outbound data transfer
Figure 1: Cyber AI Analyst incident created for anomalous outbound data transfer

The analyst can then immediately pivot to Darktrace / CLOUD’s architecture view (see below), gaining context on the asset’s environment, ingress/egress points, connected systems, potential attack paths and whether there are any current misconfigurations detected on the asset.

Darktrace / CLOUD architecture view providing critical cloud context
Figure 2: Darktrace / CLOUD architecture view providing critical cloud context

3. Automated forensic capture — No expertise required

Then comes the game-changer, Darktrace’s recent acquisition of Cado enhances its cloud forensics capabilities. From the first alert triggered, Darktrace has already kicked in and automatically processed and analyzed a full volume capture of the EC2. Everything, past and present, is preserved. No need for manual snapshots, CLI commands, or specialist intervention.

Darktrace then provides a clear timeline highlighting the evidence and preserving it. In our example we identify:

  • A brute-force attempt on a file management app, followed by a successful login
  • A reverse shell used to gain unauthorized remote access to the EC2
  • A reverse TCP connection to the same suspicious IP flagged by Darktrace
  • Attacker commands showing how the data was split and prepared for exfiltration
  • A file (a.tar) created from two sensitive archives: product_plans.zip and research_data.zip

All of this is surfaced through the timeline view, ranked by significance using machine learning. The analyst can pivot through time, correlate events, and build a complete picture of the attack — without needing cloud forensics expertise.

Darktrace even gives the ability to:

  • Download and inspect gathered files in full detail, enabling teams to verify exactly what data was accessed or exfiltrated.
  • Interact with the file system as if it were live, allowing investigators to explore directories, uncover hidden artifacts, and understand attacker movement with precision.
Figure 3 Cado critical forensic investigation automated insights
Figure 3: Cado critical forensic investigation automated insights
Figure 4: Cado forensic file analysis of reverse shell and download option
Figure 5: a.tar created from two sensitive archives: product_plans.zip and research_data.zip
Figure 6: Traverse the full file system of the asset

Why this matters?

This workflow solves the hardest parts of cloud investigation:

  1. Capturing evidence before it disappears
  2. Understanding attacker behavior in detail - automatically
  3. Linking detections to impact with full incident visibility

This kind of insight is invaluable for organizations especially regulated industries, where knowing exactly what data was affected is critical for compliance and reporting. It’s also a powerful tool for detecting insider threats, not just external attackers.

Darktrace / CLOUD and Cado together acts as a force multiplier helping with:

  • Reducing investigation time from hours to minutes
  • Preserving ephemeral evidence automatically
  • Empowering analysts with forensic-level visibility

Cloud threats aren’t slowing down. Your response shouldn’t either. Darktrace / CLOUD + Cado gives your SOC the tools to detect, contain, and investigate cloud incidents — automatically, accurately, and at scale.

[related-resource]

Continue reading
About the author
Adam Stevens
Director of Product, Cloud Security
Your data. Our AI.
Elevate your network security with Darktrace AI