Blog
/
Proactive Security
/
June 25, 2024

Let the Dominos Fall! SOC and IR Metrics for ROI

Vendors are scrambling to compare MTTD metrics laid out in the latest MITRE Engenuity ATT&CK® Evaluations. But this analysis is reductive, ignoring the fact that in cybersecurity, there are far more metrics that matter.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
John Bradshaw
Sr. Director, Technical Marketing
Default blog image
25
Jun 2024

One of the most enjoyable discussions (and debates) I engage in is the topic of Security Operations Center (SOC) and Incident Response (IR) metrics to measure and validate an organization’s Return on Investment (ROI). The debate part comes in when I hear vendor experts talking about “the only” SOC metrics that matter, and only list the two most well-known, while completely ignoring metrics that have a direct causal relationship.

In this blog, I will discuss what I believe are the SOC/IR metrics that matter, how each one has a direct impact on the others, and why organizations should ensure they are working towards the goal of why these metrics are measured in the first place: Reduction of Risk and Costs.

Reduction of Risk and Costs

Every security solution and process an organization puts in place should reduce the organization’s risk of a breach, exposure by an insider threat, or loss of productivity. How an organization realizes net benefits can be in several ways:

  • Improved efficiencies can result in SOC/IR staff focusing on other areas such as advanced threat hunting rather than churning through alerts on their security consoles. It may also help organizations dealing with the lack of skilled security staff by using Artificial Intelligence (AI) and automated processes.
  • A well-oiled SOC/IR team that has greatly reduced or even eliminated mundane tasks attracts, motivates, and retains talent resulting in reduced hiring and training costs.
  • The direct impact of a breach such as a ransomware attack can be devastating. According to the 2024 Data Breach Investigations Report by Verizon, MGM Resorts International reported the ALPHV ransomware cost the company approximately $100 million[1].
  • Failure to take appropriate steps to protect the organization can result in regulatory fines; and if an organization has, or is considering, purchasing Cyber Insurance, can result in declined coverage or increased premiums.

How does an organization demonstrate they are taking proactive measures to prevent breaches? That is where it's important to understand the nine (yes, nine) key metrics, and how each one directly influences the others, play their roles.

Metrics in the Incident Response Timeline

Let’s start with a review of the key steps in the Incident Response Timeline:

Seven of the nine key metrics are in the IR timeline, while two of the metrics occur before you ever have an incident. They occur in the Pre-Detection Stage.

‍Pre-Detection stage metrics are:

  • Preventions Per Intrusion Attempt (PPIA)
  • False Positive Reduction Rate (FPRR)

Next is the Detect and Investigate stage, there are three metrics to consider:

  • Mean Time to Detection (MTTD)
  • Mean Time to Triage (MTTT)
  • Mean Time to Understanding (MTTU)

This is followed by the Remediation stage, there are two metrics here:

  • Mean Time to Containment (MTTC)
  • Mean Time to Remediation / Recovery (MTTR)

Finally, there is the Risk Reduction stage, there are two metrics:

  • Mean Time to Advice (MTTA)
  • Mean Time to Implementation (MTTI)

Pre-Detection Stage

Preventions Per Intrusion Attempt

PPIA is defined as stopping any intrusion attempt at the earliest possible stage. Your network Intrusion Prevention System (IPS) blocks vulnerability exploits, your e-mail security solution intercepts and removes messages with malicious attachments or links, your egress firewall blocks unauthorized login attempts, etc. The adversary doesn’t get beyond Step 1 in the attack life cycle.

This metric is the first domino. Every organization should strive to improve on this metric every day. Why? For every intrusion attempt you stop right out of the gate, you eliminate the actions for every other metric. There is no incident to detect, triage, investigate, remediate, or analyze post-incident for ways to improve your security posture.

When I think about PPIA, I always remember back to a discussion with a former mentor, Tim Crothers, who discussed the benefits of focusing on Prevention Failure Detection.

The concept is that as you layer your security defenses, your PPIA moves ever closer to 100% (no one has ever reached 100%). This narrows the field of fire for adversaries to breach into your organization. This is where novel, unknown, and permuted threats live and breathe. This is where solutions utilizing Unsupervised Machine Learning excel in raising anomalous alerts – indications of potential compromise involving one of these threats. Unsupervised ML also raises alerts on anomalous activity generated by known threats and can raise detections before many signature-based solutions. Most organizations struggle to find strong permutations of known threats, insider threats, supply chain attacks, attacks utilizing n-day and 0-day exploits. Moving PPIA ever closer to 100% also frees your team up for conducting threat hunting activities – utilizing components of your SOC that collect and store telemetry to query for potential compromises based on hypothesis the team raises. It also significantly reduces the alerts your team must triage and investigate – solving many of the issues outlined at the start of this paper.

False Positive Reduction Rate

Before we discuss FPRR, I should clarify how I define False Positives (FPs). Many define FPs as an alert that is in error (i.e.: your EDR alerts on malware that turns out to be AV signature files). While that is a FP, I extend the definition to include any alert that did not require triage / investigation and distracts the SOC/IR team (meaning they conducted some level of triage / investigation).

This metric is the second domino. Why is this metric important? Every alert your team exerts time and effort on that is a non-issue distracts them from alerts that matter. One of the major issues that has resonated in the security industry for decades is that SOCs are inundated with alerts and cannot clear the backlog. When it comes to PPIA + FPRR, I have seen analysts spend time investigating alerts that were blocked out of the gate while their screen continued to fill up with more. You must focus on Prevention Failure Detection to get ahead of the backlog.

Detect and Investigate Stages

Mean Time to Detection

MTTD, or “Dwell Time”, has decreased dramatically over the past 12 years. From well over a year to 16 days in 2023[2]. MTTD is measured from the earliest possible point you could detect the intrusion to the moment you actually detect it.

This third domino is important because the longer an adversary remains undetected, the more the odds increase they will complete their mission objective. It also makes the tasks of triage and investigation more difficult as analysts must piece together more activity and adversaries may be erasing evidence along the way – or your storage retention does not cover the breach timeline.

Many solutions focusing solely on MTTD can actually create the very problem SOCs are looking to solve.  That is, they generate so much alerting that they flood the console, email, or text messaging app causing an unmanageable queue of alerts (this is the problem XDR solutions were designed to resolve by focusing on incidents rather than alerts).

Mean Time to Triage

MTTT involves SOCs that utilize Level 1 (aka Triage) analysts to render an “escalate / do not escalate” alert verdict accurately. Accuracy is important because Triage Analysts typically are staff new to cyber security (recent grad / certification) and may over escalate (afraid to miss something important) or under escalate (not recognize signs of a successful breach). Because of this, a small MTTT does not always equate to successful handling of incidents.

This metric is important because keeping your senior staff focused on progressing incidents in a timely manner (and not expending time on false positives) should reduce stress and required headcount.

Mean Time to Understanding

MTTU deals with understanding the complete nature of the incident being investigated. This is different than MTTT which only deals with whether the issue merits escalation to senior analysts. It is then up to the senior analysts to determine the scope of the incident, and if you are a follower of my UPSET Investigation Framework, you know understanding the full scope involves:

U = All compromised accounts

P = Persistence Mechanisms used

S = All systems involved (organization, adversary, and intermediaries)

E = Endgame (or mission objective)

T = Techniques, Tactics, Procedures (TTPs) utilized by the adversary

MTTU is important because this information is critical before any containment or remediation actions are taken. Leave a stone unturned, and you alert the adversary that you are onto them and possibly fail to close an avenue of access.

Remediation Stages

Mean Time to Containment

MTTC deals with neutralizing the threat. You may not have kicked the adversary out, but you have halted their progress to their mission objective and ability to inflict further damage. This may be through use of isolation capabilities, termination of malicious processes, or firewall blocks.

MTTC is important, especially with ransomware attacks where every second counts. Faster containment responses can result in reduced / eliminated disruption to business operations or loss of data.

Mean Time to Remediation / Recovery

The full scope of the incident is understood, the adversary has been halted in their tracks, no malicious processes are running on any systems in your organization. Now is the time to put things back to right. MTTR deals with the time involved in restoring business operations to pre-incident stage. It means all remnants of changes made by the adversary (persistence, account alterations, programs installed, etc.) are removed; all disrupted systems are restored to operations (i.e.: ransomware encrypted systems are recovered from backups / snapshots), compromised user accounts are reset, etc.

MTTR is important because it informs senior management of how fast the organization can recover from an incident. Disaster Recovery and Business Continuity plans play a major role in improving this score.

Risk Reduction Stages

Mean Time to Advice

After the dust has settled from the incident, the job is not done. MTTA deals with identifying and assessing the specific areas (vulnerabilities, misconfigurations, lack of security controls) that permitted the adversary to advance to the point where detection occurred (and any actions beyond). The SOC and IR teams should then compile a list of recommendations to present to management to improve the security posture of the organization so the same attack path cannot be used.

Mean Time to Implement

Once recommendations are delivered to management, how long does it take to implement them? MTTI tracks this timeline because none of it matters if you don’t fix the holes that led to the breach.

Nine Dominos

There are the nine dominos of SOC / IR metrics I recommend helping organizations know if they are on the right track to reduce risk, costs and improve morale / retention of the security teams. You may not wish to track all nine, but understanding how each metric impacts the others can provide visibility into why you are not seeing expected improvements when you implement a new security solution or change processes.

Improving prevention and reducing false positives can make huge positive impacts on your incident response timeline. Utilizing solutions that get you to resolution quicker allows the team to focus on recommendations and risk reduction strategies.

Whichever metrics you choose to track, just be sure the dominos fall in your favor.

‍

References

[1] 2024 Verizon Data Breach Investigations Report, p83

[2] Mandiant M-Trends 2023

‍

‍

‍

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
John Bradshaw
Sr. Director, Technical Marketing

More in this series

No items found.

Blog

/

AI

/

November 18, 2026

Agentic attacks in the wild: Inside three ARTEX investigations

Default blog imageDefault blog image

Key insights

·       Darktrace observed activity in several customer environments involving infrastructure and behaviors associated with reported ARTEX-enabled campaigns, including activity linked to China-nexus espionage and financially motivated attacks.

·       In the higher-confidence cases, known infrastructure appeared alongside beaconing, unusual TLS, reconnaissance, tunneling, script retrieval, SMB activity and external data transfer. One Azure case had no observed network command and control at all, only key listing and application identity activity.

·       ARTEX is now used by actors with different objectives. Identifying the framework can help locate relevant activity, while the behavior around the affected system provides the stronger evidence for understanding compromise and attribution.

‍

Introduction

Attackers have moved beyond using AI for phishing and are running agentic frameworks; open-source tools like ARTEX, CAIRN, PentAGI and Hexstrike that let AI agents scan, exploit and move through an environment with little human input [1]. ARTEX has moved quickly from an offensive security project into live intrusion activity. Since at least August 2026, the framework has been associated with espionage, financially motivated attacks and opportunistic campaigns across government, financial services, technology, defense, telecommunications and other sectors. The FBI has documented ARTEX activity spanning reconnaissance, exploitation, credential abuse, lateral movement, collection, command and control, and exfiltration [2].

Darktrace has observed activity in customer environments that overlaps with infrastructure, tooling and behaviors reported in several of these campaigns. The cases below highlight how those intrusions appeared on the network, including the activity visible before an analyst can determine which actor or framework may be responsible.

ARTEX as part of the attacker workflow

ARTEX is an open-source, multi-agent offensive security platform that converts high-level objectives into actions carried out by specialized agents [2]. Operators have used it to automate activity across the attack lifecycle, including scanning, exploitation, credential activity and post-compromise operations. Some reported intrusions moved from initial targeting to privileged access in under three hours [2].

The framework is now associated with actors pursuing different objectives and using different post-exploitation tooling. The original public repository was removed in October 2026, but archived copies and localized versions remain available. For defenders, ARTEX is more useful as context for an investigation than as an attribution shortcut.

Darktrace observations show the same point at network level. Infrastructure linked to reported ARTEX campaigns appeared alongside beaconing, tunneling, unusual TLS, reconnaissance, file retrieval and lateral activity.

Case 1: Espionage-linked activity

Darktrace observed activity associated with BLOODALCHEMY command-and-control (C2) infrastructure in two customer environments. In one environment, five devices communicated with nginxmysql[.]com and 45.153.126[.]223 between October 23 and November 2. Both indicators had previously been linked to China-nexus espionage activity involving ARTEX [4].

The connections were accompanied by suspicious beaconing, repeated successful connections and unusual self-signed TLS. One system began making suspicious TLS connections to nginxmysql[.]com on October 23. Beaconing followed on October 24, and SMB scanning appeared later in the sequence. Other affected systems subsequently communicated with previously unseen external endpoints.

Prior to the observed C2, Darktrace detected curl requests retrieving executable and script content including /systemd, /wazuh, linux_amd64, u.sh and q.sh. Related telemetry included DRSGetNCChanges activity consistent with DCSync, LDAP enumeration, DNS zone transfers, network scanning, certificate downloads and SMB writes involving start.bat.

‍

Figure 1: Darktrace investigation showing suspicious network activity associated with infrastructure linked to reported ARTEX activity.

‍

Figure 2: Darktrace detection showing anomalous self-signed TLS activity during the espionage-linked investigation.

‍

BLOODALCHEMY has been reported in more than one China-nexus cluster [5]. SoftEther infrastructure observed in one environment provided another point of correlation with reported VAPOR PANDA tradecraft. The moderate-confidence assessment tying this to a Chinese nexus threat actor is based on infrastructure, victim context, tradecraft behavior that followed the initial connection, and public reporting.

Case 2: Financial-sector activity

In early November, Darktrace observed activity in two financial-sector environments that overlap with reported ARTEX infrastructure. On November 5, a server connected to 213.111.150[.]145:8099 and downloaded wisp-agent, associated with WISP C2 activity.

In a second environment, a device attempted connections to 85.11.167[.]52 on November 1 and 2. The connections occurred alongside beaconing to new and rare destinations, anomalous TLS, possible tunneling and unusual package downloads.

The device also communicated with temp[.]sh, ssh.localhost[.]run, localhost[.]run, termbin[.]com, pastebin[.]com and GitHub-hosted release infrastructure. The sequence showed potential tooling retrieval, C2, tunneling and use of external services capable of staging or moving data.

Figure 3: Darktrace investigation showing connections to new and rare external destinations during the financial-sector activity.

‍

Case 3: Credential collection in Azure

On October 20, Darktrace observed a sweep of credential collection in a customer’s Azure environment that is consistent with ARTEX use. The intrusion began with connections to the virtual network from 43.98.196[.]73 (Alibaba Cloud). Open-source scanning data associates the address with ARTEX.

A compromised user identity made requests to the Azure Resource Manager API, though how it was compromised is unknown. The requests listed keys and secrets across API Management, web and function apps, Cognitive Services, Event Hubs, storage accounts and a managed Kubernetes cluster. With those credentials the attacker compromised multiple application identities. The attacker then used the application identities and the user identity to retrieve more credentials, change firewall and security rules and run scripts on virtual machines. Microsoft Defender alerts showed attempted execution of scripts related to Mimikatz and Impacket.

In a cloud environment, such privileged and enumerative operations are potentially indistinguishable from legitimate administrative actions. Consequently, the sequence and context of such operations signified more than any specific API call. There was no C2 traffic to match, and listing keys and using application identities are also what administrators and automation do. A compromised user, credential retrieval across several Azure services, new application identities, more credential collection and changes to security controls only made sense together, and by then access was spreading through the cloud environment.

‍

Figure 4: Darktrace Advanced Search showing Azure AD operation names associated with the activity.

‍

Figure 5: Darktrace Advanced Search showing the Azure Resource Manager action used to execute scripts.
Figure 6: Darktrace Device Event log window showing the triggered models related to the suspicious Azure activity.

‍

Shared tooling weakens attribution signals

The three cases looked different on the network: BLOODALCHEMY infrastructure and espionage-consistent tradecraft, WISP infrastructure in financial-sector systems, and Azure identity and control-plane activity with no C2 at all.

Shared tooling has complicated attribution for years, and ARTEX adds to it because the framework carries out more of the attack, so reconnaissance, exploitation attempts and some post-compromise activity can look the same whoever set the objective. Attribution still needs infrastructure, malware, victimology, timing and the sequence of behavior inside the environment, and an ARTEX link is one input to that. Moreover, the behavior and the agent will still be shaped by the biases, preferences, and tradecraft of its operators.

At its core, ARTEX’s planner and worker agents identify and record facts, while exploring intents. However, the way the Large Language Model (LLM) chooses its next steps toward an objective is shaped fundamentally by the skills and tools available in its context. This includes its starting context, as well as various possible Model Context Protocol (MCP) plugins and user-injected context (e.g. user input mid-operation). Operators still have preferred tradecraft, and this preference shapes the pathways the harness will provide to the LLM in context prompts. Even on the same platform, customization based on operator preferences and goals will remain a key facet of campaign and actor identification.

Detection and response

Across the Darktrace cases, the stronger signals came from behavior after the connection: beaconing, abnormal TLS, reconnaissance, unusual authentication, tunneling, script retrieval, SMB activity and outbound transfer. Automated activity also produced compressed bursts of requests, sequential enumeration and repeated attempts with small changes.

With agentic tools reaching privileged access in under three hours, analysts may have only a short window to connect scanning or authentication activity to credential changes, new administrative access, tooling retrieval or lateral movement. Keeping network, identity and cloud telemetry together shows what happened before and after the first match, and whether a connection was noise or an intrusion moving toward privileged access.

ARTEX won’t be the last of these frameworks. The next ones will come with new capabilities and defaults, but an intrusion still must interact with the target environment, and that shows up in how systems, identities and applications behave even after the indicator has aged out. The harder intrusions to catch will have no infrastructure to match, just ordinary API calls and legitimate identities. Detecting those depends on knowing what normal looks like for each identity and device.

‍

Credit to Nathaniel Jones (SVP, Global Threat Intelligence), Eugene Chua (Principal Analyst), Sam Lister (Senior Security Researcher), Adam Potter (Associate Principal Cyber Analyst)

Appendices

Darktrace model detections

·       Anomalous Connection / Multiple HTTP POSTs to Rare Hostname

·       Anomalous Connection / Rare External SSL Self-Signed

·       Anomalous Connection / Suspicious Self-Signed SSL

·       Compromise / Large Number of Suspicious Successful Connections

·       Compromise / Suspicious Beaconing Behaviour

·       Compromise / Agent Beacon to New Endpoint

·       Compromise / Beacon to Young Endpoint

·       Compromise / SSL Beaconing to Rare Destination

·       Compromise / Suspicious TLS Beaconing To Rare External

·       Compromise / Possible Tunnelling to Bin Services

·       Device / Anomalous Github Download

·       Device / Anomalous Package Download

·       Device / Initial Attack Chain Activity

Indicators of compromise (IoCs)

nginxmysql[.]com

45.153.126[.]223

213.111.150[.]145

wisp-agent

85.11.167[.]52

43.98.196[.]73

192.121.47[.]20

www[.]ddduuuppp[.]com

89.46.234[.]50

sqlserver-sqlite[.]com

199.19.95[.]66

graphics-hook-filter64.dll

graphics-hook_config.bin

References

1.      Google. GTIG AI Threat Tracker - https://cloud.google.com/blog/topics/threat-intelligence/agentic-threat-activity-crosses-a-significant-threshold

2.      Federal Bureau of Investigation, Threat Overview of ARTEX Autonomous Penetration Testing Framework, FLASH-20261113-001, 13 November 2026: https://www.ic3.gov/CSA/2026/261113.pdf

3.      Microsoft Security Blog, Storm-0940 Uses ARTEX Agentic AI in ConnectWise Supply Chain Compromise: https://www.microsoft.com/en-us/security/blog/2026/10/21/storm-0940-uses-artex-agentic-ai-connectwise-supply-chain-compromise/

4.      CrowdStrike, VAPOR PANDA Uses ARTEX to Scale Global Campaign: https://www.crowdstrike.com/en-us/blog/vapor-panda-uses-artex-to-scale-global-campaign/

5.      Bitdefender, SilkParasite: Tracking a China-Nexus APT Across Central Asia: https://www.bitdefender.com/en-us/blog/businessinsights/silkparasite-tracking-china-nexus-apt-across-central-asia

‍

‍

Continue reading
About the author
Nathaniel Jones
SVP, Global Threat Intelligence

Blog

/

AI

/

November 18, 2026

Darktrace qualifies for NATO’s Cybersecurity Dynamic Marketplace

Default blog imageDefault blog image

Darktrace has been selected to join NATO’s Cybersecurity Dynamic Marketplace (CSDM), an initiative designed to give NATO faster, more flexible access to cybersecurity technology and services in support of its missions.  

Established by the NATO Communications and Information Agency (NCIA), the marketplace brings together qualified suppliers whose technical capabilities have been reviewed, helping NATO match specialist expertise to project needs and respond more quickly to changing operational priorities.

Qualified across all eight cybersecurity categories

The program’s first phase, known as Lot 1, covers eight cybersecurity categories. Suppliers can qualify in individual categories; Darktrace has qualified to compete for future work across all eight:

  • Identity, Credential and Access Management
  • Endpoint Protection and Platforms
  • Network and Perimeter Security
  • Cloud Security
  • Threat Detection and Analytics Platforms
  • Vulnerability and Exposure Management
  • Digital Forensics and Evidence Platforms
  • Professional Services

Darktrace’s qualification across all eight categories reflects the breadth of our technology and services.

Supporting government and defense organizations

Nation-state attacks, compromised suppliers, the rapid exploitation of security weaknesses and attackers’ growing use of AI are increasing pressure on public-sector organizations. At the same time, agencies are adopting cloud services, connecting the technology that runs physical operations, and introducing automation and AI. Security teams must manage these changes while keeping essential operations running and sensitive information secure.

Protecting these environments requires an understanding of how people, AI and connected systems interact, and the ability to recognize when their activity creates risk. Threats can span several systems, making it essential for security teams to connect related activity and respond quickly.

The Darktrace Behavioral Defense Platform™ supports this approach through unified visibility, continuous behavioral monitoring and autonomous response across AI, people and infrastructure.

At its core, Darktrace’s Adaptive AI™ continuously learns how each organization behaves, building a unique understanding of its people, AI agents and connected systems. This enables the platform to help identify activity that does not belong, connect related behavior across the organization, and investigate and respond to threats autonomously in real time, helping prevent disruption to essential operations.

Our selection builds on work to meet government security requirements internationally. Darktrace recently completed an independent assessment under Australia’s InfoSec Registered Assessors Program (IRAP) for Darktrace / NETWORK™ and Darktrace / OT™ against the Australian Government’s Information Security Manual PROTECTED-level controls. In the United States, Darktrace Federal continues to deliver the Darktrace technology to Department of War, Department of State, and Federal civilian organizations via its on-premises and FedRAMP CLASS D (High) Authorization platforms.

Together, these achievements demonstrate Darktrace’s ability to meet rigorous public-sector security requirements and reinforce its position as a trusted provider of mission-critical cybersecurity technology for government environments.

Meet us at NATO Edge 2026

Darktrace representatives will attend NATO Edge 2026, NCIA’s flagship technology event, in İzmir, Türkiye, from 17 to 19 November. We look forward to meeting government, military, and industry partners to discuss their cybersecurity priorities and how we can support them.

To learn more about the Darktrace Behavioral Defense Platform, visit the platform hub.

Continue reading
About the author
The Darktrace Community
Your data. Our AI.
Elevate your network security with Darktrace AI