Blog
/
Network
/
March 22, 2024

What are Botnets and How Darktrace Uncovers Them

Learn how Darktrace detected and implemented defense protocols against Socks5Systemz botnet before any threat to intelligence had been published.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Adam Potter
Senior Cyber Analyst
Default blog image
22
Mar 2024

What are botnets?

Although not a recent addition to the threat landscape, botnets persist as a significant concern for organizations, with many threat actors utilizing them for political, strategic, or financial gain. Botnets pose a particularly persistent threat to security teams; even if one compromised device is detected, attackers will likely have infected multiple devices and can continue to operate. Moreover, threat actors are able to easily replace the malware communication channels between infected devices and their command-and-control (C2) servers, making it incredibly difficult to remove the infection.

Botnet example: Socks5Systemz

One example of a botnet recently investigated by the Darktrace Threat Research team is Socks5Systemz. Socks5Systemz is a proxy-for-rent botnet, whereby actors can rent blocks of infected devices to perform proxying services.  Between August and November 2023, Darktrace detected indicators of Socks5Systemz botnet compromise within a cross-industry section of the customer base. Although open-source intelligence (OSINT) research of the botnet only appeared in November 2023, the anomaly-based approach of Darktrace DETECT™ allowed it to identify multiple stages of the network-based activity on affected customer systems well before traditional rules and signatures would have been implemented.

Darktrace’s Cyber AI Analyst™ complemented DETECT’s successful identification of Socks5Systemz activity on customer networks, playing a pivotal role in piecing together the seemingly separate events that comprised the wider compromise. This allowed Darktrace to build a clearer picture of the attack, empowering its customers with full visibility over emerging incidents.

In the customer environments highlighted in this blog, Darktrace RESPOND™ was not configured to operate autonomously. As a result, Socks5Systemz attacks were able to advance through their kill chains until customer security teams acted upon Darktrace’s detections and began their remediation procedures.

What is Socks5Systemz?

The Socks5Systemz botnet is a proxy service where individuals can use infected devices as proxy servers.

These devices act as ‘middlemen’, forwarding connections from malicious actors on to their intended destination. As this additional connectivity conceals the true origin of the connections, threat actors often use botnets to increase their anonymity. Although unauthorized proxy servers on a corporate network may not appear at first glance to be a priority for organizations and their security teams, complicity in proxy botnets could result in reputational damage and significant financial losses.

Since it was first observed in the wild in 2016, the Socks5Systemz botnet has grown steadily, seemingly unnoticed by cyber security professionals, and has infected a reported 10,000 devices worldwide [1]. Cyber security researchers noted a high concentration of compromised devices in India, with lower concentrations of devices infected in the United States, Latin America, Australia and multiple European and African countries [2]. Renting sections of the Socks5Systemz botnet costs between 1 USD and 4,000 USD, with options to increase the threading and time-range of the rentals [2]. Due to the lack of affected devices in Russia, some threat researchers have concluded that the botnet’s operators are likely Russian [2].

Darktrace’s Coverage of Socks5Systemz

The Darktrace Threat Research team conducted investigations into campaign-like activity across the customer base between August and November 2023, where multiple indicators of compromise (IoCs) relating to the Socks5Systemz proxy botnet were observed. Darktrace identified several stages of the attack chain described in static malware analysis by external researchers. Darktrace was also able to uncover additional IoCs and stages of the Socks5Systemz attack chain that had not featured in external threat research.

Delivery and Execution

Prior research on Socks5Systemz notes how the malware is typically delivered via user input, with delivery methods including phishing emails, exploit kits, malicious ads, and trojanized executables downloaded from peer-to-peer (P2P) networks [1].

Threat actors have also used separate malware loaders such as PrivateLoader and Amadey deliver the Socks5Systemz payload. These loaders will drop executable files that are responsible for setting up persistence and injecting the proxy bot into the infected device’s memory [2]. Although evidence of initial payload delivery did not appear during its investigations, Darktrace did discover IoCs relating to PrivateLoader and Amadey on multiple customer networks. Such activity included HTTP POST requests using PHP to rare external IPs and HTTP connections with a referrer header field, indicative of a redirected connection.

However, additional adjacent activity that may suggest initial user execution and was observed during Darktrace’s investigations. For example, an infected device on one deployment made a HTTP GET request to a rare external domain with a “.fun” top-level domain (TLD) for a PDF file. The URI also appears to have contained a client ID. While this download and HTTP request likely corresponded to the gathering and transmission of further telemetry data and infection verification [2], the downloaded PDF file may have represented a malicious payload.

Advanced Search log details highlighting a device infected by Socks5Systemz downloading a suspicious PDF file.
Figure 1: Advanced Search log details highlighting a device infected by Socks5Systemz downloading a suspicious PDF file.

Establishing C2 Communication  

Once the proxy bot has been injected into the device’s memory, the malware attempts to contact servers owned by the botnet’s operators. Across several customer environments, Darktrace identified infected devices attempting to establish connections with such C2 servers. First, affected devices would make repeated HTTP GET requests over port 80 to rare external domains; these endpoints typically had “.ua” and “.ru” TLDs. The majority of these connection attempts were not preceded by a DNS host lookup, suggesting that the domains were already loaded in the device’s cache memory or hardcoded into the code of running processes.

Figure 2: Breach log data connections identifying repeated unusual HTTP connections over port 80 for domains without prior DNS host lookup.

While most initial HTTP GET requests across investigated incidents did not feature DNS host lookups, Darktrace did identify affected devices on a small number of customer environments performing a series of DNS host lookups for seemingly algorithmically generated domains (DGA). These domains feature the same TLDs as those seen in connections without prior DNS host lookups.  

Figure 3: Cyber AI Analyst data indicating a subset of DGAs queried via DNS by infected devices.

These DNS requests follow the activity reported by researchers, where infected devices query a hardcoded DNS server controlled by the threat actor for an DGA domain [2]. However, as the bulk of Darktrace’s investigations presented HTTP requests without a prior DNS host lookup, this activity indicates a significant deviation from the behavior reported by OSINT sources. This could indicate that multiple variations of the Socks5Systemz botnet were circulating at the time of investigation.

Most hostnames observed during this time of investigation follow a specific regular expression format: /[a-z]{7}\.(ua|net|info|com|ru)/ or /[a-z0-9]{15}\.(ua)/. Darktrace also noticed the HTTP GET requests for DGA domains followed a consistent URI pattern: /single.php?c=<STRING>. The requests were also commonly made using the “Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)” user agent over port 80.

This URI pattern observed during Darktrace’s investigations appears to reflect infected devices contacting Socks5Systemz C2 servers to register the system and details of the host, and signal it is ready to receive further instructions [2]. These URIs are encrypted with a RC4 stream cipher and contain information relating to the device’s operating system and architecture, as well as details of the infection.

The HTTP GET requests during this time, which involved devices made to a variety a variety of similar DGA domains, appeared alongside IP addresses that were later identified as Socks5Systemz C2 servers.

Figure 4: Cyber AI Analyst investigation details highlighting HTTP GET activity whereby RC4 encrypted data is sent to proxy C2 domains.

However, not all affected devices observed by Darktrace used DGA domains to transmit RC4 encoded data. Some investigated systems were observed making similar HTTP GET requests over port 80, albeit to the external domain: “bddns[.]cc”, using the aforementioned Mozilla user agent. During these requests, Darktrace identified a consistent URI pattern, similar to that seen in the DGA domain GET requests: /sign/<RC4 cipher text>.  

Darktrace DETECT recognized the rarity of the domains and IPs that were connected to by affected devices, as well as the usage of the new Mozilla user agent.  The HTTP connections, and the corresponding Darktrace DETECT model breaches, parallel the analysis made by external researchers: if the initial DGA DNS requests do not return a valid C2 server, infected devices connect to, and request the IP address of a server from, the above-mentioned domain [2].

Connection to Proxy

After sending host and infection details via HTTP and receiving commands from the C2 server, affected devices were frequently observed initiating activity to join the Sock5Systemz botnet. Infected hosts would first make HTTP GET requests to an IP identified as Socks5Systemz’s proxy checker application, usually sending the URI “proxy-activity.txt” to the domain over the HTTP protocol. This likely represents an additional validation check to confirm that the infected device is ready to join the botnet.

Figure 5: Cyber AI Analyst investigation detailing HTTP GET requests over port 80 to the Socks5Systemz Proxy Checker Application.

Following the final validation checks, devices would then attempt TCP connections to a range of IPs, which have been associated with BackConnect proxy servers, over port 1074. At this point, the device is able to receive commands from actors who login to and operate the corresponding BackConnect server. This BackConnect server will transmit traffic from the user renting the segment of the botnet [2].

Darktrace observed a range of activity associated with this stage of the attack, including the use of new or unusual user agents, connections to suspicious IPs, and other anomalous external connectivity which represented a deviation from affected devices’ expected behavior.

Additional Activities Following Proxy Addition

The Darktrace Threat Research team found evidence of the possible deployment of additional malware strains during their investigation into devices affected by Socks5Systemz. IoCs associated with both the Amadey and PrivateLoader loader malware strains, both of which are known to distribute Socks5Systemz, were also observed on affected devices. Additionally, Darktrace observed multiple infected systems performing cryptocurrency mining operations around the time of the Sock5Systemz compromise, utilizing the MinerGate protocol to conduct login and job functions, as well as making DNS requests for mining pools.

While such behavior would fall outside of the expected activity for Socks5Systemz and cannot be definitively attributed to it, Darktrace did observe devices affected by the botnet performing additional malicious downloads and operations during its investigations.

Conclusion

Ultimately, Darktrace’s anomaly-based approach to threat detection enabled it to effectively identify and alert for malicious Socks5Systemz botnet activity long before external researchers had documented its IoCs and tactics, techniques, and procedures (TTPs).  

In fact, Darktrace not only identified multiple distinct attack phases later outlined in external research but also uncovered deviations from these expected patterns of behavior. By proactively detecting emerging threats through anomaly detection rather than relying on existing threat intelligence, Darktrace is well positioned to detect evolving threats like Socks5Systemz, regardless of what their future iterations might look like.

Faced with the threat of persistent botnets, it is crucial for organizations to detect malicious activity in its early stages before additional devices are compromised, making it increasingly difficult to remediate. Darktrace’s suite of products enables the swift and effective detection of such threats. Moreover, when enabled in autonomous response mode, Darktrace RESPOND is uniquely positioned to take immediate, targeted actions to contain these attacks from the onset.

Credit to Adam Potter, Cyber Security Analyst, Anna Gilbertson, Cyber Security Analyst

Appendices

DETECT Model Breaches

  • Anomalous Connection / Multiple Failed Connections to Rare Endpoint
  • Anomalous Connection / Multiple Connections to New External TCP Port
  • Compromise / Beaconing Activity To External Rare
  • Compromise / DGA Beacon
  • Compromise / Beacon to Young Endpoint
  • Compromise / Slow Beaconing Activity To External Rare
  • Compromise / HTTP Beaconing to Rare Destination
  • Compromise / Quick and Regular Windows HTTP Beaconing
  • Compromise / Agent Beacon (Medium Period)
  • Compromise / Agent Beacon (Long Period)
  • Device / New User Agent
  • Device / New User Agent and New IP

Cyber AI Analyst Incidents

  • Possible HTTP Command and Control
  • Possible HTTP Command and Control to Multiple Endpoints
  • Unusual Repeated Connections
  • Unusual Repeated Connections to Multiple Endpoints
  • Multiple DNS Requests for Algorithmically Generated Domains

Indicators of Compromise

IoC - Type - Description

185.141.63[.]172 - IP Address - Socks5Systemz C2 Endpoint

193.242.211[.]141 - IP Address - Socks5Systemz C2 Endpoint

109.230.199[.]181 - IP Address - Socks5Systemz C2 Endpoint

109.236.88[.]134 - IP Address - Socks5Systemz C2 Endpoint

217.23.5[.]14 - IP Address - Socks5Systemz Proxy Checker App

88.80.148[.]8 - IP Address - Socks5Systemz Backconnect Endpoint

88.80.148[.]219 - IP Address - Socks5Systemz Backconnect Endpoint

185.141.63[.]4 - IP Address - Socks5Systemz Backconnect Endpoint

185.141.63[.]2 - IP Address - Socks5Systemz Backconnect Endpoint

195.154.188[.]211 - IP Address - Socks5Systemz Backconnect Endpoint

91.92.111[.]132 - IP Address - Socks5Systemz Backconnect Endpoint

91.121.30[.]185 - IP Address - Socks5Systemz Backconnect Endpoint

94.23.58[.]173 - IP Address - Socks5Systemz Backconnect Endpoint

37.187.148[.]204 - IP Address - Socks5Systemz Backconnect Endpoint

188.165.192[.]18 - IP Address - Socks5Systemz Backconnect Endpoint

/single.php?c=<RC4 data hex encoded> - URI - Socks5Systemz HTTP GET Request

/sign/<RC4 data hex encoded> - URI - Socks5Systemz HTTP GET Request

/proxy-activity.txt - URI - Socks5Systemz HTTP GET Request

datasheet[.]fun - Hostname - Socks5Systemz C2 Endpoint

bddns[.]cc - Hostname - Socks5Systemz C2 Endpoint

send-monitoring[.]bit - Hostname - Socks5Systemz C2 Endpoint

MITRE ATT&CK Mapping

Command and Control

T1071 - Application Layer Protocol

T1071.001 – Web protocols

T1568 – Dynamic Resolution

T1568.002 – Domain Generation Algorithms

T1132 – Data Encoding

T1132 – Non-Standard Encoding

T1090 – Proxy

T1090.002 – External Proxy

Exfiltration

T1041 – Exfiltration over C2 channel

Impact

T1496 – Resource Hijacking

References

1. https://www.bleepingcomputer.com/news/security/socks5systemz-proxy-service-infects-10-000-systems-worldwide/

2. https://www.bitsight.com/blog/unveiling-socks5systemz-rise-new-proxy-service-privateloader-and-amadey

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Adam Potter
Senior Cyber Analyst

More in this series

No items found.

Blog

/

/

May 28, 2026

From Efficiency to Exposure: How AI Adoption Is Creating Unseen Vulnerabilities on the Factory Floor

AI in manufacturingDefault blog imageDefault blog image

How AI agents impact the manufacturing industry

Security teams and IT personnel across the manufacturing industry are under constant pressure to protect production, maintain uptime, and safeguard critical assets but the rise of AI is bringing huge new opportunities alongside new cyber risks. Across manufacturing, AI is embedded into workflows, decision-making, and increasingly, autonomous AI agents are acting on behalf of employees and systems.  

Agentic systems are powerful because they can act independently, but that same autonomy also creates cyber and operational risk. Agents have extensive permissions and are capable of carrying out complex tasks, making decisions, and interacting with tools or external systems with little to no human intervention.

Unlike traditional AI models that perform predefined tasks, AI agents use advanced techniques to mimic human decision-making processes, dynamically adapting to new challenges, making decision and taking action based on their own judgement. They look like employees operationally but lack judgment, ethics, or fear of consequences like humans do. This means they can be easily manipulated by cybercriminals, and an AI agent embedded across an OT network creates threats that extend well beyond data exposure. For example, at BMW, AI identifies faults in welding processes as they occur. At its Spartanburg plant, AI monitors the weld of 300-400 metal studs onto every SUV frame to detect misplaced or faulty studs and correct them instantly. Corruption of BMW’s AI system could lead to catastrophic quality control errors.

Adopting agentic AI systems across manufacturing raises some concerns across security teams. New data from our State of AI Cybersecurity survey shows that 78% of manufacturing security professionals are worried about employee use of AI agents – their top concern. That’s followed by employee use of generative AI tools like CoPilot and ChatGPT, a worry for 76% of security professionals at manufacturing organizations. As these tools gain more access to business data and processes, and more autonomy within organizations, security teams, who today have minimal visibility of agent activity in their environments, increasingly have sensitive data exposure (a worry for 60%) and accidental policy and regulatory violations (59%) on their minds.

External AI-powered threats are evolving just as quickly

The same capabilities transforming manufacturing are also reshaping cyberattacks.

AI is enabling attackers to automate reconnaissance, refine targeting, and adapt in real time. What once required time and manual effort can now be executed continuously and at scale. Manufacturers are already seeing the impact. According to manufacturing security professionals we surveyed, 76% are already being impacted by AI-powered threats and 90% see AI increasing the success of social engineering attacks.

And the techniques themselves are evolving. Concerns across the manufacturing sector show growing anxiety about the range of AI-powered attack routes, most pressingly of adaptive malware that evolves in real-time – a prospect half (49%) of manufacturing security professionals we surveyed are worried by, a full 9% more than the average across industries. AI adaptive malware is followed by:

  • Automated vulnerability scanning and exploit chaining (48%) which has become even more pressing as Anthropic’s new Mythos AI Model supercharges vulnerability discovery
  • Hyper-personalized phishing campaigns (46%), which remain a mainstay in hackers’ arsenals, and AI has amplified their effectiveness by making phishing emails more convincing and harder to detect.

This is not just an increase in volume, it is a shift toward threats that evolve as they unfold - often faster than static defenses can respond.

Despite rising awareness, many manufacturers are not yet equipped to manage this shift. More than half (51%) say they are not adequately prepared for AI-driven threats, and only 37% have formal policies governing AI deployment.  

Securing AI through visibility, context, and guardrails

Addressing this challenge does not require manufacturers to slow innovation. It requires a different approach to security, one that can operate at the same speed and scale as AI. Three specific priorities are emerging for manufacturers looking to take advantage of the power of AI.

Visibility is foundational.  

Organizations need to understand where AI is being used, what it can access, and how it behaves across both IT and OT environments. Without that, risk cannot be measured or managed. It is no surprise that Darktrace’s research found that 91% of manufacturing security professionals said that they need to understand how AI makes decisions before trusting it. This is even more critical in operational settings where disruption has safety, environmental, financial, and reputational impacts.

Context is what turns visibility into action.  

In environments shaped by AI, normal behavior is constantly shifting. Detecting threats requires a behavioral approach; understanding patterns of life across the organization and identifying subtle deviations in real time – a step change in organizations’ traditional approach to security and risk management.

Guardrails ensure that agency does not become exposure  

As AI systems take on greater responsibility, organizations need clear boundaries around what they can do and when they can act independently. These controls must be embedded into systems themselves, not applied after the fact.  

Securing AI Agents Across Manufacturing IT and OT

The rise of agentic AI is transforming manufacturing - powering next-generation operations while reshaping the security landscape. This is not just an increase in threats, but a shift to autonomous systems, continuously evolving behaviors, and risks moving at machine speed. For organizations trying to grapple with the challenge of enabling AI while managing the risk, visibility, context and guardrails should be foundational.

Darktrace helps manufacturers build secure AI approaches by making those foundations possible. It provides visibility and real-time detection and response to unusual activity across IT and OT environments and allows organizations to understand AI activity from the prompts employees use and the agents they build to how those agents are behaving across the environment. For manufacturers scaling AI, this delivers a foundation for innovation without sacrificing control.

Continue reading
About the author
Oakley Cox
Director of Product

Blog

/

/

May 28, 2026

How to Evaluate AI Vendors: 5 Key categories for AI Adoption

Default blog imageDefault blog image

Understanding the AI buyers’ market

AI adoption has become a central topic of discussion in boardrooms, drawing growing interest from business leaders. Ultimately, organizations hope that an investment in AI technology will have tremendous returns. However, the process of buying an AI solution is not as straight forward as it appears on the surface.  

While business leaders may be eager to improve productivity across their operations, practitioners responsible for evaluating and selecting AI solutions may not always have the visibility or technical understanding needed to make the right decisions for their business. What is typically marketed as a holistic solution to their most critical problems is usually followed by uncertainty when AI tools are finally operationalized in real environments.

This guide is intended to support security leaders who are under growing pressure to adopt AI tools while navigating complex terminology, vendor claims, and increasingly crowded buying cycles. Ultimately, the goal is to help organizations evaluate and adopt AI in a safe, effective, and well-governed way. To support this, we’ve structured the evaluation framework across five key categories:

  1. Governance, safety, and data controls
  1. Data gathering and training
  1. Model and technique choice
  1. Performance and accuracy validation    
  1. Interpretability, adjustability, and transparency    

What buying AI looks like in cybersecurity

While investing in AI can bring immense benefits to your security team, first-time buyers of AI cybersecurity solutions may not know where to start. They will have to determine the type of tool they want, know the options available, and evaluate vendors. Research and understanding are critical to ensure purchases are worth the investment.  

With acceleration in AI adoption, accompanied by the recent boom in agentic AI and autonomous agents, CISOs must look “beneath the hood" of these tools to understand how they work, how they are governed, and to ensure the system is secure and compliant with internal policies.

Challenges in the AI buyers’ marketplace  

The AI security software market is buzzing with hype and flashy promises, which, understandably, needs to be addressed with due diligence. Potential buyers, especially in the cybersecurity space, are hesitant when it comes to allowing AI autonomous capabilities across their workflows, and a lack of vendor transparency can exacerbate those feelings.  

Reinforcing this sentiment, research from this year's Darktrace’s State of AI Cybersecurity report shows where confidence and hesitancy emerge amongst potential buyers. On the one hand, security professionals agree that they have good visibility into the logic and reasoning processes their AI solutions use. However, they lack the explainability and trust to allow AI to take independent remedial action.

  • 89% say they have good visibility into the reasoning behind the outputs generated by AI solutions
  • 92% say they need to understand how a defensive AI tool makes decisions before they can trust it
  • Only 14% say they allow AI to act independently, performing autonomous actions without human approval
  • 74% say they are limiting the autonomy of AI taking action in their SOC until explainability improves

Given the desire for trust and explainability we are seeing from buyers, it's important for them to be equipped with the right questions to ask vendors during an assessment or POV of AI tools in order to demystify marketing hype from real operational outcomes.

Below is a list of categories in which buyers can assess AI vendors or AI Service Providers (AISPs) to help reach safe adoption and maximize their ROI.  

5 categories of AI vendor assessment

Darktrace groups these AI-related questions into 5 categories: governance, data and training, model and technique choice, performance validation, and interpretability and adjustability. By asking questions regarding each of these 5 categories, buyers can gain a deeper understanding of how an AISP’s systems work and whether they suit their business requirements.

Governance, safety, and data controls

Governance of AI systems is critical for all AISPs. Whether their platform is based around a single model, or is a more complex, composite AI solution, strong governance is essential to ensure the system is safe, robust, and reliable.

A simple question you could ask is:

What AI governance policies and frameworks do you follow, and/or certifications do you currently maintain?

For more questions you can ask vendors, download the full guide here.

Darktrace is certified to the ISO/IEC 42001 standard, the world’s first AI Management System (AIMS) standard. ISO/IEC 42001 addresses the unique ethical and technical challenges AI poses by setting out a structured way to manage risks such as transparency, accuracy, and misuse. This includes a commitment to ethical AI development, and effective management and monitoring of AI systems both prior to and continually after release.

Data gathering and training

Accurate, meaningful, and unbiased data gathering is the first important step in producing any AI system. An AI model trained using inaccurate, unbalanced, or poor-quality training data will fail to perform optimally.

To alleviate concerns regarding training data quality, a question you could ask is:

What steps do you take to prevent bias in your AI models and training data?

For more questions, download the full guide here.

AISPs should be able to provide information about the steps taken, workflows followed, and auditing performed to reduce AI bias where appropriate. While it’s sometimes impossible to fully remove bias from an AI model, appropriate actions should be taken to mitigate or reduce bias where relevant.

Model and technique choice

Different AI techniques are optimal for different tasks. For example, research from Gartner suggests that relying on a single “one-size-fits-all" model can lead to data gaps, especially in highly specialized domains.

To achieve more accurate and robust AI solutions, AI leaders should move beyond using just one model or technique, embrace composite AI practices, and adopt a holistic AI system perspective.

A straightforward question you could ask is simply:

What type(s) of AI model(s) do you utilize in your solution?

For more questions, download the full guide here.

While specific detailed information about custom systems used by AISPs is likely proprietary, buyers should expect vendors to be able to provide an overview of the broad techniques used. This will allow you as a buyer to determine if the type of model is appropriate for your use case.

Performance and accuracy validation  

Testing and evaluation of performance is essential for all AI systems. Performance analysis should be performed both before release and continually after release to identify potential data or model drift.  

A question you could ask to understand an AISPs testing workflow is:

How do you audit, test, evaluate, verify, and validate your AI model outputs?

For more questions, download the full guide here.

Testing workflows will likely vary depending on the type of model – measurements relevant to one system may not always be relevant to others. Assessment of systems should also extend beyond these standard accuracy and robustness tests, and should also feature physical performance, such as latency and resource consumption.  

Interpretability, adjustability, and transparency  

AI systems are typically a black box, simply providing an output without an explanation of how that output was attained. Interpretability and transparency are critical to ensure that both SOC teams and end-users trust the outputs of a system to be accurate and meaningful.

A question you could ask is:

How do you promote a trust relationship between human analysts and AI outputs?

For more questions, download the full guide here.

In the context of cybersecurity, trust and interpretability are even more essential. This is particularly relevant for generative AI-based systems (including most AI Agents), where the risk of hallucination can reduce trust in responses.

Cybersecurity systems often need to perform autonomous actions to block incoming threats – an email filtering system may hold potentially dangerous emails; a firewall may block malicious inbound connections. If SOC teams can’t trust these systems to perform accurately, these systems may be limited or disabled, critically reducing their defensive power.

Darktrace as an AI-native cybersecurity vendor

Darktrace has been building and applying AI in cybersecurity for over a decade, developing its capabilities alongside an increasingly complex and fast‑moving threat landscape. This experience has resulted in a mature, multi-layered approach to AI, which continuously learns the normal patterns of each organization to understand behavior, interpret context, and identify meaningful deviations — without relying on predefined rules or known attack signatures. Over time, this has enabled a proven behavioral understanding that helps uncover subtle signals of risk that may otherwise be missed.

With the backing of our ISO/IEC 42001 certification, stakeholders, customers, and partners can be confident that Darktrace is responsibly, ethically, and safely developing its AI systems, and managing the use of AI in day-to-day operations in a compliant and secure manner.  

Explore the principles behind Darktrace’s responsible AI approach, informed by collaboration with global experts in academia and governments, detailing how accountability, explainability, and continuous validation are built into its cybersecurity technology.

How Darktrace secures AI systems

Darktrace now brings these capabilities to monitor and respond to risk generated from AI systems across organizations with Darktrace / SECURE AI. This solution analyzes how prompts, agents, and systems are used within the context of each organization, bringing every AI interaction into a single view. This unique approach helps teams understand intent, assess risk, protect sensitive data, and enforce policy across both human and AI agent activity.

Stay up to date

Sign up for the Secure AI Readiness Program here: This gives you exclusive access to the latest news on the latest AI threats, updates on emerging approaches shaping AI security, and insights into the latest innovations, including Darktrace’s ongoing work in this area.

Ready to talk with a Darktrace expert on securing AI? Register here to receive practical guidance on the AI risks that matter most to your business, paired with clarity on where to focus first across governance, visibility, risk reduction, and long-term readiness.  

Further Reading on AI in cybersecurity

When deciding to invest in an AI solution, it’s important to understand what this means for you and your organization. The questions presented here are only a starting point in understanding an AI solution and whether it is appropriate for your use case.  

Gain deeper knowledge on applications of AI in cybersecurity and Darktrace’s multi-layered AI in the AI Arsenal White Paper.

[related-resource]

Continue reading
About the author
Jamie Bali
Technical Author (AI) Developer
Your data. Our AI.
Elevate your network security with Darktrace AI