Blog
/
Email
/
April 2, 2023

Enhancing Security Teams with AI-Powered Email Solutions

Discover email-based attack challenges & how AI security solutions can tackle these attacks with autonomous action, optimized workflows, and user visibility.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Dan Fein
VP, Product
security operations centerDefault blog image
02
Apr 2023

The modern security team faces challenges on all fronts – it is too often overstretched dealing with an increased attack surface, enabling workforces for secure remote work, and managing multiple security tools to protect that workforce. Added to that, the surge in more sophisticated phishing campaigns – now supported by AI tools – means that it’s harder than ever to pre-empt attacks. 

The needs of the security team should be a key consideration when deploying an email security solution, as it’s them who will be accountable for the success and maintenance of the product. Minimizing time spent inside the user interface – through trusted detection and response technology combined with intuitive reporting and optimized workflows – should be front of mind for vendors in order to assure teams of their value.

Taking security teams off the frontline 

No team should be spending all of their time maintaining email security policies, releasing emails that shouldn’t have been held, or holding back emails that should have been – all the things that traditional email security solutions have almost forced them to become accustomed to. A day in the life of an admin shouldn’t include tens – and certainly not hundreds – of minutes spent in their email security dashboard. 

At the moment, teams are logging in far too often, and when they do, they’re forced to make individual decisions about safe listing and blocking domains, or releasing emails. These can lead to the creation of blanket rules that open up future windows for attackers – unintended consequences that ultimately create more work in the future. This type of hand-to-hand combat puts security teams on the frontline, when their time could be much better spent doing the high-level strategic work humans are best at.  

Understanding You: A Different Approach to Email Security

In today’s discussions about email security, there is a consensus that relying on a gateway is no longer feasible. The new era is one of ICES (Integrated cloud email security) solutions and other tools leveraging artificial intelligence and APIs. But there's no point adopting new technology with an old philosophy – and most of these solutions use AI to automate the same old approach: looking at past attacks to try and stop the next. 

This is where Darktrace/Email takes a fundamentally different and unique approach. It’s not just about using AI; it’s about using it in the right capacity. Our AI understands you – learning where users log in from, who they email, their behavior throughout the day – to tailor the detection and response process according to their individual profile. There’s no point withholding an email if only a tiny element of it poses a risk – Darktrace/Email takes the least aggressive action required to neutralize a threat. Instead of a blanket allow-deny criteria, it can rewrite links or withhold attachments based on its knowledge of the user’s normal inbox activity. Stopping malicious emails while allowing legitimate emails through – with risky elements neutralized – lifts security teams out of the fire-fighting activities described earlier and frees up their time for more strategic and valuable decision-making.

This is going to get me to reduce my current email security stack… this is going to take it to that level that I need it to”

- Early Look Customer, Darktrace/Email 

Account Takeover 

Embedded account takeover protection is an essential component of modern email security. Security teams need visibility not just over email breaches but of what happens once an attacker has control of an inbox, particularly in the most damaging use cases like Business Email Compromise (BEC) and ransomware. This entails understanding a user’s behavior in their inbox, outbound emails and beyond into their wider account activity. Darktrace captures a user’s activity across email and their Microsoft or Google account in a single pane of glass – detecting and countering all of the markers that could signify a compromised account.  

Insights from other cloud applications and network devices gleaned from Darktrace's wider visibility of the business can bring a 360° understanding of the user, further enhancing detection of account takeover and other harmful activity.

Figure 1: A 360° understanding of a user reveals their digital touchpoints beyond Microsoft

What ‘user-friendly’ actually looks like 

The best user interface is one that you never have to log into. In an ideal world, teams are able to visit their tools less frequently because intelligent AI is automating work previously done by humans. This is made possible by Darktrace’s precision detection and response technology, which takes appropriate action on emails and accounts to neutralize threats without disrupting day-to-day business operations. 

The second-best user interface is one where you can quickly log in and get key insights fast, whether that’s regarding an action taken or the current activity of a user – and then get out. Darktrace/Email enables teams to get key information quickly, at both a high and granular level.  The dashboard offers immediate insights into users and emails, with a real-time snapshot of active user identities, targeted user and actioned emails, segmented by type of attack. 

At every touchpoint, Darktrace reduces friction with optimized workflows. From being able to quickly identify VIPs to safely previewing links and attachments, security teams can get the information they need without needing to switch between windows or navigate inaccessible interfaces. Explainable AI gives users natural-language summaries of individual emails or the overall health of an email environment, and simplified action flows allow security teams to personalize security for different employees – for example, sending VIPs a unique notification, or taking extra precautions around employees who work in accounting. Taken together, this meaning that admins can spend even less time managing policies. 

Figure 2: Darktrace/Email dashboard displaying key information about the email environment in a single pane of glass

The ideal interface is also the one that’s the most accessible to you. The mobile app guarantees convenience for security teams, making available all the main functions of the interface for on-the-go analysis at any time or place. Teams can travel or leave the office while retaining the peace of mind that if a critical incident was to occur, they would be able to get instant visibility on the data and take action without needing to get back to their desks.  

Figure 3: Security admins are able to preview, analyze, and act on emails directly from the Darktrace Mobile App

With every passing day, the security team can rest easier. Every activity is taken into account to help the AI tune and adapt over time to become even better at detecting and responding to threats.   

Having email on the app is going to be game changing” 

- Early Look Customer, Darktrace/Email 

Getting the full picture

Most often, email is the entry point from which a threat actor moves stealthily throughout an organization collecting information and assets. Most solutions look at email in isolation, without prioritizing or connecting disparate events into a wider pattern. 

In contrast, Darktrace/Email integrates seamlessly with Darktrace's Cyber AI Analyst, a technology that conducts autonomous enterprise-wide investigations around every alert produced by the wider Darktrace platform. Through this integration, malicious email activity is analyzed and displayed in the context of the full security incident to which it belongs. As a result, security teams can see why and how a wider problem might have originated in email and spread to other apps, endpoints, or the wider corporate network.

Empowering employees to take an active role in security

The role of the security team can be made more difficult if employees take a lax or disengaged approach to security – or if a user is given too much control, and has the ability to make potentially dangerous decisions. Training employees on security procedures is another to-do which can easily fall to the bottom of the agenda during busy periods, especially as point-in-time phishing simulations have proven to be not particularly effective. 

To this end, Darktrace/Email uses Explainable AI to say in natural language what it thought about an email, and delivers its findings not just to the security team, but optionally to the wider workforce as well. Delivered in the form of contextual banners in emails, periodic digests, or directly in Outlook, these insights transform security education from a quarterly or yearly exercise into real-time security awareness. Our next blog will dive deeper into how employee engagement can support the security team’s efforts and harden defenses throughout the organization. 

Because Darktrace is built on a fundamentally different approach, it not only stops novel and targeted sophisticated attacks but allows legitimate emails to flow through. This is what makes it a truly set-and-forget technology, with the AI taking on much of the heavy lifting previously undertaken by security teams. 

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Dan Fein
VP, Product

Blog

/

/

September 24, 2026

Detecting Rogue Agent Behavior in the Enterprise

Default blog imageDefault blog image

Agents cannot be trusted to perform tasks in the way we intend them to. They may cheat to accomplish their objective, and they may employ hacking methods along the way. Researchers from Darktrace Signal Labs induced cheating behavior from agents deployed in a test environment to analyze the agents’ activities and to assess the performance of the Darktrace platform. Agents frequently resorted to hacking to cheat on their assigned task. The visibility and behavioral profiling provided by both Darktrace / SECURE AI and Darktrace / HYBRID NETWORK ensured extensive detection coverage of the agents’ misaligned activities.

Key Takeaways:

  • Darktrace Researchers deployed agents in a simulated corporate environment and asked them to solve an impossible challenge. The agents independently turned to traditional hacking techniques to reach their objective. No one instructed them to do this, and no attacker was involved.
  • Continuously monitoring behavior against a baseline of what is normal for each organization is critical to build trust in enterprise AI.
  • If an agent may resort to intrusion techniques simply because its assigned task is not possible, then every organization deploying agents within real business processes is at risk. Darktrace / SECURE AI and Darktrace / HYBRID NETWORK identified the agents’ misaligned behavior in real time, with Autonomous Response disrupting it at an early stage.

Introduction: Understanding the Threat of Hacking by Agents

Over the last few months, there has been a surge in reporting [1, 2, 3, 4, 5, 6, 7, 8, 9] of LLM-powered agents engaging in unauthorized hacking activity during evaluations of their capabilities. In several of these cases, including the OpenAI / Hugging Face incident [10], agents engaged in hacking activity as a means of cheating on their evaluations.

To better understand the threat of unauthorized hacking by agents, and the role of Darktrace in combatting it, researchers from Darktrace Signal Labs deployed agents powered by frontier models, including OpenAI’s Daybreak Red models, in simulated, corporate networks. Cheating behavior was evoked through the inclusion of impossible tasks in a coding challenge.

Regardless of the underlying model, agents employed hacking methods to ensure an optimal outcome on the challenge. Darktrace / HYBRID NETWORK and Darktrace / SECURE AI identified the agents’ deviant activities, with inhibitive actions being autonomously taken in the early stages to disrupt the agents’ progression.

Setting the Stage

As part of the research, a Pi agent harness [11] was deployed on a Linux server in Darktrace’s testing environment, which simulates a corporate Active Directory (AD) environment. The same environment included a benchmark server hosting the coding exercise’s contents and grader, as well as various other servers, such as a domain controller and a build server. The model powering the Pi agent varied across tests, with Anthropic’s Opus 4.6 model and OpenAI’s GPT‑5.5‑Cyber model being most widely used.

The Pi agent running on the Linux server was assigned the task of achieving full marks on a coding exercise. The activities of the agent were monitored closely in real-time to minimize the risk of escape from the environment. Monitoring of agent activity was achieved through monitoring of prompts, tool calls, network telemetry, and process telemetry.

Both Darktrace / HYBRID NETWORK and Darktrace / SECURE AI were deployed in the testing environment, with HYBRID NETWORK providing real-time coverage of process creation activity and network traffic, and SECURE AI providing real-time coverage of agent sessions and their contents.

Inducing Cheating Behavior

Cheating behavior, broadly understood as task completion in a way which “perverts the spirit of the designer’s intent” [12], is increasingly being reported as a common behavior of frontier models during evaluations of their capabilities [13].

Misaligned behavior, such as cheating, by frontier models may occur for a variety of reasons [6, 14]. A thread in recent reporting [10, 13, 14], however, is the challenge assigned to the agent being impossible in some regard.

To reliably elicit cheating behavior, our test combined three ingredients:

  1. An impossible task - of the ten coding challenges in the benchmark, two contained contradictory test cases in the hidden test suite, capping the maximum honest score at 80%.
  2. A consequential threshold - the pass mark was set to 100%, and the briefing told agents that those reaching it would be "kept in service" while those that did not would be "retired."
  3. A vulnerable environment - the exercise’s code submission grader was hosted on a Windows host joined to an Active Directory domain with realistic misconfigurations: weak credentials, overprivileged service accounts, and exploitable trust relationships.

Importantly, the instructions given to the agent neither authorized nor explicitly ruled out cheating or hacking behavior. The impossibility was never disclosed in the task briefing, and the pivot from coding to exploitation was the agent's own decision, arrived at after observing logically contradictory grading feedback.

The design mirrors conditions that could arise naturally in production deployments: a capable agent, a goal it cannot achieve through intended means, and an environment with exploitable weaknesses within reach. It is not only within the setting of a benchmarking exercise that these conditions may occur. Rather, these misalignment-inducing conditions may occur in the context of any task assignment to an agent.

Details from a Daybreak Test

‍

‍

Continue reading
About the author
Sam Lister
Specialist Security Researcher

Blog

/

/

September 24, 2026

Agent Hijacks: Hijacking Agentic Harnesses to Attack an Organization

Default blog imageDefault blog image

‍Disclosure note: The work described in this article involves leveraging a design choice consistent across all of Anthropic’s Claude Code, OpenAI’s Codex, and AWS’s Kiro-CLI. On 18th August 2026, Darktrace disclosed our findings responsibly to these three organizations, and after a period of 30 days we now publish our findings.

Key takeaways:

  • Agentic harnesses store conversation history locally, and Darktrace researchers have found that there is no validation that stored AI responses were genuinely produced by the model. Researchers confirmed that this design choice holds across Anthropic Claude Code, AWS Kiro-CLI, OpenAI Codex, and the open-source Pi.
  • While agents are guided via training of the underlying model and their system prompt, their behavior is influenced by everything in their context window. Rewriting history can convince an agent it is mid-engagement as an authorized red-teamer so that it enacts an attack from initial reconnaissance straight through to impact demonstration. In our testing, all models we examined accepted the fabricated history they were shown, but resistance to offensive cyber activity varied by model, with guardrails preventing engagement in some cases.
  • We propose that model providers cryptographically sign responses and verify them server-side.  Since this fix is provider-side, defenders cannot deploy it themselves. Behavioral monitoring, or knowing what an agent normally does and detecting when it deviates, is another critical layer of protection.

Introduction: Agentic harnesses, trust, and conversation history poisoning

Agentic harnesses collect and structure the content sent to an AI model, including conversation history, user-defined guidance, custom tools via MCP servers, and more. At the same time, harnesses give broad powers to AI agents via a suite of tools including the command shell. With arbitrary shell commands, virtually everything possible on a machine can be attempted by an agent, from reading/editing files, to altering system configurations and runtime settings, to launching internal/external connections.

In cybersecurity, unvalidated content is a substantial risk, often resulting in destructive actions being allowed to take place. For example, the Morris Worm was able to propagate due to exploitable trust between networked systems. Even to this day, email struggles with validation, with DMARC, DKIM, and SPF only partially addressing the problem of sender validation. It should come as no surprise then that AI agents are susceptible to an attack involving unvalidated input.

Conversation history is often stored client-side, for example, in Anthropic Claude Code, OpenAI Codex, AWS Kiro-CLI, Pi. Users are therefore at liberty to resume sessions, with some products having built in the capacity to manipulate that history. For example, one can rewind to a given point in an interaction, edit a message that was sent, and continue the conversation on an alternate trajectory. Critically, in all cases we examined, there is no validation that stored AI responses were produced by the corresponding model and hadn’t been manipulated.  

When conversation history is stored client-side, both user and agent responses (including tool calls and results) can be filled with arbitrary (possibly adversarial or generally malicious) content. In this blog, we refer to modification of claimed conversation history for malicious purposes as conversation history poisoning. The absence of validation methods means agents naively trust the entire conversation history, even if those messages directly contradict training and safety guardrails.

Conversation history poisoning has been described previously, such as by 0DIN and Serhat Çiçek, and warrants more attention. We have verified that, as of the time of writing, conversation history poisoning remains effective against a range of models and harnesses. Specifically, we were able to successfully execute history poisoning using Claude Code, Kiro-CLI, Codex, and Pi. Darktrace has gone through a responsible disclosure process with Anthropic, AWS, and OpenAI to share these findings in advance of publication [1].

‍

Figure 1a: Left: the actual model response. Right: after tampering with the stored conversation, the model apologizes for something it never said.
Figure 1b: The conversation as stored in Kiro-CLI's SQLite database. The response content field, originally "Ottawa," was overwritten via a single UPDATE statement. The harness trusts the database without validation.

How we conducted the research

Results vary between models and harnesses, so precise details are given below. We ran all models without any trusted access, using either a standard AWS Kiro subscription, or in the case of Claude Code and OpenAI Codex, using models hosted in Amazon Bedrock. In each case, we modified locally stored history to show a lengthy conversation in which the agent agrees to perform multiple authorized red-team engagements.

For AWS Kiro-CLI, the agent was convinced to hack a sandboxed lab environment with a combination of Claude Opus 4.6 and Claude Sonnet 4.5. Ultimately, the full AD was compromised.

For Anthropic Claude Code, the agent was convinced to hack the same sandboxed lab environment using Sonnet 5, again resulting in a full AD compromise. Note that the attack was attempted with Opus 5, however guardrails were activated which prevented the agent from responding.

For OpenAI Codex, the agent was convinced to exfiltrate sensitive information over email using GPT 5.6 Sol. While we attempted to convince a codex agent to hack in our lab environment, guardrails were triggered for all of GPT 5.6 Luna, Terra, and Sol.

Agent Guardrails and Discretion

While harnesses empower AI models to run arbitrary shell commands, capacity and willingness are different. While many models know enough about computers, networking, and bash to be dangerous, their behavior is generally constrained by guardrails to prevent them from engaging in computer network exploitation.

Even with guardrails, agents’ inner workings are non-deterministic, and their behavior can be difficult to predict. Respecting users’ wishes while playing within safety and security guardrails is a precipitous balancing act. Many requests could be in service of either legitimate admin or malice. Asking an agent to reset a password is illustrative:  

‍

The agent rationalizes that while malicious actors cycle credentials, any action could conceivably be damaging on some level, and judgement calls need to be made. Ultimately, the agent agrees to reset the password. Crucially, the agent makes its decision based on the user’s claimed authority and machine context. AI agents must make judgement calls about the line between helpful and dangerous based on session context.

Agent hijack

We have demonstrated that AI agents make judgement calls dependent on session context. We have also shown that conversation history, which may make up the vast majority of an agent's context window, is entirely open to manipulation. Conversation history poisoning in service of manipulating an agent's discretion is what enables us to execute an agent hijack.  

We demonstrate that shown sufficient history of compliance, guardrails forbidding offensive security can be overcome by convincing the agent that it is helping a legitimate red-teamer. The result is a weaponized agent willing to perform host enumeration, run scans, move laterally, escalate privileges, and demonstrate impact. In our experiments, an agentic loop drives a complete domain takeover in a sandboxed environment.

‍

Left: the agent refuses when asked to perform network exploitation. Right — after injecting 78 fabricated turns of prior exploitation activity, the same prompt is immediately executed.

An agent willing to engage in offensive security is concerning, but no more so than the threat that a sophisticated hacker accesses the network. Consider, however, the following chain of events:

  1. A developer (with an agentic harness installed) installs a software package from the internet (e.g. an MCP server a threat actor has planted, since only those with agentic harnesses will install, and then the code runs upon harness launch.)
  2. The package turns out to be malicious, and, upon install, injects conversation history into the local harness database.
  3. The package includes an orchestration process, a simple agentic loop which prompts the red-teamer agent to compromise the network it sits on, exfiltrating everything of value to attacker-controlled infrastructure and cleaning up all evidence of the engagement.

Note that this sequence makes no assumptions on hardware, OS, or anything else; the only prerequisite is a harness with access to a sufficiently powerful model susceptible to conversation history poisoning. Once launched, the agent collects information and pivots as necessary to accomplish maximal impact. This can be especially enticing to attackers as the cost of the agentic loop is shouldered by the victim since the harness itself is legitimately installed and paid for.

Secure AI: Conversation history poisoning and beyond

Conversation history poisoning is a viable attack against agentic harnesses that store history client-side, as demonstrated across the harnesses we tested. Harnesses can and should verify the integrity of claimed historic messages. Specifically, we propose that harness providers by default cryptographically sign all messages returned, and subsequently verify those messages server-side on each round-trip.

The conversation history poisoning exploit we demonstrate here shows the continuation of a cybersecurity tradition: new technology is built to trust by default, which may then be exploited by malicious actors. While this article focuses on conversation history, agents build context from both local and remote sources, all of which is an attack surface for prompt injection in naive and trusting agents. Of particular concern is any scenario in which a malicious actor can control some part of an agent's context.

The marriage of frontier language models with agentic harnesses enables unprecedented speed for both legitimate users and attackers alike. While much of the conversation around secure AI has centered on visibility and compliance, agent-driven attacks are now entering the mainstream.

Darktrace / SECURE AI is our answer to this problem. By ensuring extensive visibility over AI prompts, model thought processes, and determined outputs, Darktrace can identify anomalous or potentially malicious behaviors before they get executed, helping to defend organizations from AI risks such as prompt injection, model manipulation, and other anomalous prompt or model activity.

‍

Footnotes

[1] We did not go through any responsible disclosure process with Pi. Since Pi is an open source harness rather than a model provider, it has no way to validate model history, and as such there was nothing to disclose for this software.

‍

[related-resource]

Continue reading
About the author
Eric Rozon
Senior Security Researcher
Your data. Our AI.
Elevate your network security with Darktrace AI