Blog
/
Email
/
April 10, 2023

Detecting Malicious Email Activity & AI Impersonating

Discover how two different phishing attempts from some known and unknown senders used a payroll diversion and credential sealing box link to harm users.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Isabelle Cheong
Cyber Security Analyst
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
10
Apr 2023

Social engineering has become widespread in the cyber threat landscape in recent years, and the near-universal use of social media today has allowed attackers to research and target victims more effectively. Social engineering involves manipulating users to carry out actions such as revealing sensitive information like login credentials or credit card details. It can also lead to user account compromises, causing huge disruption to an organization’s digital estate. 

As people use social media platforms not only for personal reasons, but also for business purposes, attackers gain information they can exploit in social engineering attacks. For example, a threat actor may attempt to impersonate a known individual or legitimate service to take advantage of a user’s established trust. This is a highly successful method of social engineering because mimicking known contacts makes it difficult for traditional security tools that rely on deny-lists to detect the attack.

In October 2022, Darktrace identified and responded to two separate malicious email campaigns in which threat actors attempted to impersonate known contacts in an effort to compromise customer devices. As it learns the normal behavior of every user in the email system, Darktrace was able to instantly detect these threats and mitigate them autonomously, preventing significant disruption to the customer networks.

Payroll Diversion Fraud Attempt Impersonating a Former Employee 

While a customer in the Canadian energy sector was trialing Darktrace in October 2022, Darktrace/Email™ identified a suspicious email seemingly sent from an employee within the organization. The email was sent to the Senior Director of Human Resources (HR) with a subject line of “Change in payroll Direct Deposit.” The email requested a change in bank account information for an employee. However, Darktrace recognized that the sender was using a free mail address that contained random letters, indicating it may have been algorithmically generated. Since this incident occurred during a trial, Darktrace/Email was not configured to take action. Otherwise, it would have prevented the email from landing in the inbox. In this case though, the email went through, bypassing all other security tools in place.

Although the email was from an unknown sender, the HR director believed the email could have been legitimate as the employee who appeared to be the sender had left the organization seven days prior and no longer had access to their corporate email account. However, after reviewing it in the Darktrace/Email dashboard, the customer grew suspicious and contacted the former employee directly to verify if the request was legitimate. The former employee validated the suspicions by confirming they had sent no such email.

Further investigation by the customer revealed that the former employee had been vocal about their departure on various social media platforms. This gave threat actors valuable information to believably impersonate the former employee and defraud the organization. 

Such attempts to target organizations’ HR departments and divert payroll are common tactics for cyber-criminals and are often identified by Darktrace/Email across the customer base. Darktrace/Email is able to instantly identify the indicators associated with these spoofing attempts and immediately bring them to the attention of the customer’s security team. 

Using Legitimate File Sharing Service to Share a Phishing Link 

On October 7, 2022, a customer in the Singaporean construction sector was targeted by a phishing campaign attempting to impersonate a law firm known to the organization. Almost 200 employees received an email with the subject line “Accepted: Valuation Agreement.” 

Figure 1: Sample of an UI view of the message held showing anomaly indicators, history, association, and validation.

Four days earlier, Darktrace observed communication between another email address associated with the law firm and an employee of the customer. Darktrace/Email noted that it was the first time this correspondent had sent emails to the customer. 

Figure 2: Metrics showing how well the sender’s domain is known within the digital environment.

The emails contained a highly unusual link to a file sharing service, (hxxps://ssvilvensstokes[.]app[.]box[.]com/notes), hidden behind the text “PREVIEW OR PRINT COPY OF DOCUMENT HERE.” Darktrace analysts investigated this event further and found that around 30 similar URLs had been identified as suspicious using OSINT security tools in October 2022, suggesting the customer was not the only target of this phishing campaign.

Figure 3: Preview of the phishing email’s body.
Figure 4: Darktrace’s evaluation of the link contained in the phishing email.

Additional OSINT work revealed that the link directed to a website which appeared to host a PDF file named “Valuation Agreement.” The recipient would then be prompted to follow another link (hulking-citrine-krypton[.]glitch[.]me), again hidden behind the text “OPEN OR ACCESS DOCUMENT HERE” to view the file. Subsequently, the user would be prompted to enter their Microsoft 365 credentials. 

Figure 5: The page displayed when the phishing link was clicked, viewed in a sandbox environment.
Figure 6: Example of a page shown when recipient clicks the second link, accessing “hulking-citrine-krypton[.]glitch[.]me”. 

This page contained the text “This document has been scanned for viruses by Norton Antivirus Security.” This is another example of threat actors’ employing social engineering techniques by impersonating well-known brands, such as established security vendors, to gain the trust of users and increase their likelihood of success.

It is highly probable that a real employee of the law firm had their account hijacked and that a malicious actor was exploiting it to send out these phishing emails en masse as part of a supply chain attack. In such cases, malicious actors rely on their targets’ trust of known contacts to not question departures from their normal conversations. 

Darktrace was able to instantly detect multiple anomalies in these emails, despite the fact that they were seemingly sent by known correspondents. The activity detected automatically triggered model breaches associated with unexpected and visually prominent links. As a result, Darktrace/Email responded by locking the link, stopping users from being able to click it.

Darktrace subsequently identified additional emails from this sender attempting to target other recipients within the company, triggering the model breaches associated with a surge in email sending indicative of a phishing campaign. In response, Darktrace/Email autonomously acted and filed these emails as junk. As more emails were detected across the customer’s environment, the anomaly score of the sender increased and Darktrace ultimately held back over 160 malicious emails, safeguarding recipients from potential account compromise.           

The following Darktrace/Email models were breached throughout the course of this phishing campaign:

  • Unusual/Sender Surge 
  • Unusual/Undisclosed Recipients 
  • Antigena Anomaly 
  • Association/Unlikely Recipient Association 
  • Link/Low Link Association 
  • Link/Visually Prominent Link 
  • Link/Visually Prominent Link Unexpected For Sender 
  • Unusual/New Sender Wide Distribution
  • Unusual/Undisclosed Recipients + New Address Known Domain

Conclusion

Social engineering plays a role in many of the major threats challenging current email cyber security, as attackers can use it to manipulate users into transferring money, revealing credentials, clicking malicious links, and more. 

The above threat stories happened before language generating AI became mainstream with the release of ChatGPT in December 2022. Now, it is even easier for malicious actors to generate sophisticated social engineering emails. By using social media posts as input, social engineering emails written by generative AI can be highly targeted and produced at scale. They often avoid the flags users are trained to look for, like poor grammar and spelling mistakes, and can hide payloads or forgo them entirely.

To mitigate the risk of possible social engineering attempts, it is recommended that organizations implement social media policies that advise employees to be cautious of what they post online and enact procedures to verify if fund transfer requests are legitimate.

Yet these policies are not enough on their own. Darktrace/Email can identify suspicious email traits, whether an email is sent from a known correspondent or an unknown sender. With Self-Learning AI, it knows an organization’s users better than any impersonator could. In this way, Darktrace/Email detects anomalies within emails and neutralizes malicious components at machine-speed, stopping attacks at their earliest stages, before employees fall victim. 

Appendices

List of Indicators of Compromise (IoCs)

Domain:

hxxps://ssvilvensstokes[.]app[.]box[.]com/notes/*?s=* - 1st external link (seen in email)

hxxps://hulking-citrine-krypton[.]glitch[.]me/flk.html - 2nd external link, masked behind “OPEN OR ACCESS DOCUMENT HERE”

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Isabelle Cheong
Cyber Security Analyst

More in this series

No items found.

Blog

/

/

August 12, 2025

ISO/IEC 42001: 2023: A milestone in AI standards at Darktrace  

ISO/IEC 42001 complianceDefault blog imageDefault blog image

Darktrace announces ISO/IEC 42001 accreditation

Darktrace is thrilled to announce that we are one of the first cybersecurity companies to achieve ISO/IEC 42001 accreditation for the responsible management of AI systems. This isn’t just a milestone for us, it’s a sign of where the AI industry is headed. ISO/IEC 42001 is quickly emerging as the global benchmark for separating vendors who truly innovate with AI from those who simply market it.

For customers, it’s more than a badge, it’s assurance that a vendor’s AI is built responsibly, governed with rigor, and backed by the expertise of real AI teams, keeping your data secure while driving meaningful innovation.

This is a critical milestone for Darktrace as we continue to strengthen our offering, mature our governance and compliance frameworks for AI management, expand our research and development capabilities, and further our commitment to the development of responsible AI.  

It cements our commitment to providing secure, trustworthy and proactive cybersecurity solutions that our customers can rely on and complements our existing compliance framework, consisting of certifications for:

  • ISO/IEC 27001:2022 – Information Security Management System
  • ISO/IEC 27018:2019 – Protection of Personally Identifiable Information in Public Cloud Environments
  • Cyber Essentials – A UK Government-backed certification scheme for cybersecurity baselines

What is ISO/IEC 42001:2023?

In response to the unique challenges that AI poses, the International Organization for Standardization (ISO) introduced the ISO/IEC 42001:2023 framework in December 2023 to help organizations providing or utilizing AI-based products or services to demonstrate responsible development and use of AI systems. To achieve the accreditation, organizations are required to establish, implement, maintain, and continually improve their Artificial Intelligence Management System (AIMS).

ISO/IEC 42001:2023 is the first of its kind, providing valuable guidance for this rapidly changing field of technology. It addresses the unique ethical and technical challenges AI poses by setting out a structured way to manage risks such as transparency, accuracy and misuse without losing opportunities. By design, it balances the benefits of innovation against the necessity of a proper governance structure.

Being certified means the organization has met the requirements of the ISO/IEC 42001 standard, is conforming to all applicable regulatory and legislative requirements, and has implemented thorough processes to address AI risks and opportunities.

What is the  ISO/IEC 42001:2023 accreditation process?

Darktrace partnered with BSI over an 11-month period to undertake the accreditation. The process involved developing and implementing a comprehensive AI management system that builds on our existing certified frameworks, addresses the risks and opportunities of using and developing cutting-edge AI systems, underpins our AI objectives and policies, and meets our regulatory and legal compliance requirements.

The AI Management System, which takes in our people, processes, and products, was extensively audited by BSI against the requirements of the standard, covering all aspects spanning the design of our AI, use of AI within the organization, and our competencies, resources and HR processes. It is an in-depth process that we’re thrilled to have undertaken, making us one of the first in our industry to achieve certification for a globally recognized AI system.

The scope of Darktrace’s certification is particularly wide due to our unique Self-Learning approach to AI for cybersecurity, which uses multi-layered AI systems consisting of varied AI techniques to address distinct cybersecurity tasks. The certification encompasses production and provision of AI systems based on anomaly detection, clustering, classifiers, regressors, neural networks, proprietary and third-party large language models for proactive, detection, response and recovery cybersecurity applications. Darktrace additionally elected to adopt all Annex A controls present in the ISO/IEC 42001 standard.

What are the benefits of an AI Management System?

While AI is not a new or novel concept, the AI industry has accelerated at an unprecedented rate in the past few years, increasing operational efficiency, driving innovation, and automating cumbersome processes in the workplace.

At the same time, the data privacy, security and bias risks created by rapid innovation in AI have been well documented.

Thus, an AI Management System enables organizations to confidently establish and adhere to governance in a way that conforms to best practice, promotes adherence, and is in line with current and emerging regulatory standards.

Not only is this vital in a unique and rapidly evolving field like AI, it additionally helps organization’s balance the drive for innovation with the risks the technology can present, helping to get the best out of their AI development and usage.

What are the key components of ISO/IEC 42001?

The Standard puts an emphasis on responsible AI development and use, requiring organizations to:

  • Establish and implement an AI Management System
  • Commit to the responsible development of AI against established, measurable objectives
  • Have in place a process to manage, monitor and adapt to risks in an effective manner
  • Commit to continuous improvement of their AI Management System

The AI Standard is similar in composition to other ISO standards, such as ISO/IEC 27001:2022, which many organizations may already be familiar with. Further information as to the structure of ISO/IEC 42001 can be found in Annex A.

What it means for Darktrace’s customers

Our certification against ISO/IEC 42001 demonstrates Darktrace’s commitment to delivering industry-leading Self-Learning AI in the name of cybersecurity resilience. Our stakeholders, customers and partners can be confident that Darktrace is responsibly, ethically and securely developing its AI systems, and is managing the use of AI in our day-to-day operations in a compliant, secure and ethical manner. It means:

  • You can trust our AI: We can demonstrate our AI is developed responsibly, in a transparent manner and in accordance with ethical rules. For more information and to learn about Darktrace's responsible AI in cybersecurity approach, please see here.
  • Our products are backed by innovation and integrity: Darktrace drives cutting edge AI innovation with ethical governance and customer trust at its core.
  • You are partnering with an organization which stays ahead of regulatory changes: In an evolving AI landscape, partnering with Darktrace helps you to stay prepared for emerging compliance and regulatory demands in your supply chain.

Achieving ISO/IEC 42001:2023 certification is not just a checkpoint for us. It represents our unwavering commitment to setting a higher standard for AI in cybersecurity. It reaffirms our leadership in building and implementing responsible AI and underscores our mission to continuously innovate and lead the way in the industry.

Why ISO/IEC 42001 matters for every AI vendor you trust

In a market where “AI” can mean anything from a true, production-grade system to a thin marketing layer, ISO/IEC 42001 acts as a critical differentiator. Vendors who have earned this certification aren’t just claiming they build responsible AI, they’ve proven it through an independent, rigorous audit of how they design, deploy, and manage their systems.

For you as a customer, that means:

You know their AI is real: Certified vendors have dedicated, skilled AI teams building and maintaining systems that meet measurable standards, not just repackaging off-the-shelf tools with an “AI” label.

Your data is safeguarded: Compliance with ISO/IEC 42001 includes stringent governance over data use, bias, transparency, and risk management.

You’re partnering with innovators: The certification process encourages continuous improvement, meaning your vendor is actively advancing AI capabilities while keeping ethics and security in focus.

In short, ISO/IEC 42001 is quickly becoming the global badge of credible AI development. If your vendor can’t show it, it’s worth asking how they manage AI risk, whether their governance is mature enough, and how they ensure innovation doesn’t outpace accountability.

Annex A: The Structure of ISO/IEC 42001

ISO/IEC 42001 has requirements for which seven adherence is required for an organization seeking to obtain or maintain its certification:

  • Context of the organization – organizations need to demonstrate an understanding of the internal and external factors influencing the organization’s AI Management System.
  • Leadership – senior leadership teams need to be committed to implementing AI governance within their organizations, providing direction and support across all aspects AI Management System lifecycle.
  • Planning – organizations need to put meaningful and manageable processes in place to identify risks and opportunities related to the AI Management System to achieve responsible AI objectives and mitigate identified risks.
  • Support – demonstrating a commitment to provisioning of adequate resources, information, competencies, awareness and communication for the AI Management System is a must to ensure that proper oversight and management of the system and its risks can be achieved.
  • Operation – establishing processes necessary to support the organization’s AI system development and usage, in conformance with the organization’s AI policy, objectives and requirements of the standard. Correcting the course of any deviations within good time is paramount.
  • Performance evaluation – the organization must be able to demonstrate that it has the capability and willingness to regularly monitor and evaluate the performance of the AI Management System effectively, including actioning any corrections and introducing new processes where relevant.
  • Improvement – relying on an existing process will not be sufficient to ensure compliance with the AI Standard. Organizations must commit to monitoring of existing systems and processes to ensure that the AI Management System is continually enhanced and improved.

To assist organizations in seeking the above, four annexes are included within the AI Standard’s rubric, which outline the objectives and measures an organization may wish to implement to address risks related to the design and operation of their AI Management System through the introduction of normative controls. Whilst they are not prescriptive, Darktrace has implemented the requirements of these Annexes to enable it to appropriately demonstrate the effectiveness of its AI Management System. We have placed a heavy emphasis on Annex A which contains these normative controls which we, and other organizations seeking to achieve certification, can align with to address the objectives and measures, such as:

  • Enforcement of policies related to AI.
  • Setting responsibilities within the organization, and expectation of roles and responsibilities.
  • Creating processes and guidelines for escalating and handling AI concerns.
  • Making resources for AI systems available to users.
  • Assessing impacts of AI systems internally and externally.
  • Implementing processes across the entire AI system life cycle.
  • Understanding treatment of Data for AI systems.
  • Defining what information is, and should be available, for AI systems.
  • Considering and defining use cases for the AI systems.
  • Considering the impact of the AI System on third-party and customer relationships.

The remaining annexes provide guidance on implementing Annex A’s controls, objectives and primary risk sources of AI implementation, and considering how the AI Management System can be used across domains or sectors responsibly.

[related-resource]

Continue reading
About the author

Blog

/

/

August 12, 2025

Minimizing Permissions for Cloud Forensics: A Practical Guide to Tightening Access in the Cloud

Cloud permissions cloud forensicsDefault blog imageDefault blog image

Most cloud environments are over-permissioned and under-prepared for incident response.

Security teams need access to logs, snapshots, and configuration data to understand how an attack unfolded, but giving blanket access opens the door to insider threats, misconfigurations, and lateral movement.

So, how do you enable forensics without compromising your security posture?

The dilemma: balancing access and security

There is a tension between two crucial aspects of cloud security that create a challenge for cloud forensics.

One aspect is the need for Security Operations Center (SOC) and Incident Response (IR) teams to access comprehensive data for investigating and resolving security incidents.

The other conflicting aspect is the principle of least privilege and minimal manual access advocated by cloud security best practices.

This conflict is particularly pronounced in modern cloud environments, where traditional physical access controls no longer apply, and infrastructure-as-code and containerization have transformed the landscape.

There are several common but less-than-ideal approaches to this challenge:

  • Accepting limited data access, potentially leaving incidents unresolved
  • Granting root-level access during major incidents, risking further compromise

Relying on cloud or DevOps teams to retrieve data, causing delays and potential miscommunication

[related-resource]

Challenges in container forensics

Containers present unique challenges for forensic investigations due to their ephemeral and dynamic nature. The orchestration and management of containers, whether on private clusters or using services like AWS Elastic Kubernetes Service (EKS), introduce complexities in capturing and analyzing forensic data.

To effectively investigate containers, it's often necessary to acquire the underlying volume of a node or perform memory captures. However, these actions require specific Identity and Access Management (IAM) and network access to the node, as well as familiarity with the container environment, which may not always be straightforward.

An alternative method of collection in containerized environments is to utilize automated tools to collect this evidence. Since they can detect malicious activity and collect relevant data without needing human input, they can act immediately, securing evidence that might be lost by the time a human analyst is available to collect it manually.

Additionally, automation can help significantly with access and permissions. Instead of analysts needing the correct permissions for the account, service, and node, as well as deep knowledge of the container service itself, for any container from which they wish to collect logs. They can instead collect them, and have them all presented in one place, at the click of a button.

A better approach: practical strategies for cloud forensics

It's crucial to implement strategies that strike a balance between necessary access and stringent security controls.

Here are several key approaches:

1. Dedicated cloud forensics accounts

Establishing a separate cloud account or subscription specifically for forensic activities is foundational. This approach isolates forensic activities from regular operations, preventing potential contamination from compromised environments. Dedicated accounts also enable tighter control over access policies, ensuring that forensic operations do not inadvertently expose sensitive data to unauthorized users.

A separate account allows for:

  • Isolation: The forensic investigation environment is isolated from potentially compromised environments, reducing the risk of cross-contamination.
  • Tighter access controls: Policies and controls can be more strictly enforced in a dedicated account, reducing the likelihood of unauthorized access.
  • Simplified governance: A clear and simplified chain of custody for digital evidence is easier to maintain, ensuring that forensic activities meet legal and regulatory requirements.

For more specifics:

2. Cross-account roles with least privilege

Using cross-account IAM roles, the forensics account can access other accounts, but only with permissions that are strictly necessary for the investigation. This ensures that the principle of least privilege is upheld, reducing the risk of unauthorized access or data exposure during the forensic process.

3. Temporary credentials for just-in-time access

Leveraging temporary credentials, such as AWS STS tokens, allows for just-in-time access during an investigation. These credentials are short-lived and scoped to specific resources, ensuring that access is granted only when absolutely necessary and is automatically revoked after the investigation is completed. This reduces the window of opportunity for potential attackers to exploit elevated permissions.

For AWS, you can use commands such as:

aws sts get-session-token --duration-seconds 43200

aws sts assume-role --role-arn role-to-assume --role-session-name "sts-session-1" --duration-seconds 43200

For Azure, you can use commands such as:

az ad app credential reset --id <appId> --password <sp_password> --end-date 2024-01-01

For more details for Google Cloud environments, see “Create short-lived credentials for a service account” and the request.time parameter.

4. Tag-based access control

Pre-deploying access control based on resource tags is another effective strategy. By tagging resources with identifiers like "Forensics," access can be dynamically granted only to those resources that are relevant to the investigation. This targeted approach minimizes the risk of overexposure and ensures that forensic teams can quickly and efficiently access the data they need.

For example, in AWS:

Condition: StringLike: aws:ResourceTag/Name: ForensicsEnabled

Condition: StringLike: ssm:resourceTag/SSMEnabled: True

For example, in Azure:

"Condition": "StringLike(Resource[Microsoft.Resources/tags.example_key], '*')"

For example, in Google Cloud:

expression: > resource.matchTag('tagKeys/ForensicsEnabled', '*')

Tighten access, enhance security

The shift to cloud environments demands a rethinking of how we approach forensic investigations. By implementing strategies like dedicated cloud forensic accounts, cross-account roles, temporary credentials, and tag-based access control, organizations can strike the right balance between access and security. These practices not only enhance the effectiveness of forensic investigations but also ensure that access is tightly controlled, reducing the risk of exacerbating an incident or compromising the investigation.

Find the right tools for your cloud security

Darktrace delivers a proactive approach to cyber resilience in a single cybersecurity platform, including cloud coverage.

Darktrace’s cloud offerings have been bolstered with the acquisition of Cado Security Ltd., which enables security teams to gain immediate access to forensic-level data in multi-cloud, container, serverless, SaaS, and on-premises environments.

In addition to having these forensics capabilities, Darktrace / CLOUD is a real-time Cloud Detection and Response (CDR) solution built with advanced AI to make cloud security accessible to all security teams and SOCs. By using multiple machine learning techniques, Darktrace brings unprecedented visibility, threat detection, investigation, and incident response to hybrid and multi-cloud environments.

Continue reading
About the author
Your data. Our AI.
Elevate your network security with Darktrace AI