Evaluating Email Security: How to Select the Best Solution for Your Organization
In today’s saturated market for email security, it can be difficult to cut through the noise of AI hype and vendor claims. CISOs should be using a structured evaluation framework to support informed, objective comparisons of different vendors – to allow them to make the best decision for their organization.
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Carlos Gray
Senior Product Marketing Manager, Email
Share
21
May 2025
When evaluating email security solutions, it’s crucial to move beyond marketing claims and focus on real-world performance. One of the most effective ways to achieve this is through an A/B comparison approach – a side-by-side evaluation of vendors based on consistent, predefined criteria.
This method cuts through biases, reveals true capability differences, and ensures that all solutions are assessed on a level playing field. It’s not just about finding an objectively good solution – it’s about finding the best solution for your organization’s specific needs.
An A/B comparison approach is particularly effective for three main reasons:
Eliminates bias: By comparing solutions under identical conditions, it’s easier to spot differences in performance without the fog of marketing jargon.
Highlights real capabilities: Direct side-by-side testing exposes genuine strengths and weaknesses, making it easier to judge which features are impactful versus merely decorative.
Encourages objective decision-making: This structured method reduces emotional or brand-driven decisions, focusing purely on metrics and performance.
Let’s look at the key factors to consider when setting up your evaluation to ensure a fair, accurate, and actionable comparison.
Deployment: Setting the stage for fair evaluation
To achieve a genuine comparison, deployment must be consistent across all evaluated solutions:
Establish the same scope: All solutions should be granted identical visibility across relevant tenants and domains to ensure parity.
Set a concrete timeline: Deploy and test each solution with the same dataset, at the same points in time. This allows you to observe differences in learning periods and adaptive capabilities.
Equal visibility and synchronized timelines prevent discrepancies that could skew your understanding of each vendor’s true capabilities. But remember – quicker results might not equal better learning or understanding!
Tuning and configurations: Optimizing for real-world conditions
Properly tuning and configuring each solution is critical for fair evaluation:
Compare on optimal performance: Consult with each vendor to understand what optimal deployment looks like for their solution, particularly if machine learning is involved.
Consider the long term: Configuration adjustments should be made with long-term usage in mind. Short-term fixes can mask long-term challenges.
Data visibility: Ensure each solution can retain and provide search capabilities on all data collected throughout the evaluation period.
These steps guarantee that you are comparing fully optimized versions of each platform, not underperforming or misconfigured ones.
Evaluation: Applying consistent metrics
Once deployment and configurations are aligned, the evaluation itself must be consistent, to prevent unfair scoring and help to identify true differences in threat detection and response capabilities.
Coordinate your decision criteria: Ensure all vendors are measured against the same set of criteria, established before testing begins.
Understand vendor threat classification: Each vendor may have different ways of classifying threats, so be sure to understand these nuances.
Maintain communication: If results seem inaccurate, engage with the vendors. Their response and remediation capabilities are part of the evaluation.
Making a decision: Look beyond the metrics
When it comes to reviewing the performance of each solution, it’s important to both consider and look beyond the raw data. This is about choosing the solution that best aligns with your specific business needs, which may include factors and features not captured in the results.
Evaluate based on results: Consider accuracy, threats detected, precision, and response effectiveness.
Evaluate beyond results: Assess the overall experience, including support, integrations, training, and long-term alignment with your security strategy.
Review and communicate: Internally review the findings and communicate them back to the vendors.
Choosing the right email security solution isn’t just about ticking boxes, it’s about strategic alignment with your organization’s goals and the evolving threat landscape. A structured, A/B comparison approach will help ensure that the solution you select is truly the best fit.
For a full checklist of the features and capabilities to compare, as well as how to perform a commercial and technical evaluation, check out the full Buyer’s Checklist for Evaluating Email Security.
[related-resource]
A Buyer’s Checklist for Evaluating Email Security
Get the most effective system for comparing email security solutions – based on real-world performance and measurable capabilities.
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Out of Character: Detecting Vendor Compromise and Trusted Relationship Abuse with Darktrace
Phishing emails from compromised vendors are increasingly difficult to distinguish from genuine correspondence. They challenge workers, security teams and traditional email SEGs alike. Anomaly detection can be a game-changer in spotting the subtle signs of these meticulous attacks.
Darktrace Collaborates with Microsoft: Unifying Email Security with a Shared Vision
Darktrace and Microsoft have joined forces to enhance email security through a new integration, unifying threat response and quarantine capabilities. This collaboration strengthens defenses and streamlines visibility for security teams, reflecting a shared vision for proactive cyber protection.
WSUS Exploited: Darktrace’s Analysis of Post-Exploitation Activities Related to CVE-2025-59287
Introduction
On October 14, 2025, Microsoft disclosed a new critical vulnerability affecting the Windows Server Update Service (WSUS), CVE-2025-59287. Exploitation of the vulnerability could allow an unauthenticated attacker to remotely execute code [1][6].
WSUS allows for centralized distribution of Microsoft product updates [3]; a server running WSUS is likely to have significant privileges within a network making it a valuable target for threat actors. While WSUS servers are not necessarily expected to be open to the internet, open-source intelligence (OSINT) has reported thousands of publicly exposed instances that may be vulnerable to exploitation [2].
Microsoft’s initial ‘Patch Tuesday’ update for this vulnerability did not fully mitigate the risk, and so an out-of-band update followed on October 23 [4][5] . Widespread exploitation of this vulnerability started to be observed shortly after the security update [6], prompting CISA to add CVE-2025-59287 to its Known Exploited Vulnerability Catalog (KEV) on October 24 [7].
Attack Overview
The Darktrace Threat Research team have recently identified multiple potential cases of CVE-2025-59287 exploitation, with two detailed here. While the likely initial access method is consistent across the cases, the follow-up activities differed, demonstrating the variety in which such a CVE can be exploited to fulfil each attacker’s specific goals.
The first signs of suspicious activity across both customers were detected by Darktrace on October 24, the same day this vulnerability was added to CISA’s KEV. Both cases discussed here involve customers based in the United States.
Case Study 1
The first case, involving a customer in the Information and Communication sector, began with an internet-facing device making an outbound connection to the hostname webhook[.]site. Observed network traffic indicates the device was a WSUS server.
OSINT has reported abuse of the workers[.]dev service in exploitation of CVE-2025-59287, where enumerated network information gathered through running a script on the compromised device was exfiltrated using this service [8].
In this case, the majority of connectivity seen to webhook[.]site involved a PowerShell user agent; however, cURL user agents were also seen with some connections taking the form of HTTP POSTs. This connectivity appears to align closely with OSINT reports of CVE-2025-59287 post-exploitation behaviour [8][9].
Connections to webhook[.]site continued until October 26. A single URI was seen consistently until October 25, after which the connections used a second URI with a similar format.
Later on October 26, an escalation in command-and-control (C2) communication appears to have occurred, with the device starting to make repeated connections to two rare workers[.]dev subdomains (royal-boat-bf05.qgtxtebl.workers[.]dev & chat.hcqhajfv.workers[.]dev), consistent with C2 beaconing. While workers[.]dev is associated with the legitimate Cloudflare Workers service, the service is commonly abused by malicious actors for C2 infrastructure. The anomalous nature of the connections to both webhook[.]site and workers[.]dev led to Darktrace generating multiple alerts including high-fidelity Enhanced Monitoring alerts and alerts for Darktrace’s Autonomous Response.
Infrastructure insight
Hosted on royal-boat-bf05.qgtxtebl.workers[.]dev is a Microsoft Installer file (MSI) named v3.msi.
Figure 1: Screenshot of v3.msi content.
Contained in the MSI file is two Cabinet files named “Sample.cab” and “part2.cab”. After extracting the contents of the cab files, a file named “Config” and a binary named “ServiceEXE”. ServiceEXE is the legitimate DFIR tool Velociraptor, and “Config” contains the configuration details, which include chat.hcqhajfv.workers[.]dev as the server_url, suggesting that Velociraptor is being used as a tunnel to the C2. Additionally, the configuration points to version 0.73.4, a version of Velociraptor that is vulnerable to CVE-2025-6264, a privilege escalation vulnerability.
Figure 2: Screenshot of Config file.
Velociraptor, a legitimate security tool maintained by Rapid7, has been used recently in malicious campaigns. A vulnerable version of tool has been used by threat actors for command execution and endpoint takeover, while other campaigns have used Velociraptor to create a tunnel to the C2, similar to what was observed in this case [10] .
The workers[.]dev communication continued into the early hours of October 27. The most recent suspicious behavior observed on the device involved an outbound connection to a new IP for the network - 185.69.24[.]18/singapure - potentially indicating payload retrieval.
The payload retrieved from “/singapure” is a UPX packed Windows binary. After unpacking the binary, it is an open-source Golang stealer named “Skuld Stealer”. Skuld Stealer has the capabilities to steal crypto wallets, files, system information, browser data and tokens. Additionally, it contains anti-debugging and anti-VM logic, along with a UAC bypass [11].
Figure 3: A timeline outlining suspicious activity on the device alerted by Darktrace.
Case Study 2
The second case involved a customer within the Education sector. The affected device was also internet-facing, with network traffic indicating it was a WSUS server
Suspicious activity in this case once again began on October 24, notably only a few seconds after initial signs of compromise were observed in the first case. Initial anomalous behaviour also closely aligned, with outbound PowerShell connections to webhook[.]site, and then later connections, including HTTP POSTs, to the same endpoint with a cURL user agent.
While Darktrace did not observe any anomalous network activity on the device after October 24, the customer’s security integration resulted in an additional alert on October 27 for malicious activity, suggesting that the compromise may have continued locally.
By leveraging Darktrace’s security integrations, customers can investigate activity across different sources in a seamless manner, gaining additional insight and context to an attack.
Figure 4: A timeline outlining suspicious activity on the device alerted by Darktrace.
Conclusion
Exploitation of a CVE can lead to a wide range of outcomes. In some cases, it may be limited to just a single device with a focused objective, such as exfiltration of sensitive data. In others, it could lead to lateral movement and a full network compromise, including ransomware deployment. As the threat of internet-facing exploitation continues to grow, security teams must be prepared to defend against such a possibility, regardless of the attack type or scale.
By focussing on detection of anomalous behaviour rather than relying on signatures associated with a specific CVE exploit, Darktrace is able to alert on post-exploitation activity regardless of the kind of behaviour seen. In addition, leveraging security integrations provides further context on activities beyond the visibility of Darktrace / NETWORKTM, enabling defenders to investigate and respond to attacks more effectively.
With adversaries weaponizing even trusted incident response tools, maintaining broad visibility and rapid response capabilities becomes critical to mitigating post-exploitation risk.
Credit to Emma Foulger (Global Threat Research Operations Lead), Tara Gould (Threat Research Lead), Eugene Chua (Principal Cyber Analyst & Analyst Team Lead), Nathaniel Jones (VP, Security & AI Strategy, Field CISO),
o royal-boat-bf05.qgtxtebl.workers[.]dev – Hostname – Likely C2 Infrastructure
o royal-boat-bf05.qgtxtebl.workers[.]dev/v3.msi - URI – Likely payload
o chat.hcqhajfv.workers[.]dev – Hostname – Possible C2 Infrastructure
o 185.69.24[.]18 – IP address – Possible C2 Infrastructure
o 185.69.24[.]18/bin.msi - URI – Likely payload
o 185.69.24[.]18/singapure - URI – Likely payload
The content provided in this blog is published by Darktrace for general informational purposes only and reflects our understanding of cybersecurity topics, trends, incidents, and developments at the time of publication. While we strive to ensure accuracy and relevance, the information is provided “as is” without any representations or warranties, express or implied. Darktrace makes no guarantees regarding the completeness, accuracy, reliability, or timeliness of any information presented and expressly disclaims all warranties.
Nothing in this blog constitutes legal, technical, or professional advice, and readers should consult qualified professionals before acting on any information contained herein. Any references to third-party organizations, technologies, threat actors, or incidents are for informational purposes only and do not imply affiliation, endorsement, or recommendation.
Darktrace, its affiliates, employees, or agents shall not be held liable for any loss, damage, or harm arising from the use of or reliance on the information in this blog.
The cybersecurity landscape evolves rapidly, and blog content may become outdated or superseded. We reserve the right to update, modify, or remove any content
Patch Smarter, Not Harder: Now Empowering Security Teams with Business-Aligned Threat Context Agents
Most risk management programs remain anchored in enumeration: scanning every asset, cataloging every CVE, and drowning in lists that rarely translate into action. Despite expensive scanners, annual pen tests, and countless spreadsheets, prioritization still falters at two critical points.
Context gaps at the device level: It’s hard to know which vulnerabilities actually matter to your business given existing privileges, what software it runs, and what controls already reduce risk.
Business translation: Even when the technical priority is clear, justifying effort and spend in financial terms—especially across many affected devices—can delay action. Especially if it means halting other areas of the business that directly generate revenue.
The result is familiar: alert fatigue, “too many highs,” and remediation that trails behind the threat landscape. Darktrace / Proactive Exposure Management addresses this by pairing precise, endpoint‑level context with clear, financial insight so teams can prioritize confidently and mobilize faster.
A powerful combination: No-Telemetry Endpoint Agent + Cost-Benefit Analysis
Darktrace / Proactive Exposure Management now uniquely combines technical precision with business clarity in a single workflow. With this release, Darktrace / Proactive Exposure Management delivers a more holistic approach, uniting technical context and financial insight to drive proactive risk reduction. The result is a single solution that helps security teams stay ahead of threats while reducing noise, delays, and complexity.
No-Telemetry Endpoint: Collects installed software data and maps it to known CVEs—without network traffic—providing device-level vulnerability context and operational relevance.
Cost-Benefit Analysis for Patching: Calculates ROI by comparing patching effort with potential exploit impact, factoring in headcount time, device count, patch difficulty, and automation availability.
Introducing the No-Telemetry Endpoint Agent
Darktrace’s new endpoint agent inventories installed software on devices and maps it to known CVEs without collecting network data so you can prioritize using real device context and available security controls.
By grounding vulnerability findings in the reality of each endpoint, including its software footprint and existing controls, teams can cut through generic severity scores and focus on what matters most. The agent is ideal for remote devices, BYOD-adjacent fleets, or environments standardizing on Darktrace, and is available without additional licensing cost.
Figure 1: Darktrace / Proactive Exposure Management user interface
Built-In Cost-Benefit Analysis for Patching
Security teams often know what needs fixing but stakeholders need to understand why now. Darktrace’s new cost-benefit calculator compares the total cost to patch against the potential cost of exploit, producing an ROI for the patch action that expresses security action in clear financial terms.
Inputs like engineer time, number of affected devices, patch difficulty, and automation availability are factored in automatically. The result is a business-aligned justification for every patching decision—helping teams secure buy-in, accelerate approvals, and move work forward with one-click ticketing, CSV export, or risk acceptance.
Together, the no-telemetry endpoint and Cost–Benefit Analysis advance the CTEM motion from theory to practice. You gain higher‑fidelity discovery and validation signals at the device level, paired with business‑ready justification that accelerates mobilization. The result is fewer distractions, clearer priorities, and faster measurable risk reduction. This is not from chasing every alert, but by focusing on what moves the needle now.
Smarter Prioritization: Device‑level context trims noise and spotlights the exposures that matter for your business.
Faster Decisions: Built‑in ROI turns technical urgency into executive clarity—speeding approvals and action.
Practical Execution: Privacy‑conscious endpoint collection and ticketing/export options fit neatly into existing workflows.
Better Outcomes: Close the loop faster—discover, prioritize, validate, and mobilize—on the same operating surface.
Committed to innovation
These updates are part of the broader Darktrace release, which also included:
3. Improvements to our OT product, purpose built for industrial infrastructure, Darktrace / OT now brings dedicated OT dashboard, segmentation-aware risk modeling, and expanded visibility into edge assets and automation protocols.
Join our live broadcast to experience how Darktrace is eliminating blind spots for detection and response across your complete enterprise with new innovations in Agentic AI across our ActiveAI Security platform. Industry leaders from IDC will join Darktrace customers to discuss challenges in cross-domain security, with a live walkthrough reshaping the future of Network Detection & Response, Endpoint Detection & Response, Email Security, and SecOps in novel threat detection and autonomous investigations.
.jpg)
.jpeg)
.avif)
