Blog
/
/
May 27, 2026

How to Evaluate AI Vendors: 5 Key categories for AI Adoption

Buying AI cybersecurity tools requires more than evaluating features and marketing claims. This blog explores the challenges security teams face when assessing AI vendors, outlines five critical categories for AI evaluation, and explains how governance, transparency, model selection, and validation impact trust, adoption, and long-term security outcomes.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Jamie Bali
Technical Author (AI) Developer
Default blog image
27
May 2026

Understanding the AI buyers’ market

AI adoption has become a central topic of discussion in boardrooms, drawing growing interest from business leaders. Ultimately, organizations hope that an investment in AI technology will have tremendous returns. However, the process of buying an AI solution is not as straight forward as it appears on the surface.  

While business leaders may be eager to improve productivity across their operations, practitioners responsible for evaluating and selecting AI solutions may not always have the visibility or technical understanding needed to make the right decisions for their business. What is typically marketed as a holistic solution to their most critical problems is usually followed by uncertainty when AI tools are finally operationalized in real environments.

This guide is intended to support security leaders who are under growing pressure to adopt AI tools while navigating complex terminology, vendor claims, and increasingly crowded buying cycles. Ultimately, the goal is to help organizations evaluate and adopt AI in a safe, effective, and well-governed way. To support this, we’ve structured the evaluation framework across five key categories:

  1. Governance, safety, and data controls
  1. Data gathering and training
  1. Model and technique choice
  1. Performance and accuracy validation    
  1. Interpretability, adjustability, and transparency    

What buying AI looks like in cybersecurity

While investing in AI can bring immense benefits to your security team, first-time buyers of AI cybersecurity solutions may not know where to start. They will have to determine the type of tool they want, know the options available, and evaluate vendors. Research and understanding are critical to ensure purchases are worth the investment.  

With acceleration in AI adoption, accompanied by the recent boom in agentic AI and autonomous agents, CISOs must look “beneath the hood" of these tools to understand how they work, how they are governed, and to ensure the system is secure and compliant with internal policies.

Challenges in the AI buyers’ marketplace  

The AI security software market is buzzing with hype and flashy promises, which, understandably, needs to be addressed with due diligence. Potential buyers, especially in the cybersecurity space, are hesitant when it comes to allowing AI autonomous capabilities across their workflows, and a lack of vendor transparency can exacerbate those feelings.  

Reinforcing this sentiment, research from this year's Darktrace’s State of AI Cybersecurity report shows where confidence and hesitancy emerge amongst potential buyers. On the one hand, security professionals agree that they have good visibility into the logic and reasoning processes their AI solutions use. However, they lack the explainability and trust to allow AI to take independent remedial action.

  • 89% say they have good visibility into the reasoning behind the outputs generated by AI solutions
  • 92% say they need to understand how a defensive AI tool makes decisions before they can trust it
  • Only 14% say they allow AI to act independently, performing autonomous actions without human approval
  • 74% say they are limiting the autonomy of AI taking action in their SOC until explainability improves

Given the desire for trust and explainability we are seeing from buyers, it's important for them to be equipped with the right questions to ask vendors during an assessment or POV of AI tools in order to demystify marketing hype from real operational outcomes.

Below is a list of categories in which buyers can assess AI vendors or AI Service Providers (AISPs) to help reach safe adoption and maximize their ROI.  

5 categories of AI vendor assessment

Darktrace groups these AI-related questions into 5 categories: governance, data and training, model and technique choice, performance validation, and interpretability and adjustability. By asking questions regarding each of these 5 categories, buyers can gain a deeper understanding of how an AISP’s systems work and whether they suit their business requirements.

Governance, safety, and data controls

Governance of AI systems is critical for all AISPs. Whether their platform is based around a single model, or is a more complex, composite AI solution, strong governance is essential to ensure the system is safe, robust, and reliable.

A simple question you could ask is:

What AI governance policies and frameworks do you follow, and/or certifications do you currently maintain?

For more questions you can ask vendors, download the full guide here.

Darktrace is certified to the ISO/IEC 42001 standard, the world’s first AI Management System (AIMS) standard. ISO/IEC 42001 addresses the unique ethical and technical challenges AI poses by setting out a structured way to manage risks such as transparency, accuracy, and misuse. This includes a commitment to ethical AI development, and effective management and monitoring of AI systems both prior to and continually after release.

Data gathering and training

Accurate, meaningful, and unbiased data gathering is the first important step in producing any AI system. An AI model trained using inaccurate, unbalanced, or poor-quality training data will fail to perform optimally.

To alleviate concerns regarding training data quality, a question you could ask is:

What steps do you take to prevent bias in your AI models and training data?

For more questions, download the full guide here.

AISPs should be able to provide information about the steps taken, workflows followed, and auditing performed to reduce AI bias where appropriate. While it’s sometimes impossible to fully remove bias from an AI model, appropriate actions should be taken to mitigate or reduce bias where relevant.

Model and technique choice

Different AI techniques are optimal for different tasks. For example, research from Gartner suggests that relying on a single “one-size-fits-all" model can lead to data gaps, especially in highly specialized domains.

To achieve more accurate and robust AI solutions, AI leaders should move beyond using just one model or technique, embrace composite AI practices, and adopt a holistic AI system perspective.

A straightforward question you could ask is simply:

What type(s) of AI model(s) do you utilize in your solution?

For more questions, download the full guide here.

While specific detailed information about custom systems used by AISPs is likely proprietary, buyers should expect vendors to be able to provide an overview of the broad techniques used. This will allow you as a buyer to determine if the type of model is appropriate for your use case.

Performance and accuracy validation  

Testing and evaluation of performance is essential for all AI systems. Performance analysis should be performed both before release and continually after release to identify potential data or model drift.  

A question you could ask to understand an AISPs testing workflow is:

How do you audit, test, evaluate, verify, and validate your AI model outputs?

For more questions, download the full guide here.

Testing workflows will likely vary depending on the type of model – measurements relevant to one system may not always be relevant to others. Assessment of systems should also extend beyond these standard accuracy and robustness tests, and should also feature physical performance, such as latency and resource consumption.  

Interpretability, adjustability, and transparency  

AI systems are typically a black box, simply providing an output without an explanation of how that output was attained. Interpretability and transparency are critical to ensure that both SOC teams and end-users trust the outputs of a system to be accurate and meaningful.

A question you could ask is:

How do you promote a trust relationship between human analysts and AI outputs?

For more questions, download the full guide here.

In the context of cybersecurity, trust and interpretability are even more essential. This is particularly relevant for generative AI-based systems (including most AI Agents), where the risk of hallucination can reduce trust in responses.

Cybersecurity systems often need to perform autonomous actions to block incoming threats – an email filtering system may hold potentially dangerous emails; a firewall may block malicious inbound connections. If SOC teams can’t trust these systems to perform accurately, these systems may be limited or disabled, critically reducing their defensive power.

Darktrace as an AI-native cybersecurity vendor

Darktrace has been building and applying AI in cybersecurity for over a decade, developing its capabilities alongside an increasingly complex and fast‑moving threat landscape. This experience has resulted in a mature, multi-layered approach to AI, which continuously learns the normal patterns of each organization to understand behavior, interpret context, and identify meaningful deviations — without relying on predefined rules or known attack signatures. Over time, this has enabled a proven behavioral understanding that helps uncover subtle signals of risk that may otherwise be missed.

With the backing of our ISO/IEC 42001 certification, stakeholders, customers, and partners can be confident that Darktrace is responsibly, ethically, and safely developing its AI systems, and managing the use of AI in day-to-day operations in a compliant and secure manner.  

Explore the principles behind Darktrace’s responsible AI approach, informed by collaboration with global experts in academia and governments, detailing how accountability, explainability, and continuous validation are built into its cybersecurity technology.

How Darktrace secures AI systems

Darktrace now brings these capabilities to monitor and respond to risk generated from AI systems across organizations with Darktrace / SECURE AI. This solution analyzes how prompts, agents, and systems are used within the context of each organization, bringing every AI interaction into a single view. This unique approach helps teams understand intent, assess risk, protect sensitive data, and enforce policy across both human and AI agent activity.

Stay up to date

Sign up for the Secure AI Readiness Program here: This gives you exclusive access to the latest news on the latest AI threats, updates on emerging approaches shaping AI security, and insights into the latest innovations, including Darktrace’s ongoing work in this area.

Ready to talk with a Darktrace expert on securing AI? Register here to receive practical guidance on the AI risks that matter most to your business, paired with clarity on where to focus first across governance, visibility, risk reduction, and long-term readiness.  

Further Reading on AI in cybersecurity

When deciding to invest in an AI solution, it’s important to understand what this means for you and your organization. The questions presented here are only a starting point in understanding an AI solution and whether it is appropriate for your use case.  

Gain deeper knowledge on applications of AI in cybersecurity and Darktrace’s multi-layered AI in the AI Arsenal White Paper.

[related-resource]

Explore the CISO’s Guide to Buying AI

Learn how to understand an AI Service Provider’s solution and determine if it’s right choice for your organization

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Jamie Bali
Technical Author (AI) Developer

More in this series

No items found.

Blog

/

/

June 11, 2026

Cybersecurity for the Sports Sector: The Threats Facing a Digitized Industry in 2026

Sports Stadium cybersecurityDefault blog imageDefault blog image

Securing sporting events in 2026

When you walk into a stadium on game day, you are entering a small smart city. Ticketing, turnstiles, payments, public Wi-Fi for tens of thousands of fans, CCTV, lighting, even the HVAC all run on connected systems. The experience for fans has become unmatched, but that dependency has created a much larger attack surface than people may realize.

Our latest threat research backs that up. In the past year, a survey that Darktrace commissioned found that 84% of respondents from professional sports organizations had at least one cyber incident, and 57% were hit more than once. For a sector that relies on the impact of the live moment, those numbers translate directly into operational risk.

Why sports is a target for cyber attacks

Sport is a highly visible target with fixed timelines, so attackers know exactly when disruption will have the most impact. It also holds valuable data, athlete medical records, contracts, sponsorship deals, which carry financial, reputational, and regulatory risk if exposed. At the same time, delivery depends on a wide set of third parties: ticketing providers, broadcasters, cloud services, stadium technology. Any of those connections can become an entry point. Put visibility, timing, data, and dependency together, and you get an environment where even a small foothold can turn into a visible, time-critical incident.

How attackers target email and identity

Email and identity remain the front door. From October 2025 through March 2026, Darktrace / EMAIL™ detected more than 116,000 phishing emails aimed at sports organizations across our customer base, and our sports customers received 19% more phishing emails than organizations in other sectors. The numbers tell the story:

BY THE NUMBERS

  • 21% of phishing emails were aimed at VIPs.
  • 37% used novel social engineering.
  • 84% of malicious emails passed DMARC authentication

A large proportion of these emails passed authentication checks, which means traditional security controls are no longer a reliable barrier. Attackers are not relying on spoofed domains – they're using legitimate infrastructure and trusted platforms. Behavior matters. Once an account is compromised, the behavior shifts quickly. Login patterns change, inbox rules are created to hide responses, and accounts start being used for internal discovery or further phishing. These aren’t high-noise events. They sit in normal workflows, which is why they’re often missed.

Ransomware tells a similar story. In one case inside a sports deployment, attackers had quietly been moving data to an outside server for a full two weeks before they triggered encryption. By the time the ransom note appeared, the outcome was already set. That sequence shows up consistently is access first, movement next, disruption last. If detection starts at encryption, it’s already too late.

Why AI is an emerging blind spot in sports

The increasing adoption of AI is expanding the potential attack surface. 72% of the security professionals we surveyed expect AI to increase their cyber risk over the next year, and yet 35% are already using or planning to use it in stadium operations, the most critical functions to protect. In addition to prompt injection and AI build risks, shadow AI is becoming a more immediate issue. Staff are already putting sensitive data—performance metrics, scouting reports, contracts, health data—into tools with little or no governance. The upside is clear, but so is the exposure—and it is happening before most organizations have any visibility or control. At the same time, attackers are using the same technology to scale phishing and social engineering. The net effect is simple: more exposure, at higher speed.

How can cybersecurity professionals prepare

Across high profile events, Darktrace’s experience shows that effective cyber defense includes preparation, real‑time visibility, and the ability to respond dynamically and decisively when timing, complexity, and public exposure converge.

There are a few strategic implications for cybersecurity teams:

  • Get behavioral visibility across IT and OT, not just corporate systems.
  • Treat identity as your control plane. Most attacks in this sector start with credentials, not malware. MFA with behavioral detection helps solve that challenge.
  • Control third party and AI access the same way you control your own environment.
  • Rehearse response for live conditions, where decisions happen in minutes. Detection and response need to account for non-ideal conditions when engineers are under pressure and time constrained. In sport, timing is what turns small issues into major incidents. The same activity that would be manageable midweek becomes critical during a live event.

Why 2026 raises the cybersecurity stakes for sports

With the 2026 World Cup about to stretch across three countries and dozens of host cities, the attack surface is wide and the schedule is unforgiving.

Geopolitical signaling is raising the threat profile further. Previous international sporting events have demonstrated that nation‑state actors use the cyber domain to signal intent, influence narratives, or retaliate symbolically. In the context of the 2026 World Cup, Russia’s continued exclusion from international sport, the ongoing conflict in Ukraine, US defensive support to Ukraine, and Iran’s likely participation in the tournament introduce additional motivations for state‑aligned and non‑traditional affiliated actors to operate below the threshold of armed conflict. This doesn’t require new techniques—just the right timing and visibility.

In practice, this comes down to preparation: knowing what normal looks like across IT and OT, controlling third-party access, and spotting when behavior shifts.

In sport, disruption does not build slowly—it happens in real time and in public. By that point, the groundwork has already been set, long before the whistle goes.

About this research

Findings are based on Darktrace threat-research telemetry across sports-sector customer deployments (Q4 2025–Q1 2026) and a survey of 875 IT cybersecurity professionals in the US, UK, Australia, and Germany, fielded by Opinion Matters between May 28 and June 3, 2026. Read the full report for complete methodology, incident analysis, and strategic recommendations.

[related-resource]

Continue reading
About the author
Nathaniel Jones
VP, Security & AI Strategy, Field CISO

Blog

/

OT

/

June 11, 2026

Protecting Stadiums & Events with AI

Default blog imageDefault blog image

Stadium and large public venue operators are confronted with a unique set of cyber security challenges. Often described as a ‘honeypot’ for cyber-criminals, the sports and entertainment industry is an attractive target for threat actors for three main reasons:

  • Modern sports organizations process sensitive and highly valuable data at scale;
  • Sporting events are highly visible and time-critical, operating in front of live audiences with no room for error;
  • Sports organizations rely on sprawling vendor ecosystems and supply chains to deliver broadcast, commerce, fan engagement services, and more.

In a recent Darktrace-commissioned survey, 84% of professional sports organizations reported at least one cyber incident in the past year, and 57% were hit more than once [1]. The potential ramifications of cyber disruption during a large-scale sports event cannot be overstated. A momentary lapse in access to power could bring TV broadcasts to a halt; disruption to access controls could restrict fans from entering the grounds; CCTV outages could increase the risk of criminal behavior and physical injuries. If data is not reliable and stadium machines are outputting the wrong metrics, a venue could become dangerously overcrowded. The barrier between the cyber and physical worlds has long dissolved – cyber-attacks threaten human safety.

In this blog, I explore the key challenges of stadium cyber security and explain the unique capabilities of Self-Learning AI that led me to adopt Darktrace as a head of ICT and cyber security for international venues and events. Over my career I have helped secure football and rugby World Cups, World Athletics Championships and more than 500 events ,and the lessons from each have only sharpened my conviction in this approach.

The access paradox

The biggest challenge lies in the paradox of securing a site where various internal services are provided to a large number of unknown and unmanaged users, suppliers and devices. When it’s game time, or ‘D-Day’, you see a huge influx of thousands of people, each with their own devices, needing to connect to your network and your infrastructure. The floodgates are opened. But certain parts of your digital environment need to remain protected: your sensitive employee and customer data, your critical OT systems. I liken this to opening the door to your home, and letting the entire town come in and wander around. But you still need to secure your master bedroom.

A multitude of different actors must be able to work on-site to provide services or content during the event. Broadcasters, staff and suppliers need to have access to manage the show, and all these people need to access or interact with the IT infrastructure. In many ways, these additional bodies are already inside the perimeter and could host unknown malicious threats.

This year, the paradox is wider than ever. A tournament spread across hundreds of suppliers and vendors means the foothold an attacker needs may already belong to a trusted partner – a single compromised supplier can become the doorway to everything else. And the adversary is no longer working alone: generative AI now lets attackers probe and weaponize vulnerabilities across thousands of software dependencies at a speed no human team could match, turning the access paradox from a manageable risk into a fast-moving target.

Achieving this balance between accessibility and security requires a shift in mindset from perimeter-based security to one that can detect and respond to threats on the inside. The complexities involved requires technology that can identify malicious behavior in real time based on the wider context of an incident. A particular behavior or connection may be benign in one context and yet critically disruptive in another — tools and technology must be able to discern between the two.

This is why I considered Darktrace’s Self-Learning AI a suitable fit: rather than defending at the perimeter, it focuses on detecting and responding to malicious activity already inside. Because it learns the unique ‘patterns of life’ of its surroundings, it can detect subtle deviations that indicate a threat and initiate a targeted response – without relying on pre-programmed rules and playbooks.

IT/OT convergence

The second key challenge is the issue of IT and OT convergence. Typical stadiums and arenas consist of a wide range of Industrial Control Systems (ICS).

This involves a complex and messy array of switches, cables, CCTV cameras, as well as devices and technologies being brought in by the media and the press, and all these IT and OT components are now interconnected, which means these technologies now have Internet Protocol (IP)-based threats to manage. The same challenges that the corporate infrastructure for stadium management faces in cyber security are therefore also now an issue for ICS security.

This challenge cannot be addressed by viewing IT and OT security in isolation — these two environments are linked because of the analogue migration to IP. A unified approach is required to detect and respond to threats that start in IT before moving to industrial systems.

The stakes are physical. CCTV, Access Control, Public Annoucement system, lighting and the giant screens are all now running over IP, and a disruption to any of them can force a venue to halt play on safety grounds. Scale compounds the problem. At the Qatar 2022 World Cup, eight stadiums were purpose-built to a single technical standard, which made the digital environment relatively uniform to defend. The 2026 tournament is the opposite: dozens of host venues across three countries, each with its own operator, its own contractors and its own legacy systems.This creates a far more fragmented and unpredictable estate to secure.

In addition, cyber security technology must be able to deal with complexity. Darktrace’s AI thrives in the most complex environments, with more data points adding more context to inform the AI’s decision making. It covers OT and IT with a single, unified AI engine, that can also detect and respond across cloud infrastructure, SaaS applications, email systems and endpoints. It is ready to adapt to the messy, interconnected systems that make up large stadiums’ digital infrastructure.

The time factor

Finally, the nature of stadium events means that timing is critical and puts enormous pressure on the organizers and operators. ‘D-Day’ cannot be replayed or postponed, and so if cyber disruption occurs during the event, every minute is crucial. You cannot reschedule a World Cup final or move an opening ceremony; the date is fixed, the world is watching, and there is no second take.

There is consequently a strong emphasis on two key metrics

  • Mean Time To Know (MTTK) — how long it takes the security team need to be aware of an incident; and
  • Mean Time To Restore (MTTR) — how quickly a team can act to contain the threat.

It is perhaps more imperative in stadium event management than anywhere else that these two metrics be minimized.

This leads to the third criteria in assessing cyber security technology: does it help with response? And critically, can that response be nuanced and targeted, able to contain that threat without causing further disruption?

To this end, Darktrace’s Autonomous Response takes machine-speed action to contain cyber-attacks, when humans are too slow to react or aren’t around at all. It’s powered by Darktrace’s AI, so it has a nuanced and continuously updating understanding of what’s ‘normal’ across IT and OT systems. This means its response actions are targeted: designed to eliminate the threat, but not at the cost of disruption. Crucially, this enables responses that are surgical rather than blunt. For example, taking an entire server offline to stop a ransomware attack can cause more disruption than the attack itself, so the real value lies in neutralizing the malicious activity precisely — containing the threat without taking down the systems the event and business depends on.

Depending on the nature and severity of the threat, the technology can block specific malicious connections by enforcing the normal ‘pattern of life’ of a device or account. When every second counts, this is the speed and granularity that you need in a cybersecurity technology.

Darktrace can be deployed across every area of the digital enterprise, including network, email, cloud and SaaS environments with the same self-learning approach, stopping anomalous behaviors that point to account takeover and other cloud-based threats. Earlier this year, we announced that Darktrace is also extending its behavioral approach to help businesses deploy and scale AI securely by understanding how these AI systems and agents behave, interact with other systems and humans, and evolve over time. This is critical because 72% of cybersecurity professionals at sports organizations believe AI will increase their cyber risk over the next 12 months [2].

Wherever it is deployed, Darktrace allows the stadium operator to focus on the vital part of the game and offers real-time protection without any modification in the network topology or infrastructure.

An adaptive defense

Cyber-criminals are constantly developing their approach in an attempt to evade security tools trained to look for specific hallmarks of an attack. As they get creative and continuously experiment with new tactics and techniques, the human operators using these tools are forced into a constant state of catch up.

An AI-based approach that learns an organization and its normal behavior patterns from the ground up puts an end to this game of ‘cat and mouse’, shifting the balance in favor of the defenders and allowing them to stay ahead of the threat. This matters more than ever, because adversaries are now using AI to scale their attacks. If you do not have AI working to protect you against malicious AI, you are already at a disadvantage.

With a nuanced understanding of what’s ‘normal’ for the business, unified IT/OT coverage, and an Autonomous Response solution that takes immediate, surgical action, the playing field is leveled, and large stadium and events operators can focus on delivering the best possible experience for attendees, digital viewers, partners and performers.

References:

[1] [2] Darktrace: Cybersecurity in Global Sport, June 2026. Findings based on survey of 875 IT cybersecurity professionals based in the US, UK, Australia and Germany, working in professional sports organizations (including clubs, societies & sporting bodies) employing 10+ people. The survey was fielded between May 28, 2026 and June 3, 2026 by independent market research agency, Opinion Matters.

Continue reading
About the author
Your data. Our AI.
Elevate your network security with Darktrace AI