Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Justin Fier
SVP, Red Team Operations
Share
03
Nov 2016
The 2016 U.S. election is roiled by fears over election tampering and cyber-warfare. While such anxiety threatens to undermine confidence in the results, the up-side is that for the first time since 2000, the election is generating thoughtful discussion on the intersection of cyber-security and voting.
After the high-profile hack of the Democratic National Committee, and after attacks on voter registration databases in 20 states, these fears are certainly justified. After all, we live in a new era of threat, where foreign powers don’t hesitate to use cyber-tools for economic and political gain. The White House has now formally blamed Russia for the DNC hack, but they’re hardly the only nation-state willing to engage in cloak-and-dagger cyber-warfare.
Further complicating matters is that our voting machines are in desperate need of an overhaul. In 2006, computer scientists proved that in less than a minute, an e-voting machine could be hacked and installed with vote-changing malware, and it can even be done remotely. But intentional manipulation may not even be our biggest concern — in 2004, North Carolina lost 4,438 votes because of a system error.
But after the 2000 election, when the infamous ‘hanging chads’ forced millions of votes to be invalidated, it became clear that paper ballots are not only cumbersome, but inaccurate. Two years later, Congress passed the Help America Vote Act and introduced digitized voting and registration databases across America. Unfortunately, the new machines were plagued with errors, and many of them are still in use today.
Growing concern over election tampering prompted 33 state election agencies to petition the Department of Homeland Security for aid. The DHS responded by offering “cyber hygiene scans on Internet-facing systems as well as risk and vulnerability assessments.”
This is a good start, but hardly a long-term solution. Cyber-security for the future has to go beyond one-off scans and retrospective assessments. The answer has to involve intelligently monitoring and analyzing millions of devices — from voting machines to vulnerable IoT devices — in order to mitigate risk from unknown threats. Whether it be a state-sponsored hack or tampering from a politically motivated insider, the integrity of our elections is at stake, and its security deserves the utmost attention.
To hear more of my thoughts on the modern threat landscape, sign up for my webinar on November 9.
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
A New Security Challenge: The Curious Case of Prompt Language Analysis
Why prompt analysis is emerging as a key AI security challenge
If securing AI has been one of the defining cybersecurity conversations of the past year, prompt analysis is quickly becoming one of its most interesting frontiers.
Security leaders are under pressure to understand how AI is being used across the business. In some organizations, that means governing employee use of chatbots. In others, it means overseeing copilots embedded into SaaS platforms, monitoring coding assistants, or assessing the growing footprint of autonomous agents. However different these use cases may appear on the surface, they share a common factor: humans and machines are usually interacting with enterprise systems through language.
How prompt language differs from traditional security telemetry
For years, defenders have become used to working with familiar forms of telemetry: email traffic, network connections, API calls, endpoint processes, authentication events. Prompt language is different. It is not simply another log source. It is an expression of intent, instruction, curiosity, urgency, and sometimes manipulation. It reflects the end-goal of a user or agent, but not always with enough surrounding context to interpret the risk correctly.
Why existing security approaches only partially explain prompt risk
A growing number of vendors are approaching the task of securing AI from the angle they know best. Perimeter vendors are extending web or browser controls into AI usage. Identity vendors are emphasizing agent permissions and access governance. Data security and DLP providers are focusing on content inspection and exfiltration risk. All of these perspectives matter, but individually can’t fully explain the problem.
The challenge with securing AI is not just that a new application category has emerged. It is that language has become a new operating layer in the enterprise.
Employees now use prompts to summarize documents, generate code, analyze spreadsheets, query internal knowledge, and trigger multi-step actions through agents. In each case, prompt language acts as the interface between human intent and machine execution. That makes prompts incredibly valuable from a security perspective as they can hint at misuse, policy violations, data exposure, or attempts to circumvent controls. However, they can also be deeply ambiguous when viewed in isolation. That ambiguity is the heart of the issue.
Prompts as behavioral signals, not just text to classify
A prompt by itself tells you what was asked. It does not necessarily tell you whether the request is expected, risky, accidental, or entirely legitimate in context. Two nearly identical prompts can carry very different meanings depending on the role and function of who issued them, what systems they can access, and what actions followed. In other words, prompts are not just text to classify. They are behavioral signals to interpret.
Example: How context changes prompt risk entirely
Consider a common enterprise scenario. An employee is pulled into a new project with an aggressive deadline. Almost overnight, their use of AI tools spikes. They begin prompting more frequently, working across unfamiliar documents, querying new data sources, and interacting with more systems than usual to accelerate delivery. Viewed narrowly, this may look suspicious. Prompt volume increases, file access patterns change, API and SaaS activity rise. From some vantage points, it may resemble insider risk or unmanaged AI usage.
But now add context. Imagine that, earlier that day, the employee received instructions from a senior leader asking them to support a time-sensitive initiative. Their communication history shows that this leader is a legitimate reporting-line superior. Their recent collaboration patterns align with the new project team. Their subsequent activity, while unusual for that individual’s baseline, is consistent with the business task they were assigned.
What initially looked like a risk event may actually be a normal response to business pressure. Without the surrounding context of communication, organizational relationships, and broader behavioral patterns, prompt activity alone could generate more noise than insight.
The reverse is also true. A prompt may appear benign on the surface while the context around it suggests elevated risk. A request that seems routine could originate from a compromised user, a newly connected external agent, a shadow AI workflow, or a user acting outside their normal role. The language itself may not contain anything obviously malicious, but the surrounding conditions may tell a very different story.
What security teams need to analyze prompts effectively
The future of prompt analysis is not just about understanding language. It is about understanding language in context.
To do that well, security teams need more than prompt inspection. They need to understand:
Who is issuing the prompt, whether human or agent
How that identity normally behaves across the enterprise
What systems, data, and workflows are connected to the interaction
Which relationships and communications explain the surrounding activity
Whether the downstream actions align with expected business behavior
When those layers are absent, prompt analysis can become another isolated control surface: useful in theory, but limited in practice. Security teams may detect unusual wording but miss the operational function behind it, overreact to benign changes in behavior, or miss subtle misuse because the prompt itself did not appear dangerous.
How organizations should think about prompt analysis going forward
Security teams have seen this pattern before. In the cloud, posture without runtime context left important gaps. In identity, access control without behavioral understanding missed misuse that looked legitimate on paper. In data security, content inspection without business context often created friction without resolving risk. AI is exposing the same lesson again: controls are strongest when they are coordinated, not isolated. As organizations work to secure AI and identify gaps across their security operations, prompt analysis will become an increasingly important source of insight, but only as part of a broader strategy.
Prompt analysis will undoubtedly become more common, as prompts are one of the clearest windows into how people and agents are using AI systems. However, what matters most is not simply collecting prompts or filtering dangerous phrases, but being able to place that language inside a wider behavioral and operational picture.
Organizations that already have a broader understanding of how work gets done across the enterprise will be better positioned to make sense of prompt language as this category matures. They will be better able to distinguish urgency from abuse, experimentation from exfiltration, and productive AI adoption from hidden risk.
Figure 1: Darktrace / SECURE AI reconstructs the full sequence of events, showing every user and agent interaction in context, with risky prompts highlighted and categorized, including PII, sensitive data, and other policy violations.
At Darktrace, this is the key lesson emerging from the market: prompt language does matter, but it does not stand alone. It is most valuable when treated as a new behavioral input that can enrich understanding across the enterprise, not as a self-contained source of truth.
Why prompts become less useful when analyzed in isolation
The curious case of prompt language analysis, then, is this: the more important prompts become, the less useful they are in a vacuum.
The real opportunity is not just to see what was asked. It is to understand why it was asked, what it meant in that moment, and what happened next.
For a deeper look at how organizations are approaching this challenge from the strengths of prompt analysis to its limitations in isolation see Prompt Security in Enterprise AI: Strengths, Weaknesses, and Common Approaches, which expands on the role prompt-level controls play within a broader, context-driven security strategy.
