June 20, 2024

Post-Exploitation Activities on PAN-OS Devices: A Network-Based Analysis

This blog investigates the network-based activity detected by Darktrace in compromises stemming from the exploitation of a vulnerability in Palo Alto Networks firewall devices, namely CVE-2024-3400.

Written by

Adam Potter

Senior Cyber Analyst

Inside the SOC

Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.

Written by

Adam Potter

Senior Cyber Analyst

Jun 2024

Update:
Following the initial publication of this blog detailing exploitation campaigns utilizing the recently disclosed vulnerability, Darktrace analysts expanded the scope of the threat research investigation to identify potential earlier, pre-CVE disclosure, exploitation of CVE 2024-3400. While the majority of PAN-OS exploitation activity seen in the Darktrace customer base occurred after the public release of the CVE, Darktrace did also see tooling activity likely related to CVE-2024-3400 exploitation prior to the vulnerability's disclosure. Unlike the post-CVE-release exploitation activity, which largely reflected indiscriminate, opportunistic targeting of unpatched systems, these pre-CVE release activities likely represented selective targeting by more calculated actors.

Between March 26 and 28, Darktrace identified two Palo Alto firewall devices within the network of a public sector customer making HTTP GET requests utilizing both cURL and wget user agents, versions of which were seen in later compromise activity in April. The devices requested multiple shell script files (.sh) from rare external IP addresses. These IPs are likely associated with an operational relay box (ORB) network^[1]. The connections also occurred without a specified hostname lookup, suggesting the IPs were hardcoded into process code or already cached through unexpected running processes. One of the destination IPs was later confirmed by Palo Alto Network’s Unit 42 as associated with exploitation of the PAN-OS vulnerability^[2]. This observed activity closely resembles post-exploitation activity seen on affected firewall devices in mid-April. However, unlike the more disruptive and noisier follow-on exploitation activity seen in post-CVE-release incidents, the pre-CVE-release case observed by Darktrace appears to have been much more discreet, likely due to the relevant threat actor's desire to remain undetected.

--

Introduction

Perimeter devices such as firewalls, virtual private networks (VPNs), and intrusion prevention systems (IPS), have long been the target of adversarial actors attempting to gain access to internal networks. However, recent publications and public service announcements by leading public institutions underscore the increased emphasis threat actors are putting on leveraging such products to initiate compromises.

A blog post by the UK National Cyber Security Center (NCSC) released in early 2024 notes that as improvements are made in the detection of phishing email payloads, threat actors have again begun re-focusing efforts to exploiting network edge devices, many of which are not secure by design, as a means of breach initiation.^[i] As such, it comes as no surprise that new Common Vulnerabilities and Exposures (CVEs) are constantly discovered that exploit such internet-exposed systems.

Darktrace analysts frequently observe the impacts of such CVEs first through their investigations via Darktrace’s Security Operations Center (SOC). Beginning in April 2024, Darktrace’s SOC began handling alerts and customer requests for potential incidents involving Palo Alto Networks firewall devices. Just days prior, external researchers publicly disclosed what would later be classified as PAN-OS CVE-2024-3400, a form of remote command execution vulnerability that affects several versions of Palo Alto Networks’ firewall operating system (PAN-OS), namely PAN-OS 11.1, 11.0 and 10.2. At the time, multiple Darktrace customers were unaware of the recently announced vulnerability.

The increase in observed SOC activity for Palo Alto firewall devices, coupled with the public announcement of the new CVE prompted Darktrace researchers to look for evidence of PAN-OS exploitation on customer networks. Researchers also focused on documenting post-exploitation activity from threat actors leveraging the recently disclosed vulnerability.

As such, this blog highlights the network-based behaviors involved in the CVE-2024-3400 attack chains investigated by Darktrace’s SOC and Threat Research teams. Moreover, this investigation also provides a deeper insight into the post-compromise activities of threat actors leveraging the novel CVE. Such insights will not only prove relevant for cybersecurity teams looking to inhibit compromises in this specific instance, but also highlights general patterns of behavior by threat actors utilizing such CVEs to target internet-facing systems.

CVE-2024-3400

In mid-April 2024, the Darktrace SOC observed an uptick in activity involving recurring patterns of malicious activity from Palo Alto firewall appliances. In response to this trend, Darktrace initiated a Threat Research investigation into such activity to try and identify common factors and indicators across seemingly parallel events. Shortly before the Threat Research team opened their investigation, external researchers provided public details of CVE-2024-3400, a form of remote command execution vulnerability in the GlobalProtect feature on Palo Alto Network firewall devices running PAN-OS versions: 10.2, 11.0, and 11.1.^[ii]

In their proof of concept, security researchers at watchTowr demonstrated how an attacker can pass session ID (SESSID) values to these PAN-OS devices to request files that do not exist. In response, the system creates a zero-byte file with root privileges with the same name.^[iii] Log data is passed on devices running telemetry services to external servers through command line functionality.^[iv] Given this functionality, external actors could then request non-existent files in the SESSID containing command parameters which then be interpreted by the command line functionality.^[v] Although researchers first believed the exploit could only be used against devices running telemetry services, this was later discovered to be untrue.^[vi]

As details of CVE-2024-3400 began to surface, Darktrace’s Threat Research analysts quickly identified distinct overlaps in the observed activity on specific customer deployments and the post-exploitation behavior reported by external researchers. Given the parallels, Darktrace correlated the patterns of activity observed by the SOC team to exploitation of the newly discovered vulnerability in PAN-OS firewall appliances.

Campaign Analysis

Between the April and May 2024, Darktrace identified four main themes of post-exploitation activity involving Palo Alto Network firewall devices likely targeted via CVE-2024-3400: exploitation validation, shell command and tool retrieval, configuration data exfiltration, and ongoing command and control through encrypted channels and application protocols.

1. Exploit Validation and Further Vulnerability Enumeration

Many of the investigated attack chains began with malicious actors using out-of-band application security testing (OAST) services such as Interactsh to validate exploits against Palo Alto firewall appliances. This exploit validation activity typically resulted in devices attempting to contact unusual external endpoints (namely, subdomains of ‘oast[.]pro’, ‘oast[.]live’, ‘oast[.]site’, ‘oast[.]online’, ‘oast[.]fun’, ‘oast[.]me’, and ‘g3n[.]in’) associated with OAST services such as Interactsh. These services can be used by developers to inspect and debug internet traffic, but also have been easily abused by threat actors.

While attempted connections to OAST services do not alone indicate CVE-2024-3400 exploitation, the prevalence of such activities in observed Palo Alto firewall attack chains suggests widespread usage of these OAST services to validate initial access methods and possibly further enumerate systems for additional vulnerabilities.

Figure 1: Model alert log details showcasing a PAN-OS device making DNS queries for Interactsh domain names in what could be exploit validation, and/or further host enumeration.

2. Command and Payload Transmission

The most common feature across analyzed incidents was HTTP GET requests for shell scripts and Linux executable files (ELF) from external IPs associated with exploitation of the CVE. These HTTP requests were frequently initiated using the utilities, cURL and wget. On nearly every device likely targeted by threat actors leveraging the CVE, Darktrace analysts highlighted the retrieval of shell scripts that either featured enumeration commands, the removal of evidence of compromise activity, or commands to retrieve and start binaries on the destination device.

a) Shell Script Retrieval

Investigated devices commonly performed HTTP GET requests to retrieve shell command scripts. Despite this commonality, there was some degree of variety amongst the retrieved payloads and their affiliation with certain command tools. Several distinct types of shell commands and files were identified during the analyzed breaches. For example, some firewall devices were seen requesting .txt files associated with both Sliver C2, whose malicious use has previously been investigated by Darktrace, and Cobalt Strike. The target URIs of devices’ HTTP requests for these files included, “36shr.txt”, “2.txt”, “bin.txt”, and “data.txt”.

More interestingly, though, was the frequency with which analyzed systems requested bash scripts from rare external IP addresses, sometimes over non-standard ports for the HTTP protocol. These bash scripts would feature commands usually for the recipient system to check for certain existing files and or running processes. If the file did not exist, the system would then use cURL or wget to obtain content from external sites, change the permissions of the file, and then execute, sending output to dev/null as a means of likely defense evasion. In some scripts, the system would first make a new folder, and change directories prior to acquiring external content. Additionally, some samples highlighted multiple attempts at enumeration of the host system.

Figure 2: Packet capture (PCAP) data highlighting the incoming shell scripts providing instructions to use cURL to obtain external content, change the permissions of the file to execute, and then run the binary using the credentials and details provided.

Figure 3: PCAP data highlighting a variation of a shell script seen in an HTTP response processed by compromised devices. The script provides instructions to make a directory, retrieve and execute external content, and to hide the output.

Not every retrieved file that was not explicitly a binary featured bash scripts. Model alerts on some deployments also included file masquerading attempts by threat actors, whereby the Palo Alto firewall device would request content with a misleading extension in the URI. In one such instance, the requested URI, and HTTP response header suggests the returned content is an image/png, but the actual body response featured configuration parameters for a new daemon service to be run on the system.

Figure 4: PCAP data indicating configuration details likely for a new daemon on an investigated host. Such HTTP body content differs from the image/png extension within the request URI and declared content type in the HTTP response header.

Bash scripts analyzed across customer deployments also mirrored those identified by external security teams. External researchers previously reported on a series of identifiable shell commands in some cases of CVE-2024-3400 exploitation analyzed by their teams. Commands frequently involved a persistence mechanism they later labeled as the “UPSTYLE” backdoor.^[vii] This python-based program operates by reading commands hidden in error logs generated by 404 requests to the compromised server. The backdoor interprets the requests and writes the output to CSS files on the device. In many cases, Darktrace’s Threat Research team noted clear parallels between shell commands retrieved via HTTP GET request with those directly involving UPSTYLE. There were also matches with some URI patterns identified with the backdoor and requests observed on Darktrace deployments.

Figure 5: HTTP response data containing shell commands potentially relating to the UPSTYLE backdoor.

The presence of these UPSTYLE-related shell commands in response to Palo Alto firewall devices’ HTTP requests provides further evidence for initial exploitation of the CVE. Many bash scripts in examined cases interacted with folders and files likely related to CVE-2024-3400 exploitation. These scripts frequently sought to delete contents of certain folders, such as “/opt/panlogs/tmp/device_telemetry/minute/*” where evidence of exploitation would likely reside. Moreover, recursive removal and copy commands were frequently seen targeting CSS files within the GlobalProtect folder, already noted as the vulnerable element within PAN-OS versions. This evidence is further corroborated by host-based forensic analysis conducted by external researchers.^[viii]

Figure 6: PCAP data from investigated system indicating likely defense evasion by removing content on folders where CVE exploitation occurred.

b) Executable File Retrieval

Typically, following command processing, compromised Palo Alto firewall devices proceeded to make web requests for several unusual and potentially malicious files. Many such executables would be retrieved via processed scripts. While there a fair amount of variety in specific executables and binaries obtained, overall, these executables involved either further command tooling such as Sliver C2 or Cobalt Strike payloads, or unknown executables. Affected systems would also employ uncommon ports for HTTP connections, in a likely attempt to evade detection. Extensions featured within the URI, when visible, frequently noted ‘.elf’ (Linux executable) or ‘.exe’ payloads. While most derived hashes did not feature identifiable open-source intelligence (OSINT) details, some samples did have external information tying the sample to specific malware. For example, one such investigation featured a compromised system requesting a file with a hash identified as the Spark malware (backdoor) while another investigated case included a host requesting a known crypto-miner.

Figure 7: PCAP data highlighting compromised system retrieving ELF content from a rare external server running a simple Python HTTP server.

Figure 8: Darktrace model alert logs highlighting a device labeled “Palo Alto” making a HTTP request on an uncommon port for an executable file following likely CVE exploitation.

3. Configuration Data Exfiltration and Unusual HTTP POST Activity

During Darktrace’s investigations, there were also several instances of sensitive data exfiltration from PAN-OS firewall devices. Specifically, targeted systems were observed making HTTP POST requests via destination port 80 to rare external endpoints that OSINT sources associate with CVE-2024-3400 exploitation and activity. PCAP analysis of such HTTP requests revealed that they often contained sensitive configuration details of the targeted Palo Alto firewall devices, including the IP address, default gateway, domain, users, superusers, and password hashes, to name only a few. Threat actors frequently utilized Target URIs such as “/upload” in their HTTP POST requests of this multi-part boundary form data. Again, the User-Agent headers of these HTTP requests largely involved versions of cURL, typically 7.6.1, and wget.

Figure 9: PCAP datahighlighting Palo Alto Firewall device running vulnerable version of PAN-OSposting configuration details to rare external services via HTTP.

Figure 10: Model alert logs highlighting a Palo Alto firewall device performing HTTP POSTs to a rare external IP, without a prior hostname lookup, on an uncommon port using a URI associated with configuration data exfiltration across analyzed incidents

Figure 11: Examples of TargetURIs of HTTP POST requests involving base64 encoded IPs and potential dataegress.

4. Ongoing C2 and Miscellaneous Activity

Lastly, a smaller number of affected Palo Alto firewall devices were seen engaging in repeated beaconing and/or C2 communication via both encrypted and unencrypted protocols during and following the initial series of kill chain events. Such encrypted channels typically involved protocols such as TLS/SSL and SSH. This activity likely represented ongoing communication of targeted systems with attacker infrastructure. Model alerts typically highlighted unusual levels of repeated external connectivity to rare external IP addresses over varying lengths of time. In some investigated incidents, beaconing activity consisted of hundreds of thousands of connections over several days.

Figure 12: Advanced search details highlighting high levels of ongoing external communication to endpoints associated with C2 infrastructure exploiting CVE-2024-3400.

Some beaconing activity appears to have involved the use of the WebSocket protocol, as indicated by the appearance of “/ws” URIs and validated within packet captures. Such connections were then upgraded to an encrypted connection.

Figure 13: PCAP highlighting use of WebSocket protocol to engage in ongoing external connectivity to likely C2 infrastructure following CVE-2024-3400 compromise.

While not directly visible in all the deployments, some investigations also yielded evidence of attempts at further post-exploitation activity. For example, a handful of the analyzed binaries that were downloaded by examined devices had OSINT information suggesting a relation to crypto-mining malware strains. However, crypto-mining activity was not directly observed at this time. Furthermore, several devices also triggered model alerts relating to brute-forcing activity via several authentication protocols (namely, Keberos and RADIUS) during the time of compromise. This brute-force activity likely represented attempts to move laterally from the affected firewall system to deeper parts of the network.

Figure 14: Model alert logs noting repeated SSL connectivity to a Sliver C2-affiliated endpoint in what likely constitutes C2 connectivity.

Figure 15: Model alert logs featuring repeated RADIUS login failures from a compromised PAN-OS device using generic usernames, suggesting brute-force activity.

Conclusion

Between April and late May 2024, Darktrace’s SOC and Threat Research teams identified several instances of likely PAN-OS CVE-2024-3400 exploitation across the Darktrace customer base. The subsequent investigation yielded four major themes that categorize the observed network-based post-exploitation activity. These major themes were exploit validation activity, retrieval of binaries and shell scripts, data exfiltration via HTTP POST activity, and ongoing C2 communication with rare external endpoints. The insights shared in this article will hopefully contribute to the ongoing discussion within the cybersecurity community about how to handle the likely continued exploitation of this vulnerability. Moreover, this article may also help cybersecurity professionals better respond to future exploitation of not only Palo Alto PAN-OS firewall devices, but also of edge devices more broadly.

Threat actors will continue to discover and leverage new CVEs impacting edge infrastructure. Since it is not yet known which CVEs threat actors will exploit next, relying on rules and signatures for the detection of exploitation of such CVEs is not a viable approach. Darktrace’s anomaly-based approach to threat detection, however, is well positioned to robustly adapt to threat actors’ changing methods, since although threat actors can change the CVEs they exploit, they cannot change the fact that their exploitation of CVEs results in highly unusual patterns of activity.

Credit to Adam Potter, Cyber Analyst, Sam Lister, Senior Cyber Analyst

Appendices ‍

Pre-CVE-Release IoCs

38.54[.]104[.]14/3.sh
154.223[.]16[.]34/1.sh
154.223[.]16[.]34/co.sh
38.54[.]104[.]14/

Indicators of Compromise

Indicator – Type – Description

94.131.120[.]80 IP C2 Endpoint

94.131.120[.]80:53/?src=[REDACTED]=hour=root URL C2/Exfiltration Endpoint

134.213.29[.]14/?src=[REDACTED]min=root URL C2/Exfiltration Endpoint

134.213.29[.]14/grep[.]mips64 URL Payload

134.213.29[.]14/grep[.]x86_64 URL Payload

134.213.29[.]14/?deer URL Payload

134.213.29[.]14/?host=IDS URL Payload

134.213.29[.]14/ldr[.]sh URL Payload

91ebcea4e6d34fd6e22f99713eaf67571b51ab01 SHA1 File Hash Payload

185.243.115[.]250/snmpd2[.]elf URL Payload

23.163.0[.]111/com URL Payload

80.92.205[.]239/upload URL C2/Exfiltration Endpoint

194.36.171[.]43/upload URL C2/Exfiltration Endpoint

update.gl-protect[.]com Hostname C2 Endpoint

update.gl-protect[.]com:63869/snmpgp URL Payload

146.70.87[.]237 IP address C2 Endpoint

146.70.87[.]237:63867/snmpdd URL Payload

393c41b3ceab4beecf365285e8bdf0546f41efad SHA1 File Hash Payload

138.68.44[.]59/app/r URL Payload

138.68.44[.]59/app/clientr URL Payload

138.68.44[.]59/manage URL Payload

72.5.43[.]90/patch URL Payload

217.69.3[.]218 IP C2 Endpoint

5e8387c24b75c778c920f8aa38e4d3882cc6d306 SHA1 File Hash Payload

217.69.3[.]218/snmpd[.]elf URL Payload

958f13da6ccf98fcaa270a6e24f83b1a4832938a SHA1 File Hash Payload

6708dc41b15b892279af2947f143af95fb9efe6e SHA1 File Hash Payload

dc50c0de7f24baf03d4f4c6fdf6c366d2fcfbe6c SHA1 File Hash Payload

109.120.178[.]253:10000/data[.]txt URL Payload

109.120.178[.]253:10000/bin[.]txt URL Payload

bc9dc2e42654e2179210d98f77822723740a5ba6 SHA1 File Hash Payload

109.120.178[.]253:10000/123 URL Payload

65283921da4e8b5eabb926e60ca9ad3d087e67fa SHA1 File Hash Payload

img.dxyjg[.]com/6hiryXjZN0Mx[.]sh URL Payload

149.56.18[.]189/IC4nzNvf7w/2[.]txt URL Payload

228d05fd92ec4d19659d71693198564ae6f6b117 SHA1 File Hash Payload

54b892b8fdab7c07e1e123340d800e7ed0386600 SHA1 File Hash Payload

165.232.121[.]217/rules URL Payload

165.232.121[.]217/app/request URL Payload

938faec77ebdac758587bba999e470785253edaf SHA1 File Hash Payload

165.232.121[.]217/app/request63 URL Payload

165.232.121[.]217:4443/termite/165.232.121[.]217 URL Payload

92.118.112[.]60/snmpd2[.]elf URL Payload

2a90d481a7134d66e8b7886cdfe98d9c1264a386 SHA1 File Hash Payload

92.118.112[.]60/36shr[.]txt URL Payload

d6a33673cedb12811dde03a705e1302464d8227f SHA1 File Hash Payload

c712712a563fe09fa525dfc01ce13564e3d98d67 SHA1 File Hash Payload

091b3b33e0d1b55852167c3069afcdb0af5e5e79 SHA1 File Hash Payload

5eebf7518325e6d3a0fd7da2c53e7d229d7b74b6 SHA1 File Hash Payload

183be7a0c958f5ed4816c781a2d7d5aa8a0bca9f SHA1 File Hash Payload

e7d2f1224546b17d805617d02ade91a9a20e783e SHA1 File Hash Payload

e6137a15df66054e4c97e1f4b8181798985b480d SHA1 File Hash Payload

95.164.7[.]33:53/sea[.]png URL Payload

95.164.7[.]33/rules URL Payload

95.164.7[.]33:53/lb64 URL Payload

c2bc9a7657bea17792048902ccf2d77a2f50d2d7 SHA1 File Hash Payload

923369bbb86b9a9ccf42ba6f0d022b1cd4f33e9d SHA1 File Hash Payload

52972a971a05b842c6b90c581b5c697f740cb5b9 SHA1 File Hash Payload

95d45b455cf62186c272c03d6253fef65227f63a SHA1 File Hash Payload

322ec0942cef33b4c55e5e939407cd02e295973e SHA1 File Hash Payload

6335e08873b4ca3d0eac1ea265f89a9ef29023f2 SHA1 File Hash Payload

134.213.29[.]14 IP C2 Endpoint

185.243.115[.]250 IP C2 Endpoint

80.92.205[.]239 IP C2 Endpoint

194.36.171[.]43 IP C2 Endpoint

92.118.112[.]60 IP C2 Endpoint

109.120.178[.]253 IP C2 Endpoint

23.163.0[.]111 IP C2 Endpoint

72.5.43[.]90 IP C2 Endpoint

165.232.121[.]217 IP C2 Endpoint

8.210.242[.]112 IP C2 Endpoint

149.56.18[.]189 IP C2 Endpoint

95.164.7[.]33 IP C2 Endpoint

138.68.44[.]59 IP C2 Endpoint

Img[.]dxyjg[.]com Hostname C2 Endpoint

Darktrace Model Alert Coverage

· Anomalous Connection / New User Agent to IP Without Hostname

· Device / New User Agent (triggered by pre-CVE-release activity)

· Anomalous File / Script from Rare External Location (triggered by pre-CVE-release activity)

· Anomalous File / Masqueraded File Transfer

· Anomalous File / EXE from Rare External Location

· Anomalous File / Multiple EXE from Rare External Locations

· Anomalous File / Script and EXE from Rare External

· Anomalous File / Suspicious Octet Stream Download

· Anomalous File / Numeric File Download

· Anomalous Connection / Application Protocol on Uncommon Port

· Anomalous Connection / Posting HTTP to IP Without Hostname

· Anomalous Connection / Multiple Failed Connections to Rare Endpoint

· Anomalous Connection / Suspicious Self-Signed SSL

· Anomalous Connection / Anomalous SSL without SNI to New External

· Anomalous Connection / Multiple Connections to New External TCP Port

· Anomalous Connection / Rare External SSL Self-Signed

· Anomalous Server Activity / Outgoing from Server

· Anomalous Server Activity / Rare External from Server

· Compromise / SSH Beacon

· Compromise / Beacon for 4 Days

· Compromise / Sustained TCP Beaconing Activity To Rare Endpoint

· Compromise / High Priority Tunnelling to Bin Services

· Compromise / Sustained SSL or HTTP Increase

· Compromise / Connection to Suspicious SSL Server

· Compromise / Suspicious File and C2

· Compromise / Large Number of Suspicious Successful Connections

· Compromise / Slow Beaconing Activity To External Rare

· Compromise / HTTP Beaconing to New Endpoint

· Compromise / SSL or HTTP Beacon

· Compromise / Suspicious HTTP and Anomalous Activity

· Compromise / Beacon to Young Endpoint

· Compromise / High Volume of Connections with Beacon Score

· Compromise / Suspicious Beaconing Behaviour

· Compliance / SSH to Rare External Destination

· Compromise / HTTP Beaconing to Rare Destination

· Compromise / Beaconing Activity To External Rare

· Device / Initial Breach Chain Compromise

· Device / Multiple C2 Model Breaches

MITRE ATTACK Mapping

Tactic – Technique

Initial Access T1190 – Exploiting Public-Facing Application

Execution T1059.004 – Command and Scripting Interpreter: Unix Shell

Persistence T1543.002 – Create or Modify System Processes: Systemd Service

Defense Evasion T1070.004 – Indicator Removal: File Deletion

Credential Access T1110.001 – Brute Force: Password Guessing

Discovery T1083 – File and System Discovery

T1057 – Process Discovery

Collection T1005 – Data From Local System

Command and Control

T1071.001 – Application Layer Protocol: Web Protocols

T1573.002 – Encrypted Channel: Asymmetric Cryptography

T1571 – Non-Standard Port

T1105 – Ingress Tool Transfer

Exfiltration

T1041 – Exfiltration over C2 Protocol

T1048.002 - Exfiltration Over Alternative Protocol: Exfiltration Over Asymmetric Encrypted Non-C2 Protocol

References

[1] https://cloud.google.com/blog/topics/threat-intelligence/china-nexus-espionage-orb-networks

[2] https://unit42.paloaltonetworks.com/cve-2024-3400/

^[i] https://www.ncsc.gov.uk/blog-post/products-on-your-perimeter

^[ii] https://security.paloaltonetworks.com/CVE-2024-3400

^[iii] https://labs.watchtowr.com/palo-alto-putting-the-protecc-in-globalprotect-cve-2024-3400/

^[iv] https://labs.watchtowr.com/palo-alto-putting-the-protecc-in-globalprotect-cve-2024-3400/

^[v] https://labs.watchtowr.com/palo-alto-putting-the-protecc-in-globalprotect-cve-2024-3400/

^[vi] https://security.paloaltonetworks.com/CVE-2024-3400

^[vii] https://www.volexity.com/blog/2024/04/12/zero-day-exploitation-of-unauthenticated-remote-code-execution-vulnerability-in-globalprotect-cve-2024-3400/

^[viii] https://www.volexity.com/blog/2024/05/15/detecting-compromise-of-cve-2024-3400-on-palo-alto-networks-globalprotect-devices/

‍

Written by

Adam Potter

Senior Cyber Analyst

Inside the SOC

Written by

Adam Potter

Senior Cyber Analyst

•

July 9, 2026

Angel Arribas Lopez

Associate Principal Cyber Analyst

•

July 17, 2026

Watch the NIS2 Webinar

Blog

July 17, 2026

AI Is Taking on Stadium Operations. How Can Security Teams Keep it Protected?

How to Secure AI in Stadium Operations

Key takeaways

AI is entering high-impact stadium functions such as access control, crowd management, ticketing, facilities, and surveillance.
Shadow AI and third-party AI use can create risks that stadium security teams cannot readily see.
Security teams must understand not only which AI systems exist, but also what they can access and what actions they can take.
Live-event resilience requires continuous monitoring and response across AI, IT, OT, identities, and third parties.

Modern stadiums are infrastructure unlike any other. I’ve written before on event day sparking stadiums into life with shops and food stands, transport hubs, vast telecommunications infrastructure, field-side technology and beyond, acting as one super-sized, connected ecosystem. Stadiums’ scale and complexity make them some of the toughest environments in cybersecurity. Now, we’re adding AI to those operations and bringing a new dimension of risk.

The benefits of AI in stadium operations are easy to see. It can help stadium operators move fans safely through crowded gates, forecast demand at concession stands, support biometric entry, identify suspicious behavior on CCTV, and manage heating and ventilation. Used well, it can make live events safer, faster, and more efficient.

But it also changes the security model.

In Darktrace’s recent research into the threat landscape surrounding sports, we asked cybersecurity professionals protecting professional sports organizations where in their footprint a cyber compromise would have the greatest impact. The area they named most, highlighted by 34% of the professionals we spoke to, was stadium operations. At the same time, 35% said their organizations are already using AI in stadium operations, or plan to do so in the next 12 months.

Security teams are no longer just protecting traditional IT systems around a stadium. They are increasingly being asked to protect AI systems that are operating in the stadium’s most fundamental functions.

Approved AI vs. shadow AI in stadium operations

There is a clear difference between AI a stadium’s security team knows about and AI it does not.

Approved AI is the AI that has been reviewed, tested, and integrated into the venue’s operating environment. It may support CCTV analytics, access control, facility management, ticketing, logistics, broadcast operations, or anti-piracy monitoring. It should have clear ownership, access controls, logging, vendor review, and data protection rules. That does not make it risk-free, but it allows security teams to institute proper governance.

Shadow AI is different. It is the unapproved use of AI tools by employees, contractors, or suppliers. It often starts with good intent. Someone wants to work faster. A staff member pastes internal information into a public AI tool to draft a briefing. A developer uses an AI assistant to debug ticketing code. A supplier connects an AI scheduling tool to delivery routes. A designer uploads unreleased venue plans or sponsor material to generate a mockup.

None of those actions may feel like a security decision to the person doing them. But each one can move sensitive operational data into an environment the stadium does not control, creating hidden risk.

The approved AI stack may be visible to security teams. The shadow AI stack often is not.

Why game day increases AI cybersecurity risk

In a typical enterprise environment, a security team may have hours to investigate a strange login or an unexpected connection to a third-party service. Within a stadium, the moment an incident is likely to occur is also the moment when teams are at their most stretched and the incident can have the greatest repercussions: game day.

If an AI system used for crowd management behaves unexpectedly, the issue is not only technical. It may affect physical movement inside the venue.

If a supplier tool is sending operational data to an unapproved AI platform, the issue is not only data governance. It may expose delivery routes, restricted access schedules, or staffing plans.

The most dangerous scenario is not always a loud, dramatic attack but a hidden dependency that no one has mapped such as a vendor adding an AI feature through a software update or a staff workflow using an unapproved tool.

By the time the venue is live, those hidden connections can become operational risk.

The supply chain is part of the stadium attack surface

Any major sporting event is made by its supply chain and partnerships: catering firms, transport providers, broadcast systems, facilities teams. Every piece is necessary and each creates a security channel. The risk of supply chain compromise has been well established for some time and has been the source of some of the most high-profile breaches we’ve seen. The data breach at MSG Entertainment, owner of Madison Square Garden, that was widely reported in March, originated in a breach of Oracle’s E-Business Suite, used in MSG Entertainment’s back-office systems, while the 2018 Olympic Destroyer attack on the Pyeongchang Winter Olympics reportedly began with the compromise of the main IT service provider for the Games. The addition of AI is heightening the risk.

A stadium can have strict rules for its own AI systems, but its vendors may be using separate tools. Some may use AI to manage staffing, delivery windows, inventory, or customer communications. Others may not realize that AI features have been added into software they already use.

This is one of the hardest parts of securing AI in stadium operations. The risk does not always come from a tool the venue selected. It may come from a tool a supplier selected or a feature the supplier did not know had been turned on.

Security teams need to treat vendor AI the same way they treat vendor access. They need to know what suppliers can connect to, what data they can see, what tools they use, and whether those tools introduce new routes for data exposure or lateral movement.

A third-party AI tool does not need deep access to create risk. Sometimes it only needs the right operational detail at the wrong time.

Four questions for securing AI in stadium operations

As AI becomes part of stadium operations, security teams need to move beyond basic approval lists. There are four questions they need to ask:

1. Where is AI being used?

This includes obvious tools, such as computer vision, access control, ticketing, logistics, and facility management. But it also includes less visible AI inside SaaS platforms, vendor tools, browser extensions, developer workflows, smart building systems, and collaboration tools.

2. What can the AI access?

Can it see incident logs, staffing plans, ticketing data, video feeds, building controls, fan information, credentials, or supplier systems? Can it only analyze information, or can it also trigger actions?

3. What can the AI do?

AI agents are not just passive tools. Some can call APIs, update records, generate instructions, trigger workflows, or act with the permissions of a user or service account. In a stadium, that distinction is critical. There is a big difference between an AI system that recommends an action and one that can take an action.

4. What does normal look like?

In your security architecture, static rules will not be enough. AI use changes quickly: tools appear inside existing platforms, vendors add new services, and staff find workarounds when they are under pressure. Security teams need to understand normal behavior across people, identities, devices, networks, cloud services, suppliers, and AI tools so they can spot when something changes.

That is especially important in live-event environments, where small anomalies can matter. A connection to an unapproved AI service may be harmless in one context and serious in another, and an AI agent taking action at 3 a.m. may be expected during setup but suspicious during a match. Context is what turns raw activity into useful security insight. It’s also what enables rapid response. Your own AI-based security systems can respond to threats at machine speed if they can build the live context to know action needs to be taken.

AI can make stadiums safer, but only if it is secured

AI has a real role to play in stadium operations. It can help teams detect crowd pressure earlier, reduce bottlenecks, manage facilities more efficiently, improve the fan experience, and support event teams during high-pressure moments.

The answer is not to slow all AI adoption. That's not the goal. The answer is to make AI visible, governed, and secure before it becomes part of match-day operations.

For stadium operators and event organizers, that means mapping AI use across the venue and supplier ecosystem. It means understanding what each AI system can access and what actions it can take. It means giving staff approved tools that meet their needs, rather than leaving them to find workarounds. It means writing AI use into vendor contracts and audits. And it means monitoring behavior across the full environment, not only the systems that are easiest to see. A stadium cannot secure what it cannot see.

When AI becomes part of how a stadium moves people, controls access, manages facilities, supports suppliers, and protects media rights, it stops being a side project. It becomes part of the event infrastructure.

Event infrastructure must be thoroughly prepared before venue gates open and sustained with the operational resilience required to support a secure, seamless, and reliable event experience.

How Darktrace helps secure AI in stadium operations

Darktrace brings more than a decade of behavioral AI expertise, built on an enterprise‑wide platform designed to operate in complex, ambiguous environments. We protect the large-scale integrated IT and OT environments that underpin stadium operations from the 2022 FIFA World Cup in Qatar, to Formula 1 Grand Prixes around the world and stadiums across the USA.

Other cybersecurity technologies try to predict each new attack based on historical attacks. The problem is that AI operates like humans do. Every action introduces new information that changes how AI behaves, making it unpredictable in nature. Historical attack tactics are now only a small part of the equation, forcing vendors to retrofit unproven acquisitions to secure AI.  

Darktrace is fundamentally different. Our Adaptive AI continuously learns how your people and AI behave, building an understanding of your organization so it can detect and respond autonomously when behavior deviates. Our Behavioral Defense Platform secures your AI, people, and infrastructure as you onboard new workflows, agents, and applications, enabling your AI transformation at scale.

As AI changes what organizations can do, Darktrace helps them move forward with confidence. We give the security teams defending the people and technology within stadium infrastructure the understanding, visibility, and autonomous action they need to protect new technologies as they are integrated into operations, so their organizations drive the progress that will define the AI era.

[related-resource]

About the author

Blog

July 15, 2026

Security After Signatures: Operating in a World of Pre‑CVE Disclosure Exploitation, Collapsed Trust Boundaries, and Autonomous Systems

Three shifts have reshaped what it means to defend an enterprise securely.

First, exploitation often begins before defenders have a Common Vulnerabilities and Exposures (CVE) identifier, a security advisory, or an entry in the Cybersecurity and Infrastructure Security Agency's (CISA) Known Exploited Vulnerabilities (KEV) catalog.

Secondly, the trust boundary has moved beyond the network edge into identities, tokens, APIs, and Software-as-a-Service (SaaS) workflows.

Third, an increasing share of business activity is executed through automation, integrations, and AI agent-like systems that can act faster than teams can verify intent.

If your security model still relies on detecting known bad artefacts, triaging isolated alerts, and waiting for confirmation before acting, you are already behind the threat.

This is not a failure of security teams; it’s a failure of the operating model to keep pace with how the environment has changed.

A SOC built around alerts and signatures assumes that malicious activity will eventually surface as an event. In real incidents, however, the decisive evidence is rarely a single event. Instead, it is a chain of individually explainable actions that only appears malicious once you connect the dots across identity, non-human identity, cloud, email, SaaS, operational technology (OT), and network telemetry.

The defenders succeeding today observe behaviors, link them into sequences, understand what those sequences mean, and contain impact before the full story unfolds. That is the operating model the current threat environment demands.

Exploitation before disclosure

The first shift is the straightforward: the time to exploit has dropped to nearly zero.

In one example, Darktrace observed a sequence of subtle but strategically significant anomalies within a customer environment that later aligned with exploitation of CVE‑2025‑0994 in Trimble Cityworks by likely Chinese-nexus threat actors. Behavioral indicators were visible at least 18 days before public disclosure, with related anomalies emerging 40 to 50 days earlier during the intrusion window.

This case illustrates a familiar pattern: clusters of weak‑signal anomalies combing to form an actionable picture of intrusion long before a CVE is published. Such activity reflects long‑horizon, option‑preserving operator models often associated with mature state‑linked activity.

Figure 1: Darktrace’s detection of malicious exploitation of CVE 2025-0994, later tied to Chinese-nexus threat actors targeting critical national infrastructure (CNI) in the US, weeks before public disclosure.

Throughout 2025 and 2026, Darktrace has continued to observe the value of anomaly-based detections across a range of incidents.

CVE	CVE Public Disclosure Date	Darktrace Detection Date	Days Between Detection of Exploitation and CVE Public Disclosure
CVE 2025 0994 (Trimble City Works)	2025-02-06	2025-01-19	18 Days
CVE 2025-24183 (Apache)	2025-03-10	2025-02-18	20 days
CVE 2025-10035 (Fortra GoAnywhere)	2025-09-18	2025-09-11	7 days

Identity is the real control plane

The second shift is that identity has replaced perimeter as the primary control plane. As Darktrace’s Annual Threat Report 2026 illustrated, identity remains the main challenge in defending against modern intrusions. A clear example is the Adversary-in-the-Middle (AiTM) case published by Darktrace in December 2025. A phishing email led to the compromise of an Office 365 account. Session hijacking bypassed multi-factor authentication (MFA), and the compromised account was used for follow-on phishing and persistence activities including the creation of malicious email rules.

Every step in that sequence mattered. A successful login alone does not prove legitimacy. An inbox rule, on its own, may not appear catastrophic. Mail activity, viewed in isolation, may seem operationally normal. But the behavioral chain tells a different story: credential theft, token abuse, persistence, and onward compromise through a trusted identity.

This is why the question is no longer “Did the user authenticate successfully”. The more important question is, “Does this identity action make sense right now, in this context, given what came before it?” The AiTM case shows how identity can be compromised. In practice, however, attacks rarely remained confined to identity alone.

In another Darktrace case, a compromised SaaS account triggered activity across the email, SaaS, and network layers, including inbox rule changes, phishing propagation, and connections to suspicious infrastructure. Viewed in isolation, none of these events were decisive. Together, however, they formed a behavioral sequence that revealed the intrusion, with the full attack story automatically correlated and surfaced to defenders by Darktrace’s Cyber AI Analyst.

Figure 2: Cyber AI Analyst correlated and appended additional events to the incident, including other users who connected to the suspicious redirect link after outbound phishing emails were sent.

AI accelerates the threat

The third shift is the one many teams still underestimate: trusted tooling, integrations, and AI agent-like systems can create actions that appear legitimate but are strategically dangerous.

The shift becomes clearer when examining how governments are now framing AI risk. In 2026, guidance published by CISA, UK’s National Cyber Security Centre (NCSC) and Five Eyes partners warned that agentic systems expand attack surfaces, accumulate privilege, and can behave in ways that are difficult to predict or explain [1]. The advice is simple: assume unexpected behavior and design controls around it.

The real risk is not AI usage. It is unknown autonomy: systems with credentials, data access, and action paths that can execute workflow steps without sufficient behavioral validation, traceability, or human oversight. Darktrace’s Model Context Protocol (MCP) risk analysis provides a useful framework for understanding this challenge. Over-privileged agents, content injection, and tool abuse become high-consequence risks when connected systems can dynamically retrieve data, execute actions, and communicate externally.

Whether security teams like it or not, AI is already in the enterprise. It will help drive innovation, but it will also be abused, whether accidentally or maliciously. In each of the cases below, AI either scaled the attacker, built the tooling, or existed within the environment as something to exploit or misuse.

1. AI as an Attack Multiplier

In one campaign targeting Mexican government entities, a single operator used commercial AI platforms to generate exploits, automate reconnaissance, and process large volumes of data, compressing work that would traditionally have required an entire team into a single workflow [2].

Darktrace is also observing this trend further down the stack. In one case, Darktrace identified AI-generated malware exploiting React2Shell, where an attacker used a Large Language Model (LLM) to produce working exploit code and deploy it at scale.

[darktrace.com], [darktrace.com] ‍

2. AI as an Attack Surface

Attempted AI exploitation is now appearing within customer environments. In one case involving an automation technology manufacturer, a compromised LLM proxy was seemingly used as a stepping stone to access additional AI services. When that attempt failed, the attacker pivoted to cryptomining.

What is clear is that the AI layer has already become an asset worth probing, exploiting, and pivoting through. It is also clear that defenders benefit from rapidly understanding how these activities connect. In this case, Cyber AI Analyst automatically pieced together the intrusion, while Darktrace’s Managed Threat Detection service alerted to the customer, enabling the activity to be contained before it could progress further.

Figure 3: Cyber AI Analyst's investigation into a compromised LLM proxy that was abused for cryptomining activity.

AI as a trusted but dangerous actor

This does not require a cinematic vision of “rogue AI.” The Salesloft incident provides a more grounded example, where AI and automation operate with legitimate access but served malicious intent. In that case, attackers abused compromised OAuth tokens associated with the Drift AI chat agent to export significant volumes of data from Salesforce environments.

The activity resembled legitimate API usage and relied on trusted SaaS integrations rather than malware or other obvious signs of intrusion. That is precisely the challenge. Traditional security controls are good at detecting forced entry, but far less effective when a trusted application integration behaves in a way that is technically permitted yet operationally harmful.

In these scenarios, the security challenge shifts from validating access to validating behavior.

This is what that looks like in practice: AI-linked identities executing legitimate actions that require behavioral validation rather than access validation.

Figure 4: Darktrace / SECURE AI highlights anomalous activity across AI identities, surfacing critical behavior that requires validation and containment.

Early observations from Darktrace / SECURE AI deployments reinforce this reality. Across Darktrace's observed fleet, AI service connections per deployment increased 13% during the first half of 2026, reaching over 16 million connections overall. The typical organisation now interacts with seven different AI providers, evidence that AI is no longer operating at the edges of the enterprise. It is increasingly woven into day-to-day business activity.

The most common risks are not compromised models or advanced AI attacks. Instead, they stem from employees and business functions exposing sensitive information through entirely legitimate-looking interactions. Darktrace has observed repeated submission of personally identifiable information (PII), tax information, identification documents, and medical data into LLM prompts, alongside widespread use of unsanctioned (shadow) AI services and growing AI activity from mobile devices.

For defenders, the challenge is increasingly one of context: understanding when legitimate business use crosses into material risk, while preserving privacy and user trust.

Conclusion

Across all three shifts, the pattern is the same: behavior precedes understanding. Security teams are not losing because adversaries have become invisible. An increasingly outdated security model assumes that malicious activity will reveal itself cleanly and early. It no longer does.

In 2026 and beyond, defenders win by understanding behavioral sequences, continuously validating trust, and acting before certainty becomes hindsight. That is security after signatures. That is security in the AI era.

Credit to: Daniel Levy, Threat Hunting Data Scientist

Edited by: Ryan Traill, Content Manager

References

[1] https://www.cyber.gov.au/business-government/secure-design/artificial-intelligence/careful-adoption-of-agentic-ai-services

[2]https://www.latimes.com/business/story/2026-02-26/hacker-used-anthropics-claude-ai-to-steal-mexican-government-data

About the author

Nathaniel Jones

VP, Security & AI Strategy, Field CISO

Your data. Our AI.

Elevate your network security with Darktrace AI

Get a demo

Check out this article by Darktrace: Post-Exploitation Activities on PAN-OS Devices: A Network-Based Analysis

Post-Exploitation Activities on PAN-OS Devices: A Network-Based Analysis

Introduction

CVE-2024-3400

Campaign Analysis

1. Exploit Validation and Further Vulnerability Enumeration

2. Command and Payload Transmission

a) Shell Script Retrieval

b) Executable File Retrieval

3. Configuration Data Exfiltration and Unusual HTTP POST Activity

4. Ongoing C2 and Miscellaneous Activity

Conclusion

Appendices ‍

Pre-CVE-Release IoCs

Indicators of Compromise

Darktrace Model Alert Coverage

MITRE ATTACK Mapping

References

When AI Infrastructure Becomes Part of the Attack Surface

AI Is Taking on Stadium Operations. How Can Security Teams Keep it Protected?

Enjoying the blog?

More in this series

From Click to Command: Behavioral Detection of AppleScript-Led MacOS Intrusions

Hola VPN Abuse: From Proxy Traffic to Malware and Cryptomining

How Attackers Abuse the Chinese Nezha Monitoring Tool

Blog

July 17, 2026

AI Is Taking on Stadium Operations. How Can Security Teams Keep it Protected?

How to Secure AI in Stadium Operations

Approved AI vs. shadow AI in stadium operations

Why game day increases AI cybersecurity risk

The supply chain is part of the stadium attack surface

Four questions for securing AI in stadium operations

AI can make stadiums safer, but only if it is secured

How Darktrace helps secure AI in stadium operations

Blog

July 15, 2026

Security After Signatures: Operating in a World of Pre‑CVE Disclosure Exploitation, Collapsed Trust Boundaries, and Autonomous Systems

Exploitation before disclosure

Identity is the real control plane

AI accelerates the threat

1. AI as an Attack Multiplier

2. AI as an Attack Surface

AI as a trusted but dangerous actor

Conclusion

References