Many teams now buy "confidential" AI inference because the prompts run inside a hardware enclave that even the provider cannot read. On 9 November 2026, NEAR AI disclosed that an attacker exploited a remote code execution bug in SGLang, the inference engine running inside that enclave, and could have observed confidential requests on the server it reached.
AG Tech Consulting view: if you run or buy AI inference, put the serving layer (inference engine, GPU cluster manager, the proxy in front of them) on your patch and pen-test list with the same priority as your VPN and email gateway. The enclave protects the box. It does not fix the software inside it.
Two incidents, one week, same layer
This week gave us two reminders, and neither was about the model.
The first is NEAR AI's incident report on SGLang. The second is CVE-2026-102895, an authentication bypass in GPUStack, an open-source GPU cluster manager for AI model serving. CISA added it to its Known Exploited Vulnerabilities (KEV) catalogue on 9 November, with a federal remediation due date of 12 November.
Let us be very clear about what links them. In both cases the attacker went after the software that serves models: the engine that turns prompts into tokens, and the control plane that decides which models run on which GPUs. That layer sees plaintext prompts, holds credentials, and can start processes on GPU hosts.
What happened at NEAR AI
According to NEAR AI's own report, the attacker exploited a remote code execution vulnerability in SGLang and got code execution on one inference server. From there, they reached the rest of the infrastructure.
The impact NEAR AI has reported so far:
Account data. The attacker reached an internal system behind Private Chat, a product the company had already deprecated, and downloaded account details for approximately 100 users: email addresses, names and, for some accounts, plan and balance information.
Agents outside the enclave. They accessed 18 agents on a legacy deployment of NEAR AI Agents that was not running in the confidential computing (TEE) environment. Chat history, settings and logs were taken from those agents.
Keys in logs. Two Cloud customers had NEAR AI Cloud API keys exposed in logs. Both keys have been revoked.
Confidential requests. On 8 November, the attacker could also have observed user requests being processed on the accessed server, including requests running in confidential mode. NEAR AI says it found no indication that stored Private Chat conversations or uploaded files were accessed.
NEAR AI's response was thorough: it revoked all active sessions and administrative credentials, replaced all agent tokens, took Private Chat offline, deployed an SGLang fix across all inference servers, rotated the remaining credentials and completed a forensic review. It reported the bug to the SGLang project.
The enclave did its job. That was not enough
The most important line in the report is this one: "Confidential computing protects data from the operator and the host. It does not protect against a vulnerability in the software running inside the environment, and that is what happened here."
The point is simple. A TEE keeps the cloud operator, the hypervisor and the host OS out. Attestation can prove which code is running inside. Neither tells you that the code is free of bugs. When the inference engine itself is compromised, the attacker is inside the trust boundary, sitting next to the plaintext prompts.
NEAR AI has drawn the right lesson. It says it will now treat "the inference software and the services around the enclave with the same rigor as the enclave itself". Every buyer of confidential inference should ask its provider the same question.
Which SGLang bug? Nobody has said
NEAR AI has not named a CVE or the affected SGLang versions, and has not said whether the project has shipped a fix. So we will not guess.
What we can say is that SGLang has a public track record this year. In April, CERT/CC published VU#915947 (CVE-2026-5760), a remote code execution issue triggered by a malicious model file. CERT/CC said no response or patch was obtained during coordination.
In July, CERT/CC published VU#281278 covering six more SGLang CVEs (CVE-2026-15969, -15971, -15974, -15976, -15977 and -15978). Together they cover remote code execution, server-side request forgery and local file read, exposure of API key information, and model weight exfiltration, mostly through administrative and weight-management endpoints.
CERT/CC noted that most of these did not require authentication, and that no patches were available at publication. Its mitigations: keep SGLang off untrusted networks, segment it, change the unsafe inter-process communication default it flagged, and disable endpoints you do not use.
GPUStack: "localhost is trusted" meets a reverse proxy
The GPUStack bug is a classic, and that is exactly why it is dangerous.
Before version 2.2.0, GPUStack treated requests that appeared to come from the local machine as the built-in administrator by default. In a very common self-hosted setup, with a reverse proxy on the same host, that trust assumption can extend to remote users. The advisory describes this as an authentication bypass needing no credential.
Admin access to a GPU cluster manager is serious. It gives control over users, models, workers, API keys and configuration, access to stored worker and cloud credentials, and a path to code execution on worker hosts. The advisory says this was exploited in the wild against a v2.1.1 deployment, with attackers running remote shells on the hosts.
Here is the part CTOs should note. The fix, GPUStack 2.2.0, was released on GitHub on 18 June 2026. The project's advisory is dated 31 July. Exploitation still reached CISA's catalogue in November. The lesson is plain: a fix that exists is not a fix that is deployed. GPU cluster managers need the same patch discipline as any other internet-facing system.
The future security risk: attackers follow the plaintext
This is the need of the hour for every team running AI at scale. As more prompts carry contracts, patient notes and source code, the serving layer becomes the most valuable place to sit. Four risks deserve planning now:
Prompt harvesting from inside the trust boundary. A compromised inference engine sees every request in plaintext, including "confidential" ones. Encryption in transit and enclaves do not help once the attacker is the process.
The GPU control plane as a launchpad. Cluster managers can start containers and backends on GPU hosts. An auth bug there is code execution across your fleet, not on one box.
Model files as an exploit delivery channel. Chat templates in GGUF metadata and pickled weights have already been shown to be RCE vectors. Every model you download is input to your serving stack.
Credential sprawl in serving logs. API keys ended up in logs at NEAR AI, and sensitive keys can surface in serving infrastructure. Serving stacks tend to collect secrets quietly.
What this means for Indian and global enterprises
Many organisations, in India and abroad, are moving inference in-house or to regional GPU clouds for cost and data-residency reasons. That is a sound decision. But self-hosting moves the patching burden to you, and open-source serving projects do not always respond to coordinated disclosure, as CERT/CC's notes show. If you buy confidential inference, ask the provider how it secures and patches the engine inside the enclave, not only how it attests the enclave.
Checklist for your AI serving layer this week
List every inference engine (SGLang, vLLM and others), GPU cluster manager and gateway you run, with versions, owners and exposure. Upgrade GPUStack to 2.2.0 or later, recreate the containers, and review users, API keys, workers and inference backends for changes you did not make. If you cannot upgrade GPUStack today, apply the vendor advisory's interim hardening for local-access trust and client-supplied IP headers. Keep inference engine admin and weight-management endpoints off untrusted networks; expose only the inference API through an authenticated gateway. Never treat a source IP, loopback address or forwarding header as an identity. Disable SGLang endpoints you do not use and apply CERT/CC's configuration guidance. Load models only from sources you control, and scan chat templates and weight formats before they reach production. Scrub API keys and tokens from serving logs, and rotate any key that has ever appeared in one. Ask your confidential-inference provider how it hardens, patches and monitors the software inside the enclave, and how fast. Add the serving layer to your incident plan: who rotates credentials, who takes a product offline, and who tells customers.
Sources
https://near.ai/blog/security-incident-report-sglang-vulnerability https://thehackernews.com/2026/11/sglang-rce-exploit-breaches-near-ai.html https://www.kb.cert.org/vuls/id/915947 https://www.kb.cert.org/vuls/id/281278 https://github.com/gpustack/gpustack/security/advisories/GHSA-gj9m-f8ph-pmhr https://github.com/gpustack/gpustack/releases/tag/v2.2.0 https://www.cisa.gov/news-events/alerts/2026/11/09/cisa-adds-four-known-exploited-vulnerabilities-catalog https://nvd.nist.gov/vuln/detail/CVE-2026-102895
Also on our website: https://agtechconsulting.ai/insights/confidential-ai-enclave-sglang-exploit-got-inside.html
Running your own inference stack or buying confidential AI, and want an independent review of the serving layer? Email avinash@agtechconsulting.ai, visit agtechconsulting.ai, or book a 30-minute call: https://calendly.com/avinash-dubey-30-min-catchup/meeting-with-ag-tech
Tagging: NEAR AI SGLang LMSYS GPUStack.ai CISA Software Engineering Institute | Carnegie Mellon University National Institute of Standards and Technology (NIST) NVIDIA Confidential Computing Consortium Hugging Face vLLM OWASP GenAI Security Project Intel AMD Google Cloud Microsoft Azure
Originally published on LinkedIn in the Your CTO Advisor newsletter: read and comment on LinkedIn →
