KV-Cache Prompt Leakage: Timing Side-Channel Attacks That Steal System Prompts From Shared LLM Caches

KV-cache prompt leakage is SSA descriptive wording for a confidentiality attack against the response cache an LLM serving system runs to make repeated or shared prompt prefixes faster to process. The defining mechanism is timing, not injection: a cache hit returns measurably faster than a cache miss, and an attacker who can send requests to a shared endpoint and measure how long each one takes can use that timing difference alone to infer, token by token, the content of a system prompt or another user's prompt that was never shown to them. Documented research reports recovering per-token cache hits with 99% accuracy and reconstructing another user's prompt with a 99% average success rate against a real serving configuration, using no injected content and no compromised credential.

What it is

The defining object is the timing signal a shared inference-serving cache leaks on every request: whether the prefix just submitted matched something already cached, and, with finer measurement, which specific tokens matched. Two cache layers carry this signal. The Key-Value (KV) cache stores the attention-layer state for a prompt prefix a model has already processed, so that a later request sharing that prefix can skip recomputing it; a semantic cache stores complete ⟨query, response⟩ pairs matched by embedding similarity rather than exact text. Both exist purely as inference-cost and latency optimizations, and neither was designed to carry information to anyone other than the requester whose own prompt produced the cache entry.

The defining cause is that a cache shared across more than one tenant turns "was this prefix already in the cache" into an observable fact about a system prompt or a stranger's request, not just about the attacker's own prompt. An attacker who sends many candidate prefixes and measures response latency for each can distinguish a cache hit (fast) from a miss (slow) without ever seeing the cached content directly, then use that single bit of information, repeated token by token or prefix by prefix, to reconstruct material they were never authorized to read. The defining consequence is disclosure through a side effect of a performance feature, with no authorization check anywhere in the path to fail, because no authorization check was ever consulted in the first place.

What it is not

Semantic cache poisoning, a write-side integrity attack on the same general caching layer, where an attacker plants a crafted ⟨query, response⟩ pair so later users receive a chosen answer. This entry is the read-side counterpart on a lower-level cache: the attacker never writes anything and the cached content is never altered, only inferred from how long it took to serve.
RAG poisoning and other content-injection attacks, which require the attacker to get adversarial material into a corpus or context the model reads. Cache-timing attacks require no content delivery of any kind, only the ability to send timed requests to a shared endpoint.
A general observation that LLM APIs have variable latency. Latency varies for many benign reasons; this working term is specifically about latency differences that are reproducibly data-dependent on cache-hit status and therefore usable to recover prompt content.
A claim that prompt caching is inherently unsafe or should be turned off. Caching is a standard, often default-on, cost and latency optimization across major providers; this entry covers what happens when a cache is shared across tenants without isolation strong enough to make the timing signal useless to an outside observer.
Reproducible timing-measurement code, statistical classifiers tuned to a specific provider, or a step-by-step prompt-reconstruction recipe.

In scope

Timing-based inference of KV-cache hits and misses to recover a confidential system prompt or a prefix shared with another user's request.
Timing-based inference against a semantic response cache to learn whether a target query was recently asked by someone else.
Statistical auditing techniques that detect undisclosed, cross-tenant cache sharing in a production LLM API without access to the provider's internal implementation.
Self-hosted serving stacks (such as vLLM- or SGLang-style deployments) and semantic-cache middleware (such as GPTCache) that share cache state across requests from different callers by default.
Defensive controls for cache isolation boundaries, timing-signal normalization, and detection of probing traffic.

Target state, trust boundary, and persistence

The target state is the content of a system prompt or another tenant's prompt, surfaced indirectly through the serving layer's own cache rather than through the model's generated output. The trust boundary that fails is the assumption that a performance optimization sitting below the model, outside the context window, outside any content filter, carries no information a requester could not otherwise obtain. That assumption holds only if the cache gives every caller statistically indistinguishable timing regardless of hit or miss, or if the cache is not shared across callers who should not be able to observe each other's requests at all.

Persistence runs from session to shared-system. A single timing probe is a single request, but the underlying condition, a cache shared across tenants with a measurable hit/miss timing gap, is a standing property of the deployment. It reproduces for every subsequent probe from any caller, not only the one who first measured it, for as long as the cache remains shared and unisolated.

What the research establishes

Song, Pang, Wang, Wang, Wang, Chen, Song, Jin, Meng, and Hou, "The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems," report discovering, in the authors' words, "a set of new timing side channels in LLM systems, arising from shared caches and GPU memory allocations, which can be exploited to infer both confidential system prompts and those issued by other users." Their attack targets both the KV cache and a semantic cache, using "timing measurements and classification models to detect cache hits" combined with "a token-by-token search algorithm to efficiently recover shared prompt prefixes," which the paper frames as showing "the feasibility of stealing system prompts and those produced by peer users." The authors state their black-box testing was run "on popular online LLM services" and concluded that "such privacy risks are completely realistic, with significant consequences." The work was first submitted to arXiv on 2024-09-30, revised through five versions, and accepted for publication by IEEE Transactions on Information Forensics and Security (arXiv 2409.20002, The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems).

A separate NDSS 2025 paper, "I Know What You Asked: Prompt Leakage via KV-Cache Sharing in Multi-Tenant LLM Serving," introduces PROMPTPEEK, a side-channel attack reconstructing another user's private prompt from KV-cache sharing behavior in serving frameworks such as vLLM and SGLang. The paper reports a 99% average success rate reversing prompt input on a Llama2-13B deployment, using as few as 60 requests when the prompt template was known, and the authors disclosed the finding to SGLang, OpenAI, ByteDance, and Anthropic (NDSS Symposium 2025, I Know What You Asked: Prompt Leakage via KV-Cache Sharing in Multi-Tenant LLM Serving).

Gu, Li, Kuditipudi, Liang, and Hashimoto took a different approach, auditing production APIs from the outside rather than attacking a serving framework directly. Their ICML 2025 paper states that "prompt caching in large language models (LLMs) results in data-dependent timing variations: cached prompts are processed faster than non-cached prompts," introducing "the risk of side-channel timing attacks," and that "if the cache is shared across users, an attacker could identify cached prompts from fast API response times to learn information about other users' prompts." Running statistical audits against real providers, the authors report they "detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts," and separately found that timing variations also leaked model-architecture information, concluding that OpenAI's embedding model is a decoder-only Transformer, "which was previously not publicly known" (arXiv 2502.07776, Auditing Prompt Caching in Language Model APIs, ICML 2025).

Taken together, these three results establish the same underlying pattern from three different angles: a framework-level attack against open-source serving stacks, an attack disclosed directly to major providers, and an external audit finding real production cache sharing independent of any specific attack demonstration. None of them required injecting content, compromising a credential, or bypassing an access-control check; the information traveled entirely through response time.

Detection: observable signs

A target endpoint's response latency clusters into two or more statistically distinct groups correlated with prompt-prefix overlap rather than prompt length or complexity.
A caller issues many structurally similar requests that vary only a short, systematically swept substring, a pattern consistent with a token-by-token or prefix-recovery probe rather than ordinary usage.
Request volume or variety from a single caller is anomalously high relative to any plausible single-prompt task, especially against an endpoint serving a fixed, shared system prompt.
Latency to the same or a very similar prompt differs measurably depending on which other recent callers' requests preceded it, consistent with a cache shared across tenants rather than isolated per caller.
A self-hosted serving deployment has no documented cache-isolation boundary between callers and no monitoring of response-time variance as a signal in its own right.

Defenses, containment, and recovery

The table below is SSA editorial guidance for defenders. It is neither a standard requirement nor a measured effectiveness ranking.

Control	Transition it interrupts	Applicability	Residual risk and verification
Isolate cache state per tenant, organization, or workspace, rather than sharing one cache across all callers of a serving endpoint	A caller's request timing carrying information about a different tenant's cached prompt	Any multi-tenant deployment, whether a managed API or a self-hosted serving stack	Isolation granularity matters: both OpenAI's and Anthropic's own documentation describe isolation at the organization or workspace level, not necessarily finer, so callers sharing one workspace or one API key still share a cache with each other.
Normalize or pad response timing so cache hits and misses are not reliably distinguishable from outside	The specific bit of information, hit versus miss, that every documented attack above depends on	Any serving layer where eliminating timing variance entirely is impractical	Normalization adds latency or complexity precisely where caching was adopted to reduce it, and a sufficiently patient attacker can still extract a weaker signal from residual variance; treat this as raising attack cost, not eliminating it.
Rate-limit and monitor for systematically swept, near-duplicate request patterns against a single endpoint or system prompt	The repeated, incremental probing a token-by-token or prefix-recovery attack requires to succeed	Any endpoint serving a fixed or reused system prompt to many callers	A patient, low-rate attacker can stay under naive rate-limit thresholds; pair with the latency-clustering detection signal above rather than relying on request volume alone.
Treat a system prompt as potentially recoverable by a sufficiently motivated caller, and avoid placing secrets or access-control logic directly in it	A recovered system prompt handing an attacker the deployment's own operating instructions or embedded credentials	Any deployment with a system prompt serving more than one caller population	This does not stop the leakage itself; it limits what the leakage is worth, which is a meaningful mitigation when full cache isolation is not available.
Where self-hosting a serving stack such as vLLM, SGLang, or a semantic-cache layer such as GPTCache, review the project's current documentation for multi-tenant isolation guidance before sharing one deployment across untrusted callers	A cache designed for single-tenant efficiency being exposed to multiple mutually untrusted callers without any isolation decision having been made	Any self-hosted deployment serving more than one trust domain	Guidance and defaults change between releases; re-check after any upgrade rather than assuming a prior review still holds.

Prevention should treat cache sharing as a tenant-isolation decision made deliberately, not a default inherited from whichever serving stack or middleware was fastest to stand up. Detection should watch response-time distributions as a first-class signal, since none of the documented attacks above produce an error, a denied request, or any log entry that looks like an authorization failure. Containment should assume that an anomalous, systematically swept probing pattern against a shared endpoint may already have recovered part of a system prompt, and should rotate any credential or access-control logic that prompt contained rather than only blocking the probing traffic. Recovery should audit whether the deployment's cache-isolation boundary matches its actual tenant boundary, since, per the ICML 2025 audit findings, cross-user cache sharing has been found in production APIs that had not disclosed it.

Boundaries with similar concepts

This differs from semantic cache poisoning in which direction the compromise runs. Semantic cache poisoning is a write: an attacker plants a chosen ⟨query, response⟩ pair so a later caller receives fabricated content. KV-cache prompt leakage is a read: the attacker writes nothing, and the cached content is never altered, only inferred one bit at a time from how fast it was served.

This also differs from RAG poisoning and other context-injection working terms in requiring no content delivery at all. Every injection-based attack on this site needs the attacker to get some content, a document, a tool result, a configuration file, in front of the model. A cache-timing attack needs only the ability to send timed requests to a shared endpoint; the model may never process a single byte the attacker supplied.

Mapping to standards and research

Source	Source-owned term or item	Relationship to this article
arXiv 2409.20002 (Song, Pang, Wang, Wang, Wang, Chen, Song, Jin, Meng, Hou)	timing side channels in LLM serving systems	Primary-source attack against KV-cache and semantic-cache timing, demonstrated in black-box tests against production online LLM services.
NDSS Symposium 2025	PROMPTPEEK	Primary-source attack reconstructing another user's prompt via KV-cache sharing in multi-tenant serving, disclosed to affected vendors.
arXiv 2502.07776 (Gu, Li, Kuditipudi, Liang, Hashimoto), ICML 2025	prompt-caching audit methodology	Independent, outside-the-box statistical audit finding undisclosed cross-user cache sharing in seven production API providers.
OWASP GenAI Security Project	Top 10 for LLM Applications 2025, `LLM07:2025 System Prompt Leakage`	Standard risk category covering disclosure of a system prompt's content, the asset this attack's prompt-stealing variant targets directly.
OWASP GenAI Security Project	Top 10 for LLM Applications 2025, `LLM02:2025 Sensitive Information Disclosure`	Standard risk category covering the cross-tenant prompt-disclosure variant of this attack.
Agent State Attack	KV-cache prompt leakage	SSA descriptive framing bundling the research above under one Context target-state entry.

Related terms

Agent state attack: the umbrella definition, including Context among the listed target states.
Semantic cache poisoning: the adjacent write-side entry on the same general caching layer, contrasted above by direction of compromise, a plant there, an inference here.
RAG poisoning: the adjacent context-surface entry requiring content injection, contrasted above with an attack that requires none.

FAQ

Does this require the attacker to have any special access to the LLM provider's infrastructure?

No. All three documented results above were produced, or were independently auditable, from the outside: by sending ordinary API requests to a public endpoint and measuring how long each one took to respond. No internal access, compromised credential, or injected content is required.

Does using a well-known managed API provider instead of self-hosting make this moot?

It reduces but does not eliminate the risk. Both OpenAI's and Anthropic's own prompt-caching documentation describe cache isolation at the organization or workspace level, which stops cache sharing between unrelated customers, but the ICML 2025 audit found undisclosed cross-user cache sharing in seven providers including OpenAI, and callers who share one workspace, one API key, or one self-hosted deployment still share a cache with each other.

Is turning off prompt caching the simple fix?

It removes the specific timing signal these attacks depend on, at the cost of the latency and compute savings caching exists to provide. For most deployments, isolating the cache to match the actual tenant boundary, rather than disabling caching outright, addresses the attack without giving up the optimization; disabling caching is a defensible choice only where isolation cannot be verified and the system prompt's confidentiality is a hard requirement.

Can this attack recover a user's prompt even if the attacker never sees that user's request?

Yes, that is the specific scenario PROMPTPEEK and the Early Bird research both demonstrate: an attacker who shares a cache with a victim, but never observes the victim's actual request or response, can still reconstruct meaningful content of that request by timing their own, separate probe requests against the shared cache.

Sources

Verification and change history

Standards and versions referenced: OWASP Top 10 for LLM Applications, 2025 edition.

Last verified: 2026-10-23. The Early Bird quotations and version history were read from the arXiv abstract page. The PROMPTPEEK success rate, request count, and disclosure recipients were read from the same citation this site's semantic cache poisoning entry previously verified against the NDSS 2025 paper. The ICML 2025 audit quotations, provider count, and the OpenAI embedding-model finding were read from the arXiv abstract page. The OpenAI and Anthropic cache-isolation and retention statements were read from each provider's own current documentation page. The OWASP identifiers were read from the official project pages.

Change history: 2026-10-23, first version.

Working-term label: Mixed provenance. "PROMPTPEEK" is a Research Term owned by its NDSS 2025 authors. LLM07:2025 System Prompt Leakage and LLM02:2025 Sensitive Information Disclosure are OWASP Standard Terms. "KV-cache prompt leakage" as the umbrella title for this entry is SSA descriptive wording bundling the sources above, not a term attributed to any single one of them.