GPU Side-Channel Exploits: Leaking Private Tokens via LLM KV-Cache Timing and Memory Footprints

GPU Side-Channel Exploits: Token Leakage in Multi-Tenant LLM Clusters

To maximize GPU utilization and reduce operational costs, cloud AI hyperscalers multiplex inference workloads across shared GPU clusters (NVIDIA H100 SXM5, Blackwell B200) using high-throughput serving engines like vLLM, TensorRT-LLM, and TGI. However, shared hardware resources introduce microarchitectural side-channel vulnerabilities that expose private prompt tokens across tenant boundaries.

#1. The PagedAttention KV-Cache Attack Surface

In modern LLM engines, the Key-Value (KV) cache is partitioned into blocks managed like virtual memory pages (PagedAttention). When multiple tenants share the same inference server:

Tenant A (Victim): Prompting LLM with confidential patient record
Tenant B (Attacker): Concurrent queries monitoring GPU L2 cache & HBM3 latency
                         │
                         ▼
Contention in High-Bandwidth Memory (HBM3) access times reveals:
- Sequence length of victim prompt
- Frequency of common prefix tokens (system prompts)
- Memory allocation bursts corresponding to specific output token classes

#2. Reconstructing Tokens via Memory Bandwidth Fingerprinting

Because attention computations scale quadratically with context length, token generation timing exhibits measurable jitter when the victim's request triggers KV-cache swapping between GPU HBM and host system RAM (PCIe bus saturation).

By executing high-frequency probe queries, an attacker creates an acoustic and timing profile capable of reconstructing token sequence boundaries and identifying sensitive keywords (such as medical diagnoses or corporate ticker symbols).

#3. Remediation & Zero-Trust Local-First AI

To protect against hardware side-channels: