When thousands of requests share the same system prompt, recomputing its KV cache each time is pure waste. Prefix caching stores and reuses those vectors, cutting TTFT by up to 97% in common deployments.

Splitting a long prefill across multiple iterations keeps decode requests from stalling, with no extra FLOPs and negligible IO overhead.

How iteration-level scheduling eliminates GPU idle time, and how prefill and decode rows can share one packed forward pass.

How vLLM borrows the OS paging idea to eliminate KV cache memory fragmentation, pushing GPU utilization from ~30% to ~96%.

why the key-value cache avoids redundant computation in autoregressive decoding, and the memory/compute tradeoffs it introduces.

Use mitmproxy to capture LLM API traffic and Tampermonkey to turn mitmweb's raw JSON into a readable, chat-like viewer.