Estimating Compute and Memory Requirements for LLM Training and Inference
· â 14 min read · âī¸ k4i
A back-of-the-envelope framework for estimating LLM training FLOPs, inference FLOPs, weight memory, KV cache, and training memory.
Transformer
Why KV Cache Works in LLM Inference
· â 8 min read · âī¸ k4i
why the key-value cache avoids redundant computation in autoregressive decoding, and the memory/compute tradeoffs it introduces.
