https://ashimsharma10.github.io/blog Software engineer sharing projects, notes, and guides on ML infrastructure. en-us sharmaashim00@gmail.com (Ashim Sharma) sharmaashim00@gmail.com (Ashim Sharma) Sat, 05 Sep 2026 00:00:00 GMT https://ashimsharma10.github.io/blog/architecture-of-memory-kv-cache https://ashimsharma10.github.io/blog/architecture-of-memory-kv-cache Modern LLM inference is memory-bound, and the KV cache is the reason. This report walks through why the cache exists, how large it gets, the attention changes that shrink it (GQA, DeepSeek MLA), how engines like vLLM and SGLang manage it, how quantization and eviction compress it, and how disaggregated serving moves it between machines. Sat, 05 Sep 2026 00:00:00 GMT sharmaashim00@gmail.com (Ashim Sharma) kv-cachellminferenceattentiongpu