Ashim Sharma https://ashimsharma10.github.io/blog Software engineer sharing projects, notes, and guides on ML infrastructure. en-us sharmaashim00@gmail.com (Ashim Sharma) sharmaashim00@gmail.com (Ashim Sharma) Sat, 05 Sep 2026 00:00:00 GMT https://ashimsharma10.github.io/blog/architecture-of-memory-kv-cache The Architecture of Memory: KV Cache Dynamics, Optimization, and the Future of LLM Inference https://ashimsharma10.github.io/blog/architecture-of-memory-kv-cache Modern LLM inference is memory-bound, and the KV cache is the reason. This report walks through why the cache exists, how large it gets, the attention changes that shrink it (GQA, DeepSeek MLA), how engines like vLLM and SGLang manage it, how quantization and eviction compress it, and how disaggregated serving moves it between machines. Sat, 05 Sep 2026 00:00:00 GMT sharmaashim00@gmail.com (Ashim Sharma) kv-cachellminferenceattentiongpu https://ashimsharma10.github.io/blog/cpu-gpu-tpu-hardware-deep-dive CPUs, GPUs, and TPUs https://ashimsharma10.github.io/blog/cpu-gpu-tpu-hardware-deep-dive Why the same matrix multiply runs 100× faster on one chip than another. CPU/GPU/TPU internals, tensor cores, precision, GPU memory and the roofline, interconnects, and when to use each. Visual-first. Mon, 22 Jun 2026 00:00:00 GMT sharmaashim00@gmail.com (Ashim Sharma) gputpuhardwareml-systemscuda https://ashimsharma10.github.io/blog/vllm-how-a-token-gets-served vLLM: How a Token Actually Gets Served https://ashimsharma10.github.io/blog/vllm-how-a-token-gets-served What happens inside a serving engine: why the GPU fills up with something other than the model, what paging the cache buys you, why reading a prompt and writing an answer fight each other, and which knobs settle the fight. Thu, 13 Aug 2026 00:00:00 GMT sharmaashim00@gmail.com (Ashim Sharma) vllmllminferencegpukv-cache