https://ashimsharma10.github.io/blog Software engineer sharing projects, notes, and guides on ML infrastructure. en-us sharmaashim00@gmail.com (Ashim Sharma) sharmaashim00@gmail.com (Ashim Sharma) Mon, 28 Sep 2026 00:00:00 GMT https://ashimsharma10.github.io/blog/kimi-k3-architecture-teardown https://ashimsharma10.github.io/blog/kimi-k3-architecture-teardown What is new in Moonshot AI's Kimi K3, part by part, and how each part differs from DeepSeek-V3, Qwen3, Llama 4 and gpt-oss: hybrid KDA and MLA attention for a million-token context, a latent expert layer, quantile load balancing, attention across depth, a soft-capped activation, a per-head optimizer, and training in the serving number format. Mon, 28 Sep 2026 00:00:00 GMT sharmaashim00@gmail.com (Ashim Sharma) kimi-k3moeattentionarchitecturellmtraininginference