- Published on
Sixteen questions about sparse models, each one the thing you would ask after hearing the last answer: what the router really is, why experts do not learn topics, why sparsity stops paying once you batch, and what it costs to serve one.
Moe
All Write-ups
- llm (10)
- agents (7)
- ai (5)
- guide (5)
- infrastructure (4)
- deep-learning (3)
- gpu (3)
- inference (3)
- loop-engineering (3)
- mlops (3)
- python (3)
- pytorch (3)
- continual-learning (2)
- data-science (2)
- distributed-training (2)
- evals (2)
- guardrails (2)
- kv-cache (2)
- networking (2)
- reinforcement-learning (2)
- anyscale (1)
- architecture (1)
- attention (1)
- azure (1)
- causal-inference (1)
- claude-code (1)
- containers (1)
- context-engineering (1)
- cuda (1)
- data-engineering (1)
- devops (1)
- distributed-computing (1)
- docker (1)
- experimentation (1)
- feature-engineering (1)
- graph-engineering (1)
- grpo (1)
- hardware (1)
- harness-engineering (1)
- hooks (1)
- jepa (1)
- kernel (1)
- linux (1)
- machine-learning (1)
- mathematics (1)
- ml-systems (1)
- model-based-rl (1)
moe (1)
- multi-agent (1)
- number-theory (1)
- numpy (1)
- observability (1)
- pandas (1)
- particle-physics (1)
- physical-ai (1)
- physics (1)
- primes (1)
- quantum-mechanics (1)
- ray (1)
- reasoning (1)
- riemann-hypothesis (1)
- rlvr (1)
- robotics (1)
- self-improving-agents (1)
- skills (1)
- standard-model (1)
- statistics (1)
- subagents (1)
- system-design (1)
- systems (1)
- tpu (1)
- training (1)
- vllm (1)
- world-models (1)