STORY · VERKTOY_
Inference Engineering mastery school — Philip Kiely and Ali Taha from Basten
The Latent Space episode covers inference engineering as a distinct field, with Basten's leaders discussing optimizations of open models for fast and reliable production deployment. They walk through techniques like quantization, speculative decoding, KV-cache optimization and model parallelization, and show concrete examples such as a GLM-5.2 experiment where more aggressive quantization increased throughput by 20 percent.
WHY IT MATTERS
Inference optimization is becoming increasingly important for making AI models practically usable and cost-effective in production, and represents a distinct technical area across all companies deploying AI.
SOURCES
MACHINE-GENERATED SUMMARY This summary is written by machine from the sources below. We sort and explain — but we are a way into the field, not the final word. Check the source when something matters to you.