Tag
Read 2 articles on Inference Speed by Ashish Srivastava — practical guides covering implementation patterns, security, and production lessons.
2 posts
Explore how speculative decoding accelerates LLM inference by using smaller draft models to predict tokens, achieving significant throughput gains without compromising quality.
Explore how MHA, GQA, and MLA architectures balance memory usage and computational efficiency in large language models, with a focus on reducing KV cache overhead.
Part 2 · Transformers Under the Hood
