Tag
Read 7 articles on Quantization by Ashish Srivastava — practical guides covering implementation patterns, security, and production lessons.
7 posts
Explore the trade-offs between HNSW, IVF, and DiskANN indexes. Learn how recall, latency, and quantization impact vector search performance in production systems.
Part 2 · Retrieval EngineeringLearn how to deploy quantized models in production. This guide covers model artifacts, kernel selection, serving infrastructure, and rollback strategies for ML and platform engineers.
Part 6 · Quantization in PracticePerplexity is a common metric, but it often masks significant quality loss in quantized models. This article explores better ways to measure quantization damage.
Part 5 · Quantization in PracticeLearn how to apply quantization to the KV cache to save memory and speed up inference without significantly degrading model quality.
Part 4 · Quantization in PracticeCompare GPTQ, AWQ, and Bitsandbytes quantization techniques. We examine calibration strategies, speed, and accuracy to help ML engineers choose the right model compression method.
Part 3 · Quantization in PracticeExplore the trade-offs between INT8, FP8, and INT4 quantization formats. Learn which precision level suits your ML workloads and hardware architecture.
Part 2 · Quantization in PracticeExplore the mechanics of quantization, focusing on precision, dynamic range, and outlier management in fp16 and int8 formats for ML engineers.
Part 1 · Quantization in Practice
