Tag

Optimizer State

Read 1 article on Optimizer State by Ashish Srivastava — practical guides covering implementation patterns, security, and production lessons.

1 post · thin content (not indexed)

Oct 13, 20266 min readDevelopmentFeatured

Full Fine-Tuning vs PEFT: The Memory Math

Compare the memory requirements of full fine-tuning and PEFT. We break down the math on optimizer states, activation checkpointing, and GPU utilization.

Part 2 · Fine-Tuning LLMs Hands-On

#gpu-optimization2 #llm-training6 #peft1 #full-fine-tuning1 #memory-requirements1 #activation-checkpointing1

Optimizer State

Full Fine-Tuning vs PEFT: The Memory Math

Related tags