A mechanism-first guide to optimizers: what SGD, momentum, RMSProp, Adam, and AdamW each solve, why AdamW became a strong default for modern deep learning, and when other optimizers still matter.

A practical guide to loss functions: when to use MSE, MAE, Huber, binary cross entropy, cross entropy, KL divergence, hinge loss, contrastive loss, and triplet loss.

A mechanism-first guide to activation functions: why neural networks need nonlinearities, how sigmoid, tanh, ReLU, GELU, and SiLU differ, and why a 400-function survey is best read as a map rather than a menu.