A practical guide to loss functions: when to use MSE, MAE, Huber, binary cross entropy, cross entropy, KL divergence, hinge loss, contrastive loss, and triplet loss.

A mechanism-first guide to activation functions: why neural networks need nonlinearities, how sigmoid, tanh, ReLU, GELU, and SiLU differ, and why a 400-function survey is best read as a map rather than a menu.

understand batch gradient descent, stochastic gradient descent, and mini-batch gradient descent.

understand how backward propagation works in gradient descent.