Reference: Optimizers
A reference covering neural network optimizers from GD to AdamW, with learning-rate schedules, decision trees, and practical guidance for each architecture.
Deep Learning
Reference: Loss Functions (the Training Objective)
A reference on training loss functions—MSE, cross-entropy, Huber, hinge, contrastive, and triplet—with formulas, gradient shapes, and selection guidance.
Reference: Activation Functions
A consolidated reference of activation functions—sigmoid, tanh, ReLU, GELU, softmax—with formulas, output ranges, gradient properties, and when to use each one.
Why Isn't My Model Learning? A Friendly Guide to Diagnosing Deep Learning Dead-Ends
Learn a practical 4-step checklist to diagnose stalled deep learning models: overfit one batch, check gradients, scale data, and sweep learning rates.
Batch Normalization and Dropout: The Regularization Tricks That Make Deep Learning Actually Work
Learn how Batch Normalization and Dropout fix overfitting and training instability in deep networks, with practical PyTorch code and layer-ordering guidance.
Borrowing Brains: A Beginner's Guide to Transfer Learning and Fine-Tuning
Learn how transfer learning lets you borrow pretrained models like ResNet and BERT, swap their heads, freeze the body, and fine-tune for custom tasks.
Building a Miniature Transformer From Scratch: The 'Lego' Approach to Deep Learning
Build a miniature Transformer from scratch in PyTorch by snapping together embeddings, multi-head attention, and positional encoding like Lego bricks.
Multi-Head Attention and Positional Encoding: Giving the Transformer a Sense of Direction
Discover how positional encoding gives Transformers word-order awareness and multi-head attention tracks multiple patterns at once in this hands-on guide.
The Attention Mechanism, Finally Explained (Without the Matrix Algebra)
Understand the attention mechanism powering every modern language model—queries, keys, values, and softmax explained through analogies and Python code.
LSTMs and GRUs: Giving Networks a Memory
Learn how LSTMs and GRUs use gated memory to beat the vanishing gradient, retaining early signals across long sequences for better sequence predictions.
Why RNNs Forget: The Intuition Behind the Vanishing Gradient Problem
See why RNNs forget early sequence signals: backpropagation multiplies small weights across steps, so the gradient vanishes before reaching the first input.
Why Computers See Better with CNNs: An Intuitive Guide to Image Recognition
Learn why CNNs outperform dense networks for image recognition by using sliding filters, shared weights, and pooling to detect spatial patterns efficiently.
Why Deep Networks Die: Solving the Vanishing Gradient Problem with ReLU, ResNets, and BatchNorm
Learn why deep neural networks stop learning as they grow deeper, and discover how ReLU, ResNets, and BatchNorm solved the vanishing gradient problem.
Backpropagation Intuitively: How Networks Learn From Their Mistakes
Learn how backpropagation works intuitively—no calculus needed. See how neural networks assign blame to weights via the chain rule and learn from mistakes.
Neural Networks, Without the Calculus: What's Actually Happening Inside a Single Neuron
Learn how a single artificial neuron works without calculus—weights, bias, and sigmoid activation combine evidence into calibrated probabilities.