Deep Learning from Scratch
An intensive, collaborative research project to build a custom Neural Network library. Moving beyond "black box" APIs to understand the mathematics of intelligence.
Course Schedule
Tensors & Matrices
Introduction to high-performance computing. Moving from scalar loops to vectorized operations on GPU hardware.
The Perceptron
Geometric interpretation of linear classifiers. The XOR problem and the limitations of single neurons.
Backpropagation
Deriving the chain rule for computational graphs. Implementing the Multi-Layer Perceptron (MLP).
Tensors, Matrix Math & The GPU
Theme: "Death to the For-Loop"
Learning Objectives
Array Thinking
Shift from scalar variables to Tensor objects.
Broadcasting
Implicit expansion of dimensions `(N,1)` vs `(1,M)`.
Hardware
SIMD architecture and GPU parallelism.
Required Reading & Viewing
3Blue1Brown: Vectors, what even are they?
Visualizing linear algebra as geometric transformations rather than just lists of numbers.
PyTorch 60-Minute Blitz
Official Documentation. Read Sections 1 & 2 only (Tensors & Autograd).
NumPy Visualized
A critical visual guide to understanding Broadcasting and Slicing.
Assignment: The Tensor Gym
Instruction: Create a Google Colab notebook. Solve the following without using `for` loops.
-
1
The RGB Flip
Tensor shape
(3, 256, 256). Swap Red (0) and Blue (2) channels. -
2
The Checkerboard
Generate an 8x8 checkerboard of 0s and 1s using slicing/reshaping.
-
3
The Mask
Replace all values < 0.5 in a random tensor with 0.
Collaborative Task
Phase 1: Setup
Initialize the shared repository environment.
- Create GitHub Account
- Fork
wids_lib - Add
students/bio.txt - Submit Pull Request
The Vector Racer
Classify MNIST digits using Nearest Neighbor.
Constraint: No loops permitted. Must use GPU broadcasting.