EP 42 | CS224N: Backpropagation and Neural Networks

We are looking under the hood of deep learning to understand the mathematical engine driving modern artificial intelligence: Backpropagation. In this episode, we break down how neural networks transition away from rigid linear boundaries to build complex, non-linear understandings of language.Key Topics:Evaluating Word Vectors: The core trade-offs between Intrinsic subtask testing (like word analogies) and Extrinsic downstream evaluation in real-world applications.Named Entity Recognition (NER): How window classification allows networks to train word vectors and model weights simultaneously to classify entities in context.The Magic of Non-Linearities: Why activation functions (from classic ReLU to modern LLM standards like GELU and SwiGLU) are mathematically necessary to keep deep layers from collapsing into a single flat function.Gradients, Jacobians, and Graphs: A walk through matrix calculus, the practical engineering reality of the "Shape Convention," and how computation graphs use simple rules (Addition distributes, Max routes, Multiplication switches) to pass error signals flawlessly.Note: This is an AI-generated discussion created using Google's NotebookLM, based on publicly available Stanford University course material (specifically CS224N) and personal study notes from my learning journey.