Juan José Barrientos Salazar

LLMs in Depth: How an LLM Works Mathematically (and Its Implementation with PyTorch)

Imagine a 30-minute space where mathematics, code, thought experiments, and one of the most attractive topics of the present converge during powerful minutes — that is what this presentation seeks. The goal of this talk is to review each of the components of a Large Language Model (LLM), from the embedding system and the BPE algorithm to the attention mechanism that is the core of modern AI, passing through normalization and the small "tricks" used in both training and inference to improve results and make LLMs more optimal. Each of these components will be addressed from three perspectives: 1) the pure mathematics that composes the solution, 2) the interpretation of this mathematics (why it is useful and how we can visualize it), and 3) the implementation in code, where small code snippets will show how these systems are implemented in Python. At the end, an open-source code repository will be provided with the full implementation and training pipeline for a "playground model" implementing a GPT-style model. The intention with this talk is not only to shed light on one of the most interesting and complex topics in the modern world, but also to provide tools to question how these systems work and promote research in this field.