Math for AI

Vectors, dot products, matrices, derivatives, gradient descent, probability, and a tiny neural network that learns — the intuition behind embeddings and training, learned by building a small toolkit instead of reading proofs.

ជំពូកនេះមិនទាន់ត្រូវបានបកប្រែជាភាសាខ្មែរនៅឡើយទេ — សូមអានជាភាសាអង់គ្លេសខាងក្រោម។

Every AI explainer eventually says something like "the model turns words into vectors, multiplies them by matrices, and nudges the weights along the gradient of the loss." Most people nod and move on, because the words sound like a wall of university math. They are not. Each one is a small idea you can run in twenty lines of JavaScript, and once you have seen it run, the sentence stops being a wall.

This series teaches exactly the math that sentence uses, and nothing more. A vector is a list of numbers, and with it you can compare a student, a color, or a word. The dot product measures how much two vectors point the same way, which is the heart of search by meaning. A matrix is a table of numbers that transforms a vector, which is what one layer of a neural network does. A derivative is a slope, a gradient is a slope in many directions at once, and gradient descent is walking downhill until the error is small. Probability, logarithms, and cross-entropy explain why a language model's training loss looks the way it does. Finally you put all of it together in a small neural network that learns a problem a straight line cannot.

You build everything yourself. Across the series you grow mathai, a zero-dependency Node.js toolkit with a vector module, a matrix module, a numeric-derivative module, a gradient descent runner, a statistics module, and a tiny neural network with hand-written backpropagation. Every program prints real numbers you can check by hand, and nothing needs an API key, a GPU, or a paid account.

Two honest notes. First, this is intuition with working code, not a math course: there are almost no proofs, and where something is simplified, the chapter says so. Real systems use libraries that run the same operations thousands of times faster on specialized hardware. The ideas are identical; the speed is not. Second, you do not need to memorize any of it. The goal is that when someone says "embedding", "loss", or "gradient", you know what is being computed, and you can say it to a class of students without hiding behind jargon.

You need basic JavaScript from JavaScript & TypeScript Fundamentals and the terminal from Command Line Basics. School algebra is enough: you should be comfortable with y = 2x + 1. Math for Programmers is a helpful warm-up, especially its chapters on functions and logarithms and probability, and How LLMs Work shows where each idea ends up inside a language model. You need Node.js (the LTS version) and a code editor such as Visual Studio Code. Everything is free.

  1. 01Vectors: Lists of NumbersLearn that a vector is just a list of numbers, then add, subtract, scale, measure and average vectors in JavaScript, and set up the zero-dependency mathai toolkit you will grow through the whole series.
  2. 02Dot Product and SimilarityLearn the dot product as a score for how much two vectors agree, turn it into cosine similarity, see why cosine beats distance for meaning, and build nearest-neighbour search and a k-NN classifier, the core of embeddings and RAG search.
  3. 03Matrices and LayersA matrix is a table of numbers and also a machine that turns one vector into another: learn matrix times vector, shapes and the shape-mismatch error, transpose, matrix times matrix, and build a linear layer y = Wx + b, the step a neural network repeats thousands of times.
  4. 04Slopes and DerivativesA derivative is just a slope, and a gradient is a slope in many directions at once: measure both by nudging a number and watching what happens, check the basic rules numerically, see the chain rule in plain words, and learn why this tells a model which way to change its weights.
  5. 05Gradient Descent and LossMeasure how wrong a guess is with a loss, then walk downhill by repeatedly nudging the numbers against the gradient, fit a straight line to real data, watch what a bad learning rate does, and see why this same loop trains models with billions of parameters.
  6. 06Probability, Logs and SoftmaxTreat a prediction as a distribution that sums to 1, see why logarithms turn products into sums and save tiny numbers from vanishing, measure surprise with entropy, turn raw scores into probabilities with a stable softmax, and understand cross-entropy loss and perplexity, the numbers behind language model training.
  7. 07Build a Neural NetworkBuild one neuron, see why a stack of linear layers is still just one line, add an activation, train a tiny network on XOR with hand-written backpropagation, and count its parameters.
  8. 08Statistics and the Big PictureLearn mean, variance, and standardizing, split data to catch overfitting, understand seeds and loss curves, see where each piece of math appears in an LLM, and get ready for the questions an audience will ask.