Preface

This book is a personal study log for Mathematics for Machine Learning by Marc Peter Deisenroth, A. Aldo Faisal and Cheng Soon Ong. It is not a textbook and it is not a substitute for one: it is a record of working through the book section by section, with the reasoning, the dead ends and the corrections left visible.

Who these notes are for

The notes are written for a reader with an applied-mathematics and professional background — school and additional mathematics, ACCA, and hands-on implementation of Oracle MRP — who wants the formalism of the book connected to practice rather than left floating. Concretely, that means:

• Formal statements are kept, but always attached to an interpretation. A system $Ax=b$ is described both as a matrix equation and as a resource-consumption ledger.
• Operational framings are compared explicitly. When the book’s model differs from how a real planning or optimisation system behaves, the differences are tabulated rather than glossed over.
• Claims are checked. Numerical claims in a note are backed by a runnable script in scripts/, and mistakes are corrected in place with a dated Corrections and additions section instead of being quietly edited away.

How each note is built

Every note follows the same shape, described in detail in docs/note-format.md:

1. The question — the question that motivated the note, reproduced as asked.
2. Screenshots — the page or pages of the book under discussion, so the note is self-contained.
3. Response — the answer that was given, edited only for formatting.
4. Corrections and additions — later fixes, extensions and clarifications, each dated and each stating what it supersedes.
5. References — a link back to the exact section of the book.

Attribution

The notes, commentary, explanations and any code in this repository are original work. Quoted excerpts from the book are short and are included for study and criticism; the book text, its figures, and the page images reproduced here remain © the authors and their publisher. See References for the full citation and a link to the book, which is freely available from the authors.

How to read these notes

This page is a short orientation. The Preface explains why the book exists; this page explains how to find things.

One note per book section

Each note covers exactly one section of Mathematics for Machine Learning and is named after it, so the table of contents mirrors the book:

Further notes are added as the corresponding sections are read. Coverage at any moment is tracked in docs/state.md in the repository.

Each note is self-contained

A note always contains the question that triggered it, the page images it rests on, the response, and any later corrections. You should be able to read a single note in isolation without the conversation it came from.

Corrections are part of the record

Notes are append-only in spirit: a recorded question or response is never silently rewritten. Later fixes appear in a dated Corrections and additions section that says what it supersedes. The first note has one of these, which is worth reading as an example of the convention — the original response contained an incorrect numerical illustration, and the correction states exactly what was wrong and what the verified result is.

Runnable claims

Where a note makes a numerical claim, a script under scripts/ reproduces it. The scripts require only a Python 3 with numpy. See scripts/README.md.

python3 scripts/explore_example_2_1.py

Building this book locally

The book is built with mdBook and the mdbook-katex preprocessor (which is what renders the mathematics):

make serve     # live preview at http://localhost:3000
make build     # static output in ./book
make check     # build, then confirm ./book stays out of git
make help      # all targets

Build through make (or plain mdbook build / mdbook serve). Nothing is vendored: mdbook-katex injects the <link> for the KaTeX build it renders with, so the page loads that one stylesheet from a CDN and the mathematics always matches the stylesheet.

Part I — Linear Algebra

Notes on Chapter 2, Linear Algebra, of Mathematics for Machine Learning.

Chapter 2 builds the vocabulary that the rest of the book leans on: systems of linear equations, matrices, solving systems, vector spaces, linear independence, bases and rank, linear mappings and affine spaces. It closes with further reading; norms, inner products and orthogonality — the tools of Chapter 3’s analytic geometry — come next.

Notes in this part

The authoritative, machine-readable version of this table is docs/state.md in the repository. Section titles were checked against the book’s own table of contents (2024 free PDF at mml-book.com): Chapter 2 runs 2.1–2.8, and 2.9 is “Further Reading” — there is no affine-mappings section in this chapter. The prose above mentions affine mappings only because they appear later, in the context of affine spaces.

A note on the ordering

Notes follow the book’s own section numbering rather than a reading order chosen here, so a note for Section 2.7 can be added before Section 2.5 without renumbering anything. The first note was prompted by a question about the production-plan model in Example 2.1, which is why Chapter 1 (Introduction and Motivation) has no notes yet.

2.1 Systems of Linear Equations

Note 1. The first note in this book. It records a question asked while reading Section 2.1 of Mathematics for Machine Learning (Deisenroth, Faisal & Ong), the two page screenshots the question is based on, and the response that was given.

The question is reproduced verbatim. The response is the answer as given, lightly edited for Markdown formatting and with LaTeX set as proper math; a correction to one claim in it is appended in Corrections and additions.

The question

These are from Mathematics for Machine Learning. Having studied SPM math and additional math, ACCA and implemented Oracle MRP this production plan doesn’t make sense. Is it listing out each individual resource?

Screenshots

The two pages the question refers to. Both are from Section 2.1, “Systems of Linear Equations”.

Response

Short answer: yes — but only in the accounting sense, and that’s exactly why it doesn’t line up with MRP.

What (2.3) actually says

Read the indices carefully:

• $i=1,\dots ,m$ runs over resources $R_i$ (steel, machine hours, labour, …)
• $j=1,\dots ,n$ runs over products $N_j$
• $a_{ij}$ = units of resource $i$ needed for one unit of product $j$
• $x_j$ = the unknown decision variable: how many units of product $j$ to make
• $b_i$ = units of resource $i$ available

So each row of (2.3) is one resource’s conservation statement:

$$\underset{\text{total consumption of resource }i}{\underbrace{a_{i1}{x}_1+a_{i2}{x}_2+\cdots +a_{in}{x}_n}}=\underset{\text{supply}}{\underbrace{b_i}}$$

Each column, by contrast, is one product’s recipe — its resource-consumption vector $(a_{1j},\dots ,a_{mj})$. So the matrix $A$ is an $m\times n$ matrix: rows = resources, columns = products, and $b$ is the $m$-dimensional resource-availability vector.

Two things worth flagging, because the textbook’s framing is a bit loose:

1. “Optimal” is doing no work in (2.3). That system is only the feasibility constraint set — it says “the plan must consume exactly what’s available, leaving nothing over.” Optimality needs a separate objective (max profit, min cost), which is where linear programming enters. (2.3) alone just says “don’t waste material.”
2. The word “system” is matrix notation for a definition, not a recipe. The rows are the resources, the columns are the products. Reading a column gives you one product’s bill of materials; reading a row gives you one resource’s demand ledger.

A concrete check with $m=2$, $n=3$:

$$\begin{array}{rl}2x_1+1x_2+3x_3& =100\leftarrow \text{resource }{R}_1\text{ fully consumed}\\ 1x_1+2x_2+1x_3& =80\leftarrow \text{resource }{R}_2\text{ fully consumed}\end{array}$$

Why MRP intuition resists it

Your Oracle MRP instinct is right to flag this, but the mismatch is structural, not a math error:

In MRP terms, $a_{ij}$ is roughly your quantity-per-assembly, $b_i$ is roughly your available supply / capacity, and fixing $Ax=b$ is a single-level, infinite-capacity, exact-satisfaction feasibility question. In Oracle MRP you’d actually be solving a recursive netting problem (gross requirement → on-hand → scheduled receipts → net requirement → planned orders), not a static system of equations. That’s why this example feels incomplete — it’s a toy that captures the arithmetic of resource consumption while dropping all the machinery MRP adds.

The gap that usually bites next

Note that in the generic case $m\ne n$: with $m$ resource equations and $n$ product unknowns, you get:

• $n>m$ (more products than binding resources): an underdetermined system — a whole family of exact plans, not one answer. That’s the setup where LP becomes necessary to pick among them.
• $m>n$: overdetermined — likely inconsistent, meaning you cannot consume every resource exactly. That’s a real-world situation MRP handles by over/under-capacity warnings; (2.3) has no way to express it except “no solution.”

For example, with the $2\times 3$ case above, if $v\ne 0$ satisfies $Av=0$ then every $x=x_p+tv$ is again an exact solution, so there are infinitely many plans that consume precisely 100 and 80 units of $R_1$ and $R_2$. No MRP run would ever return a one-parameter family — it returns one plan. That difference is the crux of the intuition. (For this particular $A$, which has full row rank 2, no such $v\ne 0$ exists; see Corrections and additions.)

Corrections and additions

2026-09-29 — the underdetermined example was wrong as originally given

The response above, as first delivered, claimed that for the concrete $2\times 3$ case “every $x=x_p+tv$ (where $Av=0$) is an exact solution: infinitely many plans all consuming precisely 100 and 80 units of $R_1$ and $R_2$,” and listed four such solutions computed from an incorrect null vector. That specific numerical illustration was wrong, and the corrected paragraph above replaces it. Here is the accurate picture, verified in scripts/explore_example_2_1.py:

$$A=\left(\begin{array}{ccc}2& 1& 3\\ 1& 2& 1\end{array}\right),b=\left(\begin{array}{c}100\\ 80\end{array}\right)$$

• $\mathrm{rank}(A)=2$ (singular values $\approx 4.2500$ and $\approx 1.3920$), so the two equations are independent and the null space of $A$ is $\{0\}$: $Av=0$ forces $v=0$.
• Solving $x_p=A^{+}b$ gives the minimum-norm exact solution $x_p\approx (14.2857,25.1429,15.4286)$.
• Because the null space is trivial, $x_p$ is the only exact solution of $Ax=b$, not the base of a one-parameter family.

So the structure of the argument in the response stands — with $m$ equations in $n>m$ unknowns the solution set is generally an affine subspace of dimension $n-\mathrm{rank}(A)$, so whenever $A$ is rank-deficient there are infinitely many exact plans — but this particular matrix is a degenerate illustration of it, since $\mathrm{rank}(A)=m=2$ leaves no freedom. A correct illustration needs dependent resource rows. For instance

$$A^{\prime }=\left(\begin{array}{ccc}2& 1& 3\\ 4& 2& 6\end{array}\right),b^{\prime }=\left(\begin{array}{c}100\\ 200\end{array}\right)$$

has $\mathrm{rank}(A^{\prime })=1$ and hence a null space of dimension $n-1=2$, and the minimum-norm solution is $x_p\approx (14.2857,7.1429,21.4286)$; then $x=x_p+t_1{v}_1+t_2{v}_2$ is exact for every $t_1,t_2$, which is the free-family behaviour the original text was reaching for. A basis of that null space is

$$v_1\approx (0.8452,-0.1690,-0.5071),v_2\approx (0,-0.9487,0.3162).$$

Lesson recorded for future notes: an “infinitely many solutions” claim must be backed by an explicit check that $\mathrm{rank}(A)<n$, and any null vector quoted must be verified with $Av=0$ rather than asserted.

References

• Deisenroth, Faisal & Ong, Mathematics for Machine Learning, Section 2.1 (“Systems of Linear Equations”), Example 2.1 and Equations (2.2)–(2.3). See References for the full citation and link.
• Reproducibility script: scripts/explore_example_2_1.py.

References

The book

This repository contains notes on the following book, which is freely available from the authors:

Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for Machine Learning. Cambridge University Press, 2020. ISBN 978-1-108-47004-9 (hardback), 978-1-108-45514-5 (paperback). DOI: 10.1017/9781108679930. Free PDF: https://mml-book.github.io/

Please cite the book, not these notes, for the mathematics itself.

Sections covered in these notes

Standing conventions

• Page images. Screenshots of book pages live under src/images/ch<NN>/ and are referenced only from the note that needs them. They are included as short excerpts for study; copyright remains with the authors and publisher.
• Numbering. Note filenames use the book’s own section numbers (for example ch02/01-systems-of-linear-equations.md is Section 2.1), so a cross-reference in a note can always be traced back to the book by number alone.
• Reproducibility. Numerical claims are backed by scripts in scripts/, which use only numpy (and optionally scipy).