Mathematics#
The math the rest of the book uses. Math basics, calculus and linear algebra are reference chapters: identities, derivative tables, matrix facts, norms and decompositions. Optimization builds on them, from convex sets and duality to gradient descent and the optimizers used for neural networks.
The other chapters go deeper into single topics: real analysis proves the calculus facts from the axioms of the real numbers, differential equations and nonlinear dynamics cover how systems evolve in time, and signal processing covers the Fourier and wavelet transforms.
Handy bounds, common functions, sets and logic, proof techniques, log and trig identities, and complex numbers in polar form.
Taylor expansions, tables of derivatives, integrals and series, and Lagrange multipliers for constrained minimization.
Matrix properties, matrix calculus, vector and matrix norms, and the eigenvalue and singular value decompositions.
Recipes for ordinary differential equations, from separable and first-order linear equations to constant coefficients, variation of parameters and Laplace transforms.
Completeness of the reals, sequences and series, open and compact sets, continuity, derivatives, uniform convergence and the Riemann integral.
Convex sets and functions, duality and the KKT conditions, descent methods, the optimizers used for neural nets, EM and branch-and-bound.
Sampling, filters, the Fourier transform and DFT, and wavelets, which are localized in both time and frequency.
Bifurcations, stability of fixed points, maps that period-double into chaos, conservative systems and the Lorenz equations.