Statistics#
These chapters go from probability to inference. Start with the basics chapter: expectations and variances, the common distributions, the limit theorems, and the bias and variance of estimators. The data science chapter is the practical side: plotting, imbalanced and missing data, feature selection and reproducibility.
Testing and linear models are classical inference: hypothesis tests, confidence intervals, least squares and its regularized versions. Time series drops the iid assumption, and information theory measures uncertainty in bits. Graphical models encode which variables depend on which, and causal inference asks when that kind of structure is enough to read an effect, not just a correlation, from data.
Expectations and variances, the usual distributions, the law of large numbers and CLT, and the bias-variance trade-off.
Sampling, clear plots, smoothing, imbalanced and missing data, feature selection, and tools and principles for reproducible analysis.
Null and alternative hypotheses, p-values, type 1 and 2 errors and power, t- and F-tests, confidence intervals, and ANOVA.
Least squares and its geometry, ridge, lasso and elastic net, GLS and GLMs, and nonparametric extensions like kernel regression and GAMs.
Trend, seasonality and stationarity, AR, MA and ARMA models, autocovariance and Yule-Walker estimation, and similarity measures like DTW.
Entropy, KL divergence and mutual information, their chain rules and inequalities, and minimum description length.
Bayesian networks and undirected models, exact inference by elimination and message passing, sampling and variational inference, and HMMs and Kalman filters.
Potential outcomes and causal graphs, randomized experiments, quasi-experiments like instruments and regression discontinuity, and effect estimation from observational data.