Research overviews#
These chapters are reading lists rather than lecture notes. Each collects papers on one research topic, grouped by approach, usually with a short note on what each paper does and a link to it. Recent entries sit next to the classic papers they build on.
The first group is about modern models: LLMs (prompting, architectures, mechanistic interpretability, automated science), embeddings and retrieval, transfer learning and adversarial robustness, and uncertainty. Interpretability, the longest list after LLMs, covers transparent models, post-hoc explanations and how to evaluate them. The rest cover the statistics and theory side (generalization, complexity, deep learning theory, causal inference, disentanglement, the scattering transform) and applications in medicine and in protein and RNA structure, plus a short list of science results that are simply interesting.
A grab bag of surprising results from neuroscience, biology, statistics and machine learning.
Clinical datasets, bias in clinical decision instruments, and how doctors reason about and use models.
Training embedding models, contrastive losses, approximate nearest-neighbor search, RAG and interpretable embeddings.
Prompting, architectures and reasoning, mechanistic interpretability, and LLMs that explain data and do science.
Training models that hold up when the test data shift, from domain adaptation and test-time training to adversarial attacks.
Calibration, outlier detection, ensembles, Bayesian neural nets and conformal prediction.
Evaluating interpretations, transparent models like rules, trees and GAMs, and post-hoc explanations of fitted models.
Generalization error as risk minus empirical risk, bounds from uniform stability, and a lower bound for trees on additive data.
Disentangled VAE losses and their variants, GAN latent spaces edited during or after training, weak supervision, and how disentanglement is measured.
Classic causal studies, mediation, heterogeneous effects, causal discovery, invariant prediction, and neural nets for treatment effects.
Predicting RNA and protein structure from sequence, from dynamic programming to AlphaFold, plus engineered viruses as treatment.
Fixed wavelet filters in place of learned ones, from Mallat’s scattering transform to hybrid networks and adaptive wavelets.
What complexity means for a model, from philosophy and Kolmogorov complexity to VC dimension, double descent and minimum description length.
Why overparameterized nets generalize, seen through implicit regularization, kernels, expressiveness, statistical physics and empirical studies.