Glossary

Every term the notes define with a term - definition line (1157 terms). Each entry links to the place where it is defined; filter to find one quickly.

#

50-percent rule 1/3 of mem is unusable Operating systems

A

abduction adjust noise variables to be consistent with \(x\) Causal inference

abductive reasoning deriving a hypothesis to explain observed phenomenon Psychology

Abel's thm if a power series converges at the point x = R > 0, the the series converges uniformly on the interval [0,R]. A similar result holds if the series converges at x = -R Real analysis

accounting keeping stats on users / processes Operating systems

Acetylcholine excites muscle cells Brain basics

action perform do-intervention, setting \(T=t\) Causal inference

action potential cause by the other 2 Brain basics

active explore states to find utilities and exploit to get highest reward RL and decisions

active stereopsis one projector and one camera vs passive (ex. eyes) Computer vision

Actor-critic estimate value function or Q-function of the current policy (critic), use it to improve policy (actor) RL and decisions

adaboost weight models based on their performance Classification

adaptive dynamic programming 2 steps RL and decisions

adaptive learning too much dataset bias - imagenet things only work on imagenet Cognitive science

addition A + B = (0, 0, 2, 0, 2,-2, 0, ....) Computational neuroscience

adiabatic optimization like quantum simulated annealing, maybe can solve NP-complete problems Quantum computing

adjoint compute with mini-dets Linear algebra

adversarial sample from G, but tricks D to predicting 1 Unsupervised

affine hull aff C = {\(\sum \theta_i x_i \vert x_i \in C, \sum \theta_i =1\)} Optimization

affine set \(x_1, x_2 \in C, \theta \in \mathbb{R} \implies \theta x_1 + (1 - \theta) x_2 \in C\) Optimization

ALE-plots take points conditioned on value of interest, then look at differences in predictions around a window Interpretability

aleatoric uncertainty inherent and irreducible data noise (e.g. features contradict each other) Uncertainty

algebraic limit thm let lim \(a_n = a\) and lim \(b_n\) = b. Then Real analysis · Real analysis

aliasing signals traveling in disguise as other frequencies Computer vision

allodynia when mechanosensation hurts - not understood Sensory input

alpha-level subset \(C_\alpha = \{x\vert f(x) \leq \alpha\}\) Optimization

ALS Lou Gehrig's disease Motor system

amdahl's law \(speedup \leq \frac{1}{S+(1-S)/N_{cores}}\) Operating systems

amplitudes used to calculate probabilites, but can be negative / complex Quantum computing

amygdala contains emotional memories Motor system

analog signal continuous in time Signal processing

anchoring effect buy middle-tier wine because expensive is there RL and decisions

anchors opposite of counterfactuals, once we have these other things won't change the prediction Interpretability

AND-mask minimize the err only in directions where the sign of the gradient of the loss is the same for most environments Causal inference

anencephaly neural tube fails to close higher up Development

anonymous memory pages not associated with a file Operating systems

ANOVA decomposition decompose model into conditional expectations \(Y = \cgreen{g_0} + \cblue{\sum_i g_i (X_i)} + \corange{\sum_i \sum_{j > i} g_{ij} (X_i, X_j) + \dots + g_{1,2,..., p}}\) Interpretability

antecedent body - consists of conditions = binary features e.g. \(X_1 > 0\), \(X_2=0\) Search and logic

anterograde source to termination Brain basics

aperture problem looking through certain hole can change perception (ex. see movement in wrong directions) Computer vision

AR model \(X_t = \sum_{i=1}^p \cblue{\phi_i} X_{t-i}+ \corange{\varepsilon_t}\) Time series

arbitrary coherence market prices themselves that influence consumers' willingness to pay. What this means is that demand is not, in fact, a completely separate force from supply. Psychology

arc consistency satisfy binary constraints (every node is made arc-consistent with all other nodes) Search and logic

ARIMA model \(ARIMA(p, d, q)\): - generalizes ARMA model to non-stationarity (using differencing) Time series

arity fixes number of args Search and logic

ARMA model \(ARMA(p, q)\): \(X_t = \sum_{i=1}^p \cblue{\phi_i} X_{t-i} + \sum_{i=1}^q \cgreen{\theta_i} \corange{\varepsilon_{t-i}} + \corange{\varepsilon_t}\) Time series

assumption no crossings - given a subsequence \([i,j]\), there is either no edge connecting to the ith base (meaning it is unpaired) or there is some edge connecting the ith base to the kth base where ... Omics

astrocytes in CNS, maintain chemical environment, retain stem cell properties Brain basics

asymmetric clustering one machine runs while other monitors it (hot-standby mode) Operating systems

asymmetric multiprocessing boss processor controls the system Operating systems

asynchronous cancellation one thread immediately terminates target thread Operating systems

asynchronous policy iteration don't have to update all states at once RL and decisions

atomic sentence formed from a predicate symbol optionally followed by a parenthesized list of terms Search and logic

atp sensitivity to pain Brain basics

attention vector of importance weights LLMs · Psychology

attentive vision serial search with small focal attention in 50 ms steps Computer vision

autoassociative can return stored X based on a noisy version of X (since it is a point attractor), maybe with some iteration Computational neuroscience

automatic knowledge-acquisition set of techniques for gaining new knowledge Cognitive science

autonomic ganglia peripheral motor neurons that take inputs from brainstem / spinal cord Brain basics

autonomy individual rights Psychology

availability heuristic judging the frequency or probability of an event by the ease with which examples of the event come to mind Cognitive science

axiom of completeness every nonempty set \(A \subseteq \mathbb{R}\) that is bounded above has a least upper bound Real analysis

B

back-door criterion establishes if variables \(T\) and \(Y\) are confounded in a graph given set \(S\) Causal inference

backfitting traditional way to fit - each \(f_i\) is fitted sequentially to the residuals of the previously fitted \(f_0,...,f_{i-1}\) (hastie & tibshirani, 1989) Interpretability

backjumping backtracks to most recent assignment in conflict set Search and logic

backpropagation application of reverse mode automatic differentiation to neural networks's loss Deep learning

backtracking search dfs but only one successor is generated at a time; each partially expanded node remembers which succesor to generate next Search and logic · Search and logic

backward chaining check if something is true Cognitive science

bagging bootstrap aggregation - an ensemble method Classification

balanced tree no 2 nodes differ in height by more than 1 Data structures

bandwidth total number of bytes transferred, divided by time Operating systems

basal forebrain spread synapses all over cortex Disease

base rate neglect grounding one’s judgments in how well sets of characteristics feel like they fit together, and neglecting how common each characteristic is in the population at large Cognitive science

base register holds smallest usable address Operating systems

basis of adapted waveform best basis function for a given signal representation Signal processing

Bayes ball algorithm is \(( X_A \perp X_B )\vert X_C\)? Graphical models

bayesian networks directed Graphical models

Bayesian reinforcement learning assume a prior \(P(h)\) on transition model h RL and decisions

beam search consider only n best moves RL and decisions

Belady's anomaly for some algorithms, page-fault rate may increase as number of allocate frames increases Operating systems

bell's palsy loss of facial nerve - recovers with miswiring (salivary / tear) Development

Bellman curse of dimensionality Data science

bellman eqn \(U(s) = \cblue{R(s)} + \cgreen{\gamma \: \underset{a}{\max} \sum_{s'} P(s' \vert s, a) U(s')}\) RL and decisions

Benjamini–Hochberg procedure controls for false discovery rate Testing

beowulf cluster use ordinary PCs to make cluster Operating systems

best-first choose nodes with best f Search and logic

best-fit allocate smallest hole that is big enough Operating systems

Betz cells if they fire, you will do something Motor system

bias \(E(\hat{\theta}) - \theta\) Stat basics

biased proxy do proxy variables (e.g. weight/height) properly capture the legitimate target variable? Fairness and STS

Bidirectional / unmasked self-attention primary/context vectors are the same LLMs

bidirectional search search from start and goal and see if frontiers intersect Search and logic

binary cross entropy loss corresopnds to bernoulli distr., most common - doesn't penalize (0.1, 0.2) and (0.4, 0.5) the same way, which might be problematic Disentanglement

binary semaphore 0 and 1 Operating systems

bitmap string of n binary digits Operating systems

blade server multiple independence multiprocessor systems in same chassis Operating systems

BMP roofplate Development

BOLD blood oxygenation level-dependent signal Vision

bolzano-weierstrass thm every bounded sequence contains a convergent subsequence Real analysis

bonferroni procedure we are doing 3 tests with 5% confidence, so we actually do 5/3% for each test in order to restrict everything to 5% total Testing

bootstrap resampling with replacement Classification · Evaluation

bounded buffer consumer waits if empty, producer waits if full Operating systems

bounded waiting every process should eventually get to critical section Operating systems

bounds propagation make sure variables can be allotted to solve resource constraint Search and logic

bouton end of presynaptic dendrite Brain basics

bracket entropy function is both upper and lower bounded by bounding functions, which are within epsilon Complexity

bradykinesia slowness of movement Disease

bus shared set of wires Operating systems

C

CACE LATE (local average treatement effect) Causal inference

cache coherency make sure cache coherency is properly updated with parallel processes Operating systems

caching save frequent items on faster things Operating systems

calibration predicted probabilities should match real probabilities Uncertainty

camera calibration determine camera parameters from known 3d points Computer vision

candidate-elimination represent most general (G) and specific (S) members of version space Learning theory

Candide short story, denouncing optimism for pragmatism Philosophy

canonical parameterization \(p(x\vert\mu, \Sigma) = \frac{1}{(2\pi )^{n/2} \vert\Sigma\vert^{1/2}} \exp\left[ -\frac{1}{2} (x-\mu)^T \Sigma^{-1} (x-\mu) \right]\) Stat basics

canons repeat w/ subtle changes (e.g. pitch shift) Cognitive science

cantor's thm Given any set A, there does not exist a function f:\(A \to P(A)\) that is onto Real analysis

capsule vector specifying the features of an object (e.g. position, size, orientation, hue texture) and its likelihood Computational neuroscience

cascading termination if parent dies, its children die Operating systems

cathexis the concentration of mental energy on onen particular person, idea, or object (esp. to an unhealthy degree) Psychology

cauchy condensation test suppose \(a_n\) is decreasing and satisfies \(a_n \geq 0\) for all \(n \in \mathbb{N}\). Then, the series \(\sum_{n=1}^\infty a_n\) converges iff the series \(\sum_{n=1}^\infty 2^na_{2^n}\) converges Real analysis

cauchy criterion a sequence converges \(\iff\) it is a cauchy sequence Real analysis

cauchy criterion for series series converges \(\iff\) \((s_m)\) is a cauchy sequence Real analysis

cauchy-schwartz inequality \(\vert x^T y\vert \leq \vert \vert x\vert \vert _2 \vert \vert y\vert \vert \vert _2\) Linear algebra

cauchy-schwarz \(\vert x \cdot y\vert \leq \vert \vert x\vert \vert \:\vert \vert y\vert \vert\) Stat basics

caudal pontine feeds into spinal cord Motor system

caudate output to globus pallidus and substantia nigra (pars reticulata) Motor system

causal can predict given only past values (for Gaussian processes no difference) Time series

Causal Coarsening Theorem causal partition is coarser version of the observational partition Causal inference

causal connection process people use to identify the causes of events Psychology

causal interaction interaction of such processes, e.g. a car crash Philosophy

causal model causes \(\to\) symptoms Graphical models

causal process leaves marks on the world which persist spatiotemporally (these marks hint at counterfactuals) Philosophy

causal responsibility to which cause(s) do we attribute a given effect? Psychology

causal transportability seeks to identify conditions under which causal knowledge learned from experiments can be reused in different domains with observational data only Causal inference

cell-search sequential Causal inference

Cerebrum The cerebrum is the largest portion of the brain, and contains tools which are responsible for most of the brain's function. It is divided into four sections: Brain basics

certainty effect people are drawn to things that are certain RL and decisions

chain rule let f:A-> R and g:B->R satisfy f(A)\(\subseteq\) B so that the composition g \(\circ\) f is defined. If f is differentiable at c in A and g differentiable at f( c) in B, then g \(\circ\) f is ... Real analysis

chance nodes represent RVs (like BN) RL and decisions

change blindness blind to things you're not attending to Vision

change factual associations modify feedforward weights to update specific factual associations using Rank-One Model Editing (ROME) LLMs

change the inputs how can we make it more natural for humans to specify what they want? Interpretability

change the loop how can humans direct where / when machine aid is helpful? Interpretability

change the outputs how can we help humans understand and solve their own problems? Interpretability

chebyshev's \(P(\vert X-\mu\vert \geq a) \leq \frac{Var[X]}{a^2}\) Stat basics

chi-squared \(\chi_d^2\) ~ \(\sum_i^d U_i^2\) w/ d degrees of freedom Testing

china brain what if different people hit buttons to fire individual neurons Cognitive science

chinese room argument rebuts turing test { cite searle1980minds } Cognitive science

church-turing thesis a function on the natural numbers can be calculated by an effective method if and only if it is computable by a Turing machine Cognitive science

class method static Languages and parallel computing

class selectivity index derived from classes of images that make NF Interpretability

climbing fiber comes from inferior olive and goes back to purkinje cell (this is the efferent copy) = training signal Motor system

clinical staging based on non-tissue things Disease

CLIPS expert system shell Cognitive science

closure set with its limit points Real analysis

cloud-computing computing, storage, and applications as a service accross a network Operating systems

cloze questions same as masked LMs: task is to replace some missing words LLMs

clustered page tables each entry stores everal pages, can be faster Operating systems

clustered system multiple loosely coupled cpus Operating systems

clustering use mixture models Stat basics

clustering algorithms join tree algorithms (see propagation factor graphs) Graphical models

CNF conjunctive normal form - conjunction (and) of clauses (with ors) Search and logic

coalescing buddies can be combined quickly Operating systems

cognitive neuroscience understanding higher-order functions Brain basics

color constancy ability to perceive invariant color despite ecological variations Computer vision · Vision

command interpreter shell - gets and executes next user-specified command Operating systems

communications between processes / computer systems Operating systems

community group relationships Psychology

complementary log-log model \(p(Y=1\vert x, \theta) = 1-\text{exp}[e^{-\theta^Tx}]\) Classification

complete terminates in finite steps Search and logic · Search and logic · Search and logic

completeness when all statements which are true (in some imageinable world), and which can be expressed as well-formed strings of the system, are theorems Cognitive science

complex cells simple cells with some shift invariance (oriented lines but with shifts) Computer vision

computational complexity how much computational effort needed to converge Learning theory

compute-server performs actions for user Operating systems

computer system hierarchical approach = layered approach Operating systems

concept learning acquiring the definition of a general category given a sample of positive and negative training examples of the category Learning theory · Search and logic

conclusion consequent = head Search and logic

concordance inter-rate reliability Evaluation

conditional mixture model variable z can be used to pick out regions of input space where different regression functions are used Stat basics

conditional plan considers more things RL and decisions

cone \(\theta \geq 0 \implies \theta x \in C\) Optimization

confidence predicted probability = confidence, max margin, entropy of predicted probabilities across classes Uncertainty

confidence interval if multiple samples of trained typists were selected and an interval constructed for each sample mean, 95 percent of these intervals contain the true preferred keyboard height Testing

confidence intervals take sample data + produce a range of values that likely contains population parameter you are interested in Stat basics · Interpretability

confounding difference between groups other than the treatment which affects the response Causal inference

conic form problem \(\min \: c^Tx \\ s.t. \: Fx +g \preceq_K 0\\Ax=b\) ~ set \(K=S_+^K\) Optimization

conjunction fallacy humans assign a higher probability to a proposition of the form “A and B” than to one of the propositions “A” or “B” in isolation Cognitive science

conjunction search conjunction of features (ex. red circle) takes longer Vision

conservative approximation contains belief state, but also extraneous stuff Search and logic

consistency \(Y=Y^{t=0}(1-T) + Y^{t=1}T\) - outcome agrees with the potential outcome corresponding to the treatment indicator Causal inference · Computer vision · Cognitive science

consistent \(\hat{\theta_n} \to some \: value\) Stat basics · Search and logic · Search and logic

consistent learner outputs hypotheses that perfectly fit training data whenever possible Learning theory

constant symbols stand for objects Search and logic

constituent structure words combine with other words to form units which are substitutable NLP

constitutive parts during event Philosophy

constraint graph nodes are variables and links connect any 2 variables that participate in a constraint Search and logic

constraint learning findining minimum set of variables/values from conflict set that causes the problem = no-good Search and logic

constraint logic programming allows variables to be constrained rather than bound Search and logic

constraint-based algorithms use conditional indep. checks to determine graphs up to markov equivalence Causal inference

constructivism jean piaget - cognitive development Cognitive science · Cognitive science

constructivists innate general learning mechanisms operate on experience Cognitive science

context switch occurs when switching processes Operating systems

context-free grammar set of grammar rules that define a gramm NLP

contiguous memory allocation each process has a section Operating systems

continuous limit thm Let (\(f_n\)) be a sequence of functions defined on A that converges uniformly on A to a function f. If each \(f_n\) is continuous at c in A, then f is continuous at c Real analysis · Real analysis

contraction function that brings variables together RL and decisions

contrapositive \(-q \to -p\) - logically equivalent Real analysis

contrastive loss InfoNCE loss uses cross-entropy loss to measure how well the model can classify the “future” representation among a set of unrelated “negative” samples Unsupervised

convex function lies below any chord Information theory

convex barrier limitation in the tightness of the bounds obtainable by the convex relaxation of the output of a neural network Deep learning theory

convex hull conv C = {\(\sum \theta_i x_i \: \vert x_i \in C, \theta_i \geq 0, \sum \theta_i = 1\)} Optimization

convex set \(x_1, x_2 \in C, 0 \leq \theta \leq 1 \implies \theta x_1 + (1 - \theta) x_2 \) Optimization

convolution cross-correlation where filter is flipped horizontally and vertically Computer vision

convolution theorem \(F[gh] = F[g]F[h]\) where F is Fourier, is convolution Computer vision

cookbookery forcing all problems into 1 or 2 routine techniques Data science

coordconv break translation equivariance by passing in i, j coords as extra filters Deep learning

copy-on-write allows parent and child processes intially to share the same pages Operating systems

cores cpus = processors - individual processing units available on a single node Languages and parallel computing

coreset algorithms summarize datasets with smaller dataset that accurately represent the full dataset on downstream tasks Interpretability

correlation coding correlations betweeen spikes carries extra info Computational neuroscience

cortex sheet-like arrays of neurons Brain basics

cortical dysplasia missing a layer / duplicating a layer Development

countable \(\mathbb{N}\)~A. Real analysis

counterfactual relating to or expressing what has not happened or is not the case Cognitive science

counterfactual explanations like adversarial, counterfactual explanation describes smallest change to feature vals that changes the prediction to a predefined output Interpretability

counterfactual fairness replace attributes w/ flipped values Fairness and STS

counterfactual memorization difference between a training data’s expected loss under a model that has and has not been trained on that data LLMs

counting keep track of ptr to next block and the number of free blocks after that Operating systems

counting semaphore unrestricted domain Operating systems

covariance rule \(\tau_w \frac{d\mathbf{w}}{dt} = \mathbf{x}(v-E[v])\) Computational neuroscience

covert attention posner cueing task Vision

CPG central pattern generator Motor system

CPU scheduling what to run first (memory -> cpu) Operating systems

CPU-bound process spends more time doing computations Operating systems

critical inpatient portality / transfer to an ICU within 12 hours ML in medicine

cross correlation what cnns do, dot product between kernel and neighborhood Computer vision

cross validation don't have enough data for a test set Evaluation

Cross-attention primary/context vectors can come from different places LLMs

Cuneate nucleus everything goes into this Sensory input

cup region where there are no axons Vision

cyclopia fatal defect in BMP Development

cytokines increase immune system (like neurotrophins), inflammation Development

D

d-separation directed separation Graphical models

daemons system programs for receiving connections (like servers waiting for a client) Operating systems

Darboux's thm if f is differentiable on an interval [a,b], and a satisfies f'(a) \(\alpha\) > f'(b)), then there exists a point c \(\in (a,b)\) where f'( c) = \(\alpha\) Real analysis

dask parallelizing tasks, distributed datasets, one or more machines Languages and parallel computing

data frame when you want different types of data Languages and parallel computing

data parallelism distributing subsets of data across cores and performing same operation on each core Operating systems

data snooping decide which hypotheses to test after examining data Testing

data-driven methods assume independence of cause and mechanism, like ICP, and use data from different shifts to find invariant subsets Causal inference

Datalog language restricted to first-order definite clauses with no function symbols Search and logic

DCI Eastwood and Williams (2018) - informativeness based on predicting gt factors using latent factors Disentanglement

deadlocked 2 processes are in waiting queues, can't wakeup unless other process signals them Operating systems

decision nodes points where decision maker has a choice of actions RL and decisions

decision procedure decides whether something is a theorem - must terminate Cognitive science

decision theory models don't require finding \(p(y \vert x)\) at all Classification

decoded filter rafegas & vanrell 2016 Interpretability

decoding P(stimulus \| response) Computational neuroscience

decomposability 2 consecutive lotteries can be compressed into single equivalent lottery RL and decisions

deconvolution transposed convolution = fractionally-strided convolution - like upsampling Deep learning

deductive general-to-specific Search and logic

deep more than 1 hidden layer Deep learning

deep deep cnns not just a chain, combine different layers and maybe represent semantic concepts Cognitive science

deep visualization yosinski 15 Interpretability

defective matrices lack a full set of eigenvalues Linear algebra

deferred cancellation target thread periodically checks whether it should terminate Operating systems

definite clause disjunction of literals with exactly one positive: ex. (\(A \lor \neg B \lor \neg C\)) Search and logic

defn humans are intelligent to the extent that our actions can be expected to achieve our objectives (given what we perceive) AI futures

delta-method sets comparable bw (wrt variance) after log or sqrt transform: \(Var(g(X)) \approx [g'(\mu_X)]^2 Var(X)\) where \(\mu_X = E(X)\) Data science

demand paging load pages only when they are needed Operating systems

demographic parity mean predictions for each group should be approximately equal Fairness and STS

density estimation given samples of X, estimate P(X) Stat basics

deontology rule-based ethics Psychology

dependence counterfactually - if cause didn't occur, effect wouldn't have occurred Psychology

depolarized less negative - we'll use +58 mV Brain basics

dequantization add noise (uniform) to discrete data Unsupervised

derivative theorem of convolution differentiation can also be though of as convolution Computer vision

descriptive theory how actual agents work RL and decisions

designation then given the expression, the system can affect the object Cognitive science

diagnostic model symptoms \(\to\) causes Graphical models

diagonalization eigenvalue decomposition = spectral decomposition Linear algebra

diameter of state space longest possible distance to goal from any start Search and logic

difference in means \(\hat \tau = \hat{\bar{Y}}^{T=1} - \hat{\bar{Y}}^{T=0}\) Causal inference

differentiable limit theorem let \(f_n \to f\) pointwise on the closed interval [a,b], and assume that each \(f_n\) is differentiable. If \((f'_n)\) converges uniformly on [a,b] to a function g, then the function f is ... Real analysis · Real analysis

differential privacy removing any data from the training set should not considerably change trained models LLMs

digital signal discrete in time and value Signal processing

direct access relative access Operating systems

direct causes parent nodes Causal inference

direct memory access read large chunks instead of one byte at a time Operating systems

direct memory access (DMA) transfers entire blocks of data w/out CPU intervention Operating systems

direct utility estimation treat states independently RL and decisions

directed arc consistency ordered variables \(X_i\), every \(X_i\) is consistent with each \(X_j\) for j>i Search and logic

discrete Fourier transform this is by far the most common Signal processing

discriminative model \(P(Y\vert X)\) directly Classification

disjoint pattern database break into independent possible subproblems Search and logic

disparity difference in image between eyes Computer vision · Computer vision

distributed system shares resources among separate computer systems Operating systems

doctors adapt to personal experience may be unfair to evaluate them on larger dataset ML in medicine

document representation of info that could satisfy need Information retrieval and databases

domain expert has domain knowledge Cognitive science

domain-adversarial techniques find a repr. which does not differ across environments, then predict Causal inference

dorsal horn top layer that controls sensory information Sensory input

dropout randomly zero outputs of p fraction of the neurons during training Deep learning

drusen show up as yellow spots on fundus image Vision

duality gap \(f_0(x) - g(\lambda, \nu)\) Optimization

dummy coding matrix is singular so we drop the last indicator variable - called reference class / baseline class Linear models

durbin-levinson algorithm the coefficients \(\phi_{n1}, \dots, \phi_{nn}\) can be computed recursively from the equations Time series

dynamic bayesian nets represents a temporal prob. model Graphical models

dynamic decision network online agent RL and decisions

dynamic loading don't load whole process, only load things when called Operating systems

dynamically linked libraries system libraries linked to user programs when the programs are run Operating systems

E

E-value min strength of association (risk ratio) that an unmeasured confounder would require with both \(T\) and \(Y\) to explain away a specific treatment-outcome association, conditional on \(X\) Causal inference

ecologically imposed if people are struggling to survive, a woman shouldn't share a man with another Psychology

edge place of rapid change in the image intensity function Computer vision

edinger westphal nucleus pupil-constriction Motor system

effect axioms specify the effect of an action at the next time step Search and logic

effective branching factor if total nodes generated by A is N and solution depth is d, then b is branching factor for uniform tree of depth d for N+1 nodes: \(N+1 = 1+b^ +(b^)^2 + ... + (b^)^d\) Search and logic

efficiency basically generalizability Philosophy

efficient coding in order to have maximum entropy output, a good encoder should match its outputs to the distribution of its inputs Computational neuroscience

efficient coding hypothesis brain learns different combinations (e.g. lines) that can efficiently represent images Computational neuroscience

efficiently PAC learnable time that is polynomial in \(1/\epsilon, 1/\delta, n, size(c )\) Learning theory

eidetic memorization a string s is k-eidetic memorized by LLM \(f\) if a prompt p exists such that \(f(p) = s\) and s appears at most k times in the training set LLMs

eigenvalue eqn \(Ax = \lambda x \implies (A-\lambda I)x=0\) Linear algebra

elastic net hybrid of the other two Linear models

electrophysiological recording uses electrodes, neuron-level Brain basics

elimination cliques includes X and its neighbors when X is removed Graphical models

ellipsoid {\(x \in \mathbb{R}^n \vert (x-x_c)^T P^{-1} (x-x_c) \leq 1\)} where P symmetric, PSD Optimization

elliptic envelope assume data is Gaussian and fit elliptic envelop (maybe robustly) to tell when data is an outlier Uncertainty

embedded systems like microwaves / robots Operating systems

emergentist appproach complex behavior is emergent from simple neural properties Cognitive science

Emile Durkheim father of modern sociology Psychology

empiricism aristotle - learning w/out representation Cognitive science · Cognitive science

empiricist morality is from childhood learning Psychology

encoding P(response \| stimulus) Computational neuroscience

endocrine get into your blood stream - body-wide signaling Brain basics

endogenous attention voluntary, slow, effortful, interruptible Vision

endogenous variable = x independent variable Time series

enhanced second-chance uses reference bit and modify bit Operating systems

ensemble uncertainty ensemble predictions yield uncertainty (e.g. variance within ensemble) Uncertainty

enteric system small ganglia / neurons in gut that influence gastric motility and secretion Brain basics

enumeration just try summing over all hidden variables Graphical models

epigenetics histones control transcription and other things Development

epiphenomenon a correlated effect (not a cause) Causal inference

epipolar plane contains cameras, point of fixation Computer vision

epipole intersection of \(\vec{c_1 c_2}\) and image plane Computer vision

epistemic rationality systematically improving the accuracy of your beliefs Cognitive science

epistemic uncertainty uncertainty in the DNN model parameters Uncertainty

epistemological commitments the possible states of knowledge that a logic allows with respect to each fact Search and logic

epistemology theory of knowledge Causal inference

equalized odds predictions are independent of group given label Fairness and STS

equifinality perturb behavior (like use grayscale images instead of color) and they can still do it (like stability) AI futures

equivalent heuristics compatible heuristics \(H\), \(G\) rank rules in the same order (antagonistic rank rules in opposite order) Search and logic

ergodic stronger condition, says samples approach the expectation of functions on the time series: for any function \(g\) and shift \(\tau_1, ... \tau_k\): Time series

ergodicity a time average is equivalent to averging over the s ensemble Computational neuroscience

ERM empirical risk minimization - minimize total training err Causal inference

essential matrix constraint \(x_1, x_2\) homogoneous coordinates of \(M_1, M_2 \implies x_2^T \hat{T} R x_1 = 0\) Computer vision

etiological leading up to event Philosophy

euclidean transformations isometries = rigid body transform Computer vision

event latency amount of time that elapses from when an event occurs to when it is serviced Operating systems

events actions Cognitive science

evidence marginal correlation, regression, invariance, causal Causal inference

ex. program induction given a dataset consisting of input/output pairs, generate a (simple?) program that produces the same pairs Interpretability

exact line search \(t = \underset{s \geq 0}{\text{argmin}} \:f(x+s \Delta x)\) Optimization

exclusion restriction \(Y_i^{I=1} = Y_i^{I=0}\) for always-takers and never-takers Causal inference

exclusion restrictions For every variable \(Y\) having parents \(PA(Y)\) and for every set of endogenous variables \(S\) disjoint of \(PA(Y)\), we have \(Y^{PA(y)} = Y^{PA(Y), S}\) Causal inference

existential instantiation variable is replaced by a new constant symbol Search and logic

exogenous attention involuntary, fast effortless, disruptive Vision

exogenous variable = y dependent variable Time series

exogenous variables noise variables = disturbances = errors - node in the network that represents all the data not collected Causal inference

expectation step values of unobserved latent variables are filled in Optimization

experience replay instead of learning from samples one by one, want to reduce correlation between subsequent samples RL and decisions

experiment experiment, RCT, natural experiment, observation Causal inference

expert system program that contains some of the subject-specific knowledge of one or more human experts. Cognitive science

explanandum sentence describing phemonenon to be explained Philosophy

explanans class of sentences used to account for the phenomenon Philosophy

explanation evaluation processes an explainee uses to evaluate the quality of an explanation Psychology

explanation generation explains the proxy (ex. just give most important 2 features in rule list proxy, ex. LIME gives coefficients of linear model, Shap: sums of elements) Interpretability

explanation selection process people use to select a small subset of identified causes as the explanation Psychology

explanation subsystem provide explanation of reasoning that led to conclusion Cognitive science

explicit graph methods assume explicit knowledge of graph representing the data-generating process Causal inference

exploding gradients problem gradients are significantly larger in earlier layers than later layers Deep learning

extended value extension \(\tilde{f}(x) = f(x)\) if \(x \in dom f\) else \(\infty\) Optimization

extensor extend (tricep) Motor system

external fragmentation there is enough free mem, but it isn't contiguous Operating systems

extreme value theorem if f if continuous on a compact set K, then f attains a maximum and minimum value. In other words, there exist \(x_0,x_1 \in K\) such that \(f(x_0) \leq f(x) \leq f(x_1)\) for all x \(\in\) K Real analysis

F

f-test test if any of non-zero means Testing

factor data type for storing categorical variable Languages and parallel computing

factor analysis models continuous latent variable Graphical models

factor tree if factors are made nodes, resulting undirected graph is tree Graphical models

faithful preserve info required for task Disentanglement

false positive rate FP / N \(= 1 - \text{specificity}\) Evaluation

feasible GLS Aitken estimator - use \(\hat{G}\) Linear models

feature-based grammars add things to word representations (e.g. plurality) and use these in the grammar rules NLP

fenchel's inequality \(f(x) + f^(y) \geq x^ty\) Optimization

FGFs axial symmetry Development

FID Frechet inception score works directly on embedded features from inception v3 model Unsupervised

file-server stores files Operating systems

filler one vector that embeds the content of the constituent Computational neuroscience

filopodium huge curves (fingers) Development

filtering state estimation - compute \(P(X_t \vert e_{1:t})\) Graphical models

find-S finding a maximally specific hypothesis Learning theory

first-fit allocate first hole big enough Operating systems

first-order definite clauses (remember this is a type of Horn clause) Search and logic

fisher null hypothesis \(H_{0F}: Y_i^{T=0} = Y_i^{T=1}\quad \forall i\) Causal inference

flexor curl in (bicep) Motor system

fluents state variables that change over time Search and logic

foreshortening objects slanted w.r.t line of sight become smaller w/ scaling factor cos \(\sigma\) ~ \(\sigma\) is angle between line of sight and the surface normal Computer vision

forward checking when we assign a variable in search, check arc-consistency on its neighbors Search and logic

forward pruning ignore some branches RL and decisions

forward/backward chaining checks if q is entailed by KB of definite clauses Search and logic

fraction of variance explained \(R^2 = \cblue{var(X \hat{\theta})} / \cpurple{var(Y)}\) Linear models

frame axioms assert that all propositions remain the same under actions Search and logic

frame representations semantic networks where nodes have structure Cognitive science

framing effect wording can influence people's judgements RL and decisions

free-energy principle \(a, \mu, m = \arg\min F(\tilde{s}, \mu \vert m)\) - minimization of the free energy of sensations and the representation of their causes Computational neuroscience

front-door criterion establishes if variables \(T\) and \(Y\) are confounded in a graph given mediators \(M\) Causal inference

Frontal Lobe This lobe controls a several elements including creative thought, problem solving, intellect, judgment, behavior, attention, abstract thinking, physical reactions, muscle movements ... Brain basics

frontier open list = set of all leaf nodes available for expansion at any given point Search and logic

fugue repeat w/ more substantial changes Cognitive science

fully observable agent knows its state RL and decisions

function symbols stand for functions Search and logic

functional brain imaging noninvasive, records local activity Brain basics

functional causal models assume a variable can be written as a function of its direct causes and some noise term Causal inference

fuzzy rules probabilistically split Search and logic

G

gain modulation adding constant amount to input h multiplies the output Computational neuroscience

gain-of-function lesion excitatory lesion - like epilepsy Sensory input

galvanic skin response rises if people hear their own voice Psychology

Gamma \(\Gamma(n)=(n-1)!=\int_0^\infty x^{n-1}e^{-x}dx\) Math basics

ganglia accumulations of cell bodies / supporting cells Brain basics

gastrulation process by which early embryo undergoes folds = shapes of NS Development

Gauss-markov thm assume linear model and assumption above: when X is fixed, OLS estimator is BLUE = best linear unbiased estimator Linear models

gaussian filter blur, better than just box blur Computer vision

general hypothesis more generally true Learning theory

generalized eigenvalue decomposition for 2 symmetric matrices Linear algebra

generalized mean value theorem if f and g are continuous on the closed interval [a,b] and differentiable on the open interval (a,b), then there exists a point c \(\in (a,b)\) where \|f(b)-f(a)\|g'( c) = \|g(b)-g(a)\|f'( ... Real analysis

generate permutations recursive, add char at each spot Algorithms

generative data are generated by a given stochastic model Data science · Classification

genetic algorithms population of k individuals Search and logic

geometric margin functional margin / \(\vert \vert \theta \vert \vert \) Classification

geometric series \(\sum_{n=0}^\infty a r^n = \frac{a}{1-r}\) Real analysis

Gittins index function of number of pulls / payoff RL and decisions

global constraint arbitrary number of variables (doesn't have to be all) Search and logic

global robustness maximize objective subject to robustness constraint (trades off robustness with objective value) Deep learning theory

Glutamate excites pyramidal cells Brain basics

glutamate increase excitotoxicity Development

GMMs \(p(x\vert \theta) = \underset{i}{\Sigma} \pi_i \mathcal{N}(x\vert \mu_i, \Sigma_i)\) Unsupervised

goal 1st step Search and logic

goal clause no positive: ex. (\(\neg A \lor \neg B \lor \neg C\)) Search and logic

godel's thm limitation of any formal axiomatic system: cannot make a program to find a complete + consistent set of axioms Cognitive science

good explanation hard to vary while still accounting for what it purports to account for Philosophy

Grad-dot \(I\left(x_{\text {train }}, x_{t e s t}\right)=\nabla_{\theta} L\left(x_{\text {train }}, \hat{\theta}\right)^{T} \nabla_{\theta} L\left(x_{\text {test }}, \hat{\theta}\right) .\) Interpretability

graded depends on how strong the input is Brain basics

gradient vector that points to direction of maximum increase Optimization

Grand central dispatch extends C Operating systems

grandiosity the person who is "grandiose" is admired everywhere and needs this admiration; indeed, he cannot live without it. Psychology

gray matter has more cell bodies Brain basics

greedy best-first search let f = h: keep expanding node closest to goal Search and logic

grid cells in the entorhinal cotex (provides inputs to the hippocampus) - not particular locations but rather hexagonal coordinate system Computational neuroscience

grid-based hypothesis tests splitting the sample space (X, Y) into grids and then testing whether the patterns of sample distributions across different grid cells are random Interpretability

grounding connection between logic and real environment (usually sensors) Search and logic

GroupDRO distributionally robust optimization (e.g. encourage strict equality between err of each group) Causal inference

grouping block stores n-1 free blocks and 1 pointer to next block Operating systems

Grover's algorithm with quantum computers, something like you can only use sqrt of number of steps Quantum computing

growth cone motile tip of axon Development

H

H & E hematoxylin and eosin stain - most popular (~80%) - colors the cells in a specific way, bc cells are usually pretty transparent Disease

H-statistic 0 for no interaction, 1 for complete interaction Interpretability

habituation decrease vesicles on sensory neuron after too much stimuli Brain basics

hacking intervals the range of a summary statistic one may obtain given a class of possible endogenous manipulations of the data Interpretability

Hedge hogs at floor plate Development

Heine-Borel thm let K \(\subseteq \mathbb{R}\). All of the following are equivalent Real analysis

heteroassociative can return stored X based on its address A Computational neuroscience

Heuristic Search Hypothesis solutions are represented as symbol structures. A physical symbol system exercises its intelligence in problem solving by search--that is, by generating and progressively modifying symbol ... Cognitive science

hierarchical paging ex. two-level page table Operating systems

higher-order logic views relations and functions as objects in themselves Search and logic

highly stat. significant p = 0.01 Testing

Hilbert space vector space w/ additional structure of inner product which allows length + angle to be measured Deep learning theory

hill-climbing greedy local search Search and logic

histamine not well-known Brain basics

homeotic genes hox genes - set of genes (transcription factors) in order on chromosome Development

homogenous coordinates use n + 1 coordinates for n-dim space to help us represent points at \(\infty\) Computer vision

homoscedasticity \(\text{var}(Y_i\vert X)\) is the same for all i Linear models

horizon effect a position is bad but isn't apparent for a few moves RL and decisions

horn clause at most one positive Search and logic

hospitalization met with an inpatient care site admission immediately following an ED visit ML in medicine

human fMRI datasets https://docs.google.com/document/d/1bRqfcJOV7U4f-aa3h8yPBjYQoLXYLLgeY6_af_N2CTM/edit Computational neuroscience

Huntington's death of specific class of spiny neurons Motor system

hyperalgesia increased pain sensitivity Sensory input

hypercolumn cells of different orientations, scales grouped close together for a location Computer vision · Vision

hypercomplex cells complex cell, but only lines of certain length Computer vision

hyperplane {\(x\vert a^Tx = b\)} ~ creates a halfspace Optimization

hypothalamus controls a lot Motor system

hysteresis thresholding high threshold to start edge curves then low threshold to continue them Computer vision

I

I/O operations file or device Operating systems

I/O-bound process spends more time doing I/O Operating systems

ICP invariant causal prediction - find feature set where, after conditioning, loss is the same for all environments Causal inference

idea demon makes all fast particles go to hot side, all slow particles go to slow side - this is against entropy Quantum computing

iFoldRNA three-dimensional RNA structure prediction and folding (sharma et al. 2008 ) Omics

ill-conditioned matrix matrix is close to being singular - very small determinant Linear algebra

immunohistochemistry (IHC) tries to identify cell lineage: 10-15% Disease

implicit threading handle threading in run-time libraries and compilers Operating systems

important different spatial/temporal frequencies - differentiate from the beginning Vision

indefinite blocking=starving could happen if we remove processes from waiting queue in LIFO order Operating systems

independence restrictions variables are independent of noise variables given their markov blanket Causal inference

independent-spike coding each spike is independent of other spikes within the spike train Computational neuroscience

individual fairness similar individuals should be treated similarly Fairness and STS

inductive specific-to-general Search and logic

inductive logic programming propose new concepts and definitions in order to identify theories that are both accurate and concise AI futures

inference prediction/association, intervention, counterfactuals Causal inference

influence functions \(I\left(x_{\text {train }}, x_{\text {test }}\right)=\) ... Interpretability

information filter mathematically the same but different parameterization Graphical models

information gain decrease in entropy Classification

information need desire to locate and obtain info to satisfy a need Information retrieval and databases

information value theory enables agent to choose what info to acquire RL and decisions

informed search use path costs \(g(n)\) and problem-specific heuristic \(h(n)\) Search and logic

Inner Alignment This is to ensure that AI systems are actually trained to achieve the goals set by their designers. Interpretability

innovations one-step prediction errors \(U_n = X_n - \hat X _n\) Time series

instance method invoked on specific instance of the class Languages and parallel computing

instantaneous code no code word is a prefix of another code word Information theory

instrument measurable quantity that correlates with the treatment, and is \(\underbrace{\cblue{\text{only related to the outcome via the treatment}}}_{\textbf{exclusion restriction}}\) Causal inference

instrumental rationality systematically achieving your values Cognitive science

instrumental variable criterion sufficient condition for identifying the causal effect \(P(y\vert do(t))\) is that every path between \(T\) and any of its children traces at least one arrow emanating from a measured variable Causal inference

insular flashes of moods Motor system

insular cortex emotional part of brain Sensory input · Motor system

integrable limit thm Assume that \(f_n \to f\) uniformly on [a,b] and that each \(f_n\) is integarble. Then, f is integrable and \(\lim_{n \to \infty} \int_a^b f_n = \int_a^b f\). Real analysis · Real analysis

intelligent agent system that perceives its environment and takes actions that maximize its chances of success Cognitive science

inter-class discriminative features of samples from different classes/clusters are uncorrelated + different low-dim linear subspaces Disentanglement

interior extremum thm let f be differentiable on an open interval (a,b). If f attains a maximum or minimum value at some point c \(\in\) (a,b), then f'( c) = 0. Real analysis

intermediate value theorem Let f:[a,b]\( \to R\) be continuous. If L is a real number satisfying f(a) L > f(b), then there exists a point c \(\in (a,b)\) where f( c) = L Real analysis

internal fragmentation extra mem a proc is allocated but not using (because given in block sizes) Operating systems

interpretation run programs as non-native code (ex. java runs on JVM) Operating systems · Search and logic · Cognitive science · Cognitive science

interprocess communication (IPC) allows exchange of data and info Operating systems

interrupt chaining each element in the interrupt vector points to the had of a list of interrupt handlers Operating systems

interrupt latency period of time from the arrival of an interrupt at the CPU to the start of the routine that services the interrupt Operating systems

intra-class compressible features of samples from same class/cluster are correlated (i.e. belong to low-dim linear subspace) Disentanglement

intrafusal muscle=spindle stretches the proprioceptor so that it can measure even when muscle is already stretched Motor system

introjection the unconscious adoption of the ideas or attitudes of others Psychology

intuitionism calls reasoning rider and intution elephant Psychology

inverse transform \(x_n = \frac{1}{N} \sum_{k=0}^{N-1} X_k\cdot e^{i 2 \pi k n / N}\) Signal processing

ion channels make membranes selectively permeable - wires Brain basics

ionotropic channel-linked receptors Brain basics

ipsilateral same side Sensory input

iris colored annulus w/ radial muscles Computer vision

IRM invariant risk minimization (v1) - find a feature repr. such that the optimal classifier, on top of that repr., is the identity function for all environments Causal inference

irradiance how much light (photons) is captured in some time interval Computer vision

Ising model if x is binary Graphical models

isolated point not a limit point Real analysis

IsRNA1 De Novo Prediction and Blind Screening of RNA 3D Structures (zhang et al. 2021) Omics

iterative deep aggregation generalizes skip connections - new layers for the skips Cognitive science

iterative deepening dfs like bfs explores entire depth before moving on Search and logic

iterative lengthening search instead of depth limit has path-cost limit Search and logic

iterative-deepening iterative deepening with cutoff f-cost Search and logic

J

Jacobian matrix ... Linear algebra

jensen's \(f(E[X]) \leq E[f(X)]\) for convex \(f\) Stat basics

Jensen's inequality if f is a convex function and X is an R.V., \(f(E[X]) \leq E[f(X)]\) Information theory

job scheduling picking jobs from job pool (disk -> memory) Operating systems

julesz conjecture textures can't be spontaneously discriminated if they have same first-order + second-order statistics (ex. density) Computer vision

K

kernel density plot Gaussian kernels Data science

knowledge engineer computer scientist who designs / implements ai Cognitive science

knowledge-based agents intelligence is based on reasoning that operates on internal representations of knowledge Search and logic

kolmogorov-smirnov statistic \(D = \max_y\vert \hat F_1(y) - \hat F_0 (y)\vert\) where \(\hat F_1(y) = \frac 1 {n_1} \sum_i T_i 1(Y_i \leq y)\), \(\hat F_0(y) = \frac 1 {n_0} \sum_i (1 -T_i) 1(Y_i \leq y)\) Causal inference

Kraft inequality \(\sum_x 2^{-\ell(x)} \leq 1\) Information theory

Kullback-Leibler divergence calculates difference between 2 distributions Computational neuroscience

L

L'Hospital's Rule: 0/0 let f and g be continuous on an interval containing a, and assume f and g are differentiable on this interval with the possible exception of the point a. If f(a) = g(a) = 0 and g'(x) ≠ 0 ... Real analysis

l1 loss laplace distr. Disentanglement

lagrange dual problem \(\max \: g(\lambda, \nu)\\s.t. \: \lambda \succeq 0\) Optimization

Lagrange's Remainder thm Let f be differentiable N+1 times on (-R,R), define \(a_n = \frac{f^{(n)}(0)}{n!}.....\) Real analysis

Lagrangian solves minimize f subject to g = c Calculus

lambertian surfaces outgoing radiance same in all directions Computer vision

lamellipodium sheet (hand) Development

LAN local-area network connects computers within a room, building, or campus Operating systems

LATE local average treatment effect - this estimate is only valid for the patients who were influenced by the instrument Causal inference

latent vars values not specified in the observed data Unsupervised

lazy pager only swaps a page into memory when it is needed Operating systems

learnable upsampling deconvolution = upconvolution = fractionally strided convolution = backward strided convolution - transpose the convolution Computer vision

learning form of EM Graphical models · Computational neuroscience

leave-one-out calculate loss on \(x_{test}\) when trained on all points except \(x_{\text{train}}\) - loss when trained on all points Interpretability

legitimate target variable is unobservable target characteristic (e.g. strength) one that can justify hiring disparities? Fairness and STS

lemma Let \(p_1,...,p_M\) and \(q_1,...,q_M\) be arbitrary positive numbers with \(\sum p_i = \sum q_i = 1\). Then \(-\sum p_i \log p_i \leq - \sum p_i \log q_i\). Only equal if \(p_i = q_i \: \forall i\) Information theory

leverage scores measure how much each \(x_i\) influences the LS fit Linear models

libc provided by Linux/Mac OS for C Operating systems

lightweight process LWP - between user thread and kernel thread Operating systems

likelihood ratio statistic compare distr in rule to distr in full dataset Search and logic

likelihood weighting fix evidence to be more efficient Graphical models

LIM like LIME Interpretability

limit register specifies size of range Operating systems

line search repeatedly double \(\alpha\) until f starts to increase again Search and logic

linear filtering can be modeled by convolution Computer vision

linear Gaussian for continuous children Graphical models

link pointer to another thing Operating systems

lipofuscin undigested material from photoreceptor turniover accumulates in the RPE (and some in Bruch's membrane) Vision

list-then-eliminate list all hypotheses and eliminate any that are inconsistent (slow) Learning theory

lists ordered, can hold any data type Languages and parallel computing

LMS least mean squares (p-dimensional geometries) Linear models

load balancing try to make sure all processes share work Operating systems

local beam search pick k starts, then choose the best k states from their neighbors Search and logic

local circuit neurons interneurons - short axons Brain basics

local outlier factor score based on nearest neighbor density Uncertainty

local replacement algorithm priority replacement algorithm - if one process starts thrashing, cannot steal frames from another Operating systems

local robustness maximize radius surrounding parameter subject to all constraints (no objective to maximize) Deep learning theory

locality model each locality is a set of pages actively used together Operating systems

localize factual associations causal intervention for identifying neuron activations that are decisive in a model’s factual predictions LLMs

locked-in syndrome can only move eyes vertically Motor system

locking protecting critical regions using locks Operating systems

loess multiple predictors / lowess - only 1 predictor Data science

logical inference showing entailment Search and logic

logit distr. \(P(buys\vert Cost=c) =s\left(\frac{-2 (\mu - c)}\sigma \right)\) Graphical models

long-term scheduler selects from processes on disk to load into memory Operating systems

look-ahead mask might want to mask to only use previous words (e.g. if our final task is decoding) LLMs

loss cross-entropy (with binary labels) ends up just the likelihood of the data: NLP

loss-of-function lesion piece of cortex is lost - lose awareness Sensory input

lots of good data http://home.earthlink.net/~perlewitz/index.html Computational neuroscience

lottery any function of a random variable RL and decisions

low-level formatting dividing disk into sectors that controller can read/write Operating systems

lower sum sum all the possible smallest rectangles Real analysis

lowpass filter removes high frequencies Computer vision

luddite person opposoed to new technology or ways of working - stems from secret organization of english textile workers who protested AI futures

Lyapunov eqn evolution of variance of states Graphical models

M

MA model \(X_t = \sum_{i=1}^q \cgreen{\theta_i} \corange{\varepsilon_{t-i}} + \corange{\varepsilon_t}\) Time series

madonna-whore dichotomy a psychological phenomenon which groups females into 2 categories: marriage / fling Psychology

maintaining arc consistency (MAC) when we assign a variable, call AC-3, intializing with arcs to neighbors Search and logic

MAN metropolitan-area network Operating systems

Manichaeism polarization, believing one side only Psychology

many-to-one model maps user-level threads to one kernel thread Operating systems

Markov chain monte carlo ex. Gibbs sampling, Metropolis-Hastings Graphical models

markov condition given parents, node is conditionally independent of its non-descendants Graphical models

markov decision process all these things are given RL and decisions

markov's \(P(X \geq a) \leq \frac{E[X]}{a}\) Stat basics

mathematistry development of theory for theory's sake Data science

max-unpooling use positions from pooling layer Computer vision

maximally diverse dimension (or variance) of features for each class/cluster should be as large as possible as long as uncorrelated from other classes/clusters Disentanglement

maximization step parameters are adjusted based on filled-in variables Optimization

mean value theorem if f:[a,b] -> R is continuous on [a,b] and differentiable on (a,b), then there exists a point c \(\in\) (a,b) where \(f'( c) = \frac{f(b)-f(a)}{b-a}\) Real analysis

mean-field variational inference each point has its own params (e.g. different encoder DNN) vs amortized inference - same encoder for all points Unsupervised

measure nonnegative real function from a delta-ring F such that \(m(\empty) = 0\) and \(m(A) = \sum_n m(A_n)\) Scattering transform

mechanism how to do something Operating systems · Psychology

memory resident residing in memory Operating systems

message passing messages between coordinating processes Operating systems

message-passing protocol a node can send a message to a neighboring node when, and only when, it has received messages from all of its other neighbors (parallel algorithm) Graphical models

meta-knowledge knowledge about what we know Cognitive science

metabotropic G-protein-coupled receptors Brain basics

metamer 2 different but indistinguishable spectra Computer vision

metric entropy want functions to be close (within epsilon) Complexity

microangiopathy mainly affects capillaries Vision

microelectrode fine glass tubing filled with good conductor Brain basics

microglia from immune system Development

microkernel approach used in os Mach Operating systems

middleware set of software frameworks that provide additional services to application developers Operating systems

midline center of nervous system Sensory input

minis MEPP - not big enough to fire the neuron Brain basics

mistake bound how many training examples will learner misclassify before converging Learning theory

mixture models discrete latent variable Graphical models

MN pool group of MNs=motor units Motor system

model averaging \(p(x_{new}\vert x) = \int dm \int d\theta \: p(x_{new}\vert \theta, m) p(\theta\vert x, m) p(m\vert x)\) Stat basics

model checking try everything to see if A \(\implies\) B Search and logic

model does not actually change that much predictions when manipulating pixels in order of saliency remains similar, very different from random (fig 4) Interpretability

model reliance MR - like permutation importance, measures how much a model relies on covariates of interest for its accuracy Interpretability

Model-based RL estimate the transition model, and then... RL and decisions

model-class reliance MCR = highest/lowest degree of MR within a class of well-performing models Interpretability

modify bit dirty bit reduces overhead Operating systems

modularity if each dimension of r(x) depends on at most a factor of variation using their mutual info Disentanglement

moment parameterization \(p(x\vert\eta, \Omega) = \text{exp}\left( a + \eta^T x - \frac{1}{2} x^T \Omega x\right)\) ~ also called information parameterization Stat basics

monitor highl-level synchronization construct Operating systems

monotone increasing or decreasing (not strictly) Real analysis

monotone convergence thm if a sequence is monotone and bounded, then it converges Real analysis

monotonicity \(P(C=d) = 0\) or \(T_i^{U=1} \geq T_i^{U=0} \; \forall i\) - there are no defiers Causal inference · Search and logic · RL and decisions

morphology study of the internal structure of words in a language NLP

most likely explanation \(\underset{x_{1:t}}{\text{argmax}}\:P(x_{1:t}\vert e_{1:t})\) Graphical models

motivation explaining plays a motivational role (e.g. gopnik 2000 "Explanation as orgasm") Psychology

Motor Cortex This helps the brain monitor and control movement throughout the body. It is located in the top, middle portion of the brain. Brain basics

mse loss gaussian distr. - tends to focus on a fex pixels that are very wrong Disentanglement

MSO - medial superior olive decides where the sounds is coming from Sensory input

multi-headed attention just like having many filters, get many encodings for each word LLMs

multicollinearity predictors highly correlated Linear models

multiple-partition method free partitions take a new process Operating systems

multiplication A B = (-1, -1, -1, 1, 1, -1, 1, ...) - this is XOR Computational neuroscience

multiprogramming increases CPU utilization so CPU is always doing something Operating systems

multiset bag (stores set with frequency counts) - can store things with order by adding them multiple times, but hard to actually retrieve frequencies Computational neuroscience

mutual exclusion 2 processes can't concurrently do critical section Operating systems

mutual info gap MIG Chen et al. 2018 - mutual info to compute the same thing Disentanglement

mutual info I(X; Y) how much you can predict about one given the other Information theory

MYCIN one of first successful expert systems { cite shortliffe2012computer } Cognitive science

myopic greedily obtain evidence which yields highest VPI until some threshold RL and decisions

N

n-armed bandit pulling n levelers on a slot machine, each with different distr. RL and decisions

nadaraya-watson kernel smoother locally weighted scatter plot smoothing Data science

naked predictions numbers with no real interpretation Philosophy

named pipe can be bidirectional Operating systems

named tuples have smth like (name: x, date: m, age: y) and store as holistic vector \(H = NX + D M + A Y\) Computational neuroscience

nativism plato - representation w/out learning Cognitive science · Cognitive science

nativist morality is innate Psychology

nativists children guided by universal grammar Cognitive science

negative exposure assume we have a secondary informative treatment \(T'\) Causal inference

negative outcome assume we have a secondary informative outcome \(Y'\) Causal inference

negative predictive value TN / (FN + TN) Evaluation

nerve bundles of axons in PNS Brain basics · Sensory input

Nested Compact Set Property intersection of nested sequence of nonempty compact sets is not empty Real analysis

nested interval property for each \(n \in N\), assume we are given a closed interval \(I_n = [a_n,b_n]=\{ x \in \mathbb{R} : a_n \leq x \leq b_n \}\) Assume also that each \(I_n\) contains \(I_{n+1}\). Then, the resulting ... Real analysis

netrin/dcc guidance moleculereceptor = DCC Development

network communication path between two or more computers Operating systems

network computers are essentially terminals that understand web-based computing Operating systems

neural crest pinches off top of roofplate Development

neuroblastoma developed early - severe problem because missing parts of NS Development

neuroblasts classic stem cells Development

neuroethology complex behaviors of animals Brain basics

neuroexins further recognition Development

neuron feature weighted average version of a set of maximum activation images that capture essential properties - rafegas_17 Interpretability

neuropil bundle of dendrites/axons/glia - where synaptic connectivity occurs Brain basics

Neyman-Pearson lemma likelihood ratio test is the most efficient statistic, in that is has the most power for a given size Computational neuroscience

neyman-rubin model \(Y_i = \begin{cases} Y_i^{T=1}, &\text{if } T_i=1\\Y_i^{T=0}, &\text{if } T_i=0 \end{cases}\) Causal inference

niche places where stem cells stay alive Development

no interference my outcome is unaffected by anyone else's treatment Causal inference

node different computers with distinct memory Languages and parallel computing · Search and logic

node consistency prune domains violating unary constraints Search and logic

nogo protein that blocks regrowth (but there are other proteins as well) Development

non-linear ica \(X \approx f(z)\), where assumptions on \(z\) are the same, but \(f\) can be nonlinear Unsupervised

non-maximum suppression does thinning, check if pixel is local maxima Computer vision

nonconformity measure how unusual an examples looks relative to previous examples Stat basics

nonsingular invertible = nonzero determinant = null space of zero Linear algebra

norm cone {\((x, t)\vert \: \vert \vert x\vert \vert \leq t\)} Optimization

normalized cuts partition so within-group similarity is large and between-group similarity is small Computer vision

normative theory how idealized agents work RL and decisions

nuclei local accumulations of similar neurons Brain basics

nucleus bundle of neurons related to some function Sensory input

O

object recognition visual similarity via labels Computer vision

objective minimize the average word length \(\sum p_i n_i\) where \(n_i\) is average word length of \(x_i\) Information theory

observed stat. significance level P-value - area of normal curve beyond \(\pm \hat{\theta_k} / \hat{SE}\) Testing

OCT optical coherence tomography Vision · Computational neuroscience

ocular dominance columns one eye has bands, other eye has other bands Vision

odds ratio \(\frac{P(Y=1\vert T=1) / P(Y=0\vert T=1)}{P(Y=1\vert T=0) / P(Y=0\vert T=0)}\) Causal inference

Oja's rule \(\tau_w \frac{d\mathbf{w}}{dt} = \cblue{\mathbf{x}v}- \corange{\alpha v^2 \mathbf{w}}\) where \(\alpha>0\) Computational neuroscience

oligodendrocytes in CNS, lay down myelin - in PNS, Schwann cells do this Brain basics

oligonucleotide oligo = short single strands of synthetic DNA or RNA Omics

one-vs-one all-vs-all Classification

one-way CV prequential analysis - keep testing on next data point, updating model Evaluation

ontologies structuring knowledge in graph form Cognitive science

ontology study of being, concepts, categories Causal inference

OpenMP set of compiler directives / API for parallel programming Operating systems

optic neuritis ms optic nerve disease that shifts map Development

optical flow movement in the image plane Computer vision

optical flow constraint equation \(I_x u + I_y v + I_t = 0\) Computer vision

optimal finds best solution Search and logic

optimal brain damage starts with fully connected and weeds out connections (Lecun) Deep learning

optimal duality gap \(p^ - d^\) Optimization

optimizer's curse tendency for E[utility] to be too high because we keep picking high utility randomness RL and decisions

optokinetic system tracks with eyes Motor system

order limit thm Assume lim \(a_n = a\) and lim \(b_n\) = b. Real analysis

orderability A>B, A~B, or A<B RL and decisions

ordered statistics variables \(Y_i\) such that \(Y_i\) is the ith smalless Stat basics

ordinary pipe write at one end, read at the other Operating systems

orthogonal complement set of orthogonal vectors Linear algebra

orthogonal matrix all columns are orthonormal Linear algebra

orthogonal transformations preserves inner products \(\forall a,b \: a \cdot b =a^T A^TA b\) Computer vision

orthographic projection appproximation to perspective when object is relatively far Computer vision

Outer Alignment choose the right loss functions or reward fuctions and ensure that the training objectives of AI systems match human values. Interpretability

P

P-quadratic norm \(\vert \vert x\vert \vert _P = (x^TPx)^{1/2} = \vert \vert P^{1/2} x \vert \vert _2\) where \(P \in S_{++}^n\) Linear algebra

p-value probability, calculated assuming that the null hypothesis is true, of obtaining a value of the test statistic at least as contradictory to \(H_0\) as the value calculated from the available ... Testing

PAC learnable consider concept class C defined over set of instances X of length n and a learner L using hypothesis space H Learning theory

padding mask add zeros to the end of the sequence LLMs

page-fault frequency add / decrease pages based on targe page-fault rate Operating systems

pairs could just multiply (XOR), but then get some weird things, e.g. A A = 0 Computational neuroscience

PAN personal-area network (like bluetooth) Operating systems

paracrine medium distance, neurotransmitter sprinkled and nearby targets take it up Brain basics

parallax effect where near objects move when you move but far don't Computer vision

parasympathetic relaxing (ACh) Motor system

parent process that created another process Operating systems

Parietal Lobe this lobe focuses on comprehension. Visual functions, language, reading, internal stimuli, tactile sensation and sensory comprehension will be monitored here. Brain basics

Parkinson's specific loss of dopaminergic neurons Motor system

parochial having a limited or narrow outlook or scope; of or relating to a church parish Psychology

pars compacta dopaminergic neurons (input to striatum) Motor system

partial correlation coefficient wipe out correlations due to other variables Interpretability

passive given \(\pi\), learn \(U^\pi (s)\) RL and decisions

path consistency consider constraints on triplets - PC-2 algorithm Search and logic

path model graphical way to represent a regression equation Linear models

pathological staging elements of staging pTNM Disease

pathology branch of medicine where you take some tissue from a patient (e.g. tumor), look at it under a microscope, and make an assesment of what the disease is ML in medicine

pattern database heuristic stores exact solution cost for every possible subproblem instance Search and logic

pattern matching logic in the if statements Cognitive science

pattern Rashomon ratio considers unique predictions on the data (called “patterns”) rather than the count of functions themselves. Interpretability

pdp plots marginals (force value of plotted var to be what you want it to be) Interpretability

percept tree describe what's in an image using a tree Computer vision

perceptron tries to find separating hyperplane Classification

perceptron convergence thm if data is linearly separable, perceptron learning algorithm wiil converge Deep learning

performance knowledge about how to do things Cognitive science

performance tuning removing bottlenecks Operating systems

permutation shuffles values (like bit-shift) Computational neuroscience

permutation equivariant \(f(PX) = P f(X)\) - useful for when we want answers at the node level Deep learning

permutation invariant \(f(PX) = f(X)\) for all permutation matrices \(P\) Deep learning

perplexity how to balance between local/global aspects of data Unsupervised

perplexity (PP) exponent of the loss NLP

Perron-Frobenius thm if A is a positive matrix (positive values), so is its largest eigenvalue and every component of the corresponding eigenvector is also positive Linear algebra

persistent failure model additional variable describing status of battery meter Graphical models

peter-clark (PC) algorithm first learns undirected graph, then detects edge directions and returns equivalence class Causal inference

philosophy organization of science Philosophy

phonetics study of the physical aspects of sounds NLP

phonology study of the organization of sounds independent of their physical realization in speech / sign language NLP

photo-transduction converts photons into voltage-changes Vision

physical symbol system hypothesis a physical symbol system has the necessary and sufficient means for general intelligent action Cognitive science

pipes conduit allowing 2 processes to communicate Operating systems

PL condition Polyak-Lojawsiewicz condition guarantees global convergence of loca methods Deep learning theory

place cells in the hippocampus - fire to indicate one's current location Computational neuroscience

planning devising a plan of action to achieve one's goals Search and logic

Planning Domain Definition Language uses factored representation of world Search and logic

planning fallacy people think they can plan e.g. "best guess" scenarios are same as "best case" scenarios Cognitive science

ply half a move in a tree RL and decisions

point estimate single number prediction Stat basics

point estimator statistic that predicts a parameter Stat basics

policy what will be done Operating systems

policy evaluation given a policy \(\pi_i\), calculate \(U_i=U^{\pi_i}\), the utility of each state if \(\pi_i\) were to be executed RL and decisions

Policy gradients directly differentiate the above objective RL and decisions

policy improvement calculate a new MEU policy \(\pi_{i+1}\) using \(U_i\) RL and decisions

politics arrangements of power and authority in human associations as well as the activitites that take place within those arrangements Fairness and STS

polyhedron {x \| Ax=b, Cx=d} = \(\{ \sum_i^k \theta_i v_i \; \vert \sum_i^m \theta_i = 1, \theta_i \geq 0 \} \: m \leq k\) Optimization

polytree singly-connected network - only 1 undirected paths between any 2 nodes Graphical models · Graphical models

pontine micturation center > parasympathetic preganglionic neurons -> parasympathetic ganglionic neurons Motor system

population vector sums vectors for cells that point in different directions weighted by their firing rates Computational neuroscience

population vector coding ex. neurons coded for direction sum to get final direction Computational neuroscience

pose position and orientation of object w.r.t. the camera (6 numbers - 3 translation, 3 rotation) Computer vision

position coding want to represent a position Computational neuroscience

position vector add this into the embedding of each word (so words know how far apart they are) - usually use sin/cos rather than actual position number LLMs

positive semi-definite \(A \in R^{nxn}\) Linear algebra

positivity overlap = common support - probability of receiving any treatment is positive for every individual Causal inference

post-intervention distribution result of setting \(T = t_0\) Causal inference

potential outcomes counterfactual outcomes \(Y^{t=1}, Y^{t=0}\) Causal inference

Potts model x is multiclass Graphical models

poverty of the stimulus children get this info which doesn't seem to be in the stimulus Cognitive science

power \(1 - \beta\) Testing

power method want to find eigenvector \(v\) corresponding to largest eigenvalue Linear algebra

power spectrum how much of the signal is at a frequency \(\omega\)? - square of the magnitude of the coefficients of the Fourier coefficients for \(\omega\) Signal processing

pragmatics study of the way in which context during communication contributes to meaning NLP

preattentive vision parallel, instantaneous (~100-200 ms) Computer vision

precision positive predictive value = TP / (TP + FP) Evaluation

predicate symbols stand for relations Search and logic

prediction compute \(P(X_{t+k}\vert e_{1:t})\) for \(k>0\) Graphical models · Causal inference

predictive use algorithmic model and data mechanism unknown Data science

predictive coding use surrounding regions to predict the center value Vision · Vision

predictive rate parity Y is independent of group given prediction Fairness and STS

preemptive can stop and switch a process that is currently running Operating systems

preference bias search bias - models can search entire space (e.g. NN, decision tree) Stat basics

preference elicitation finds utility function RL and decisions

prefrontal loop daydreaming (higher-order function) Motor system

presenting problem the issue that sends a person into therapy Psychology

preservation of compact sets if f continuous and K compact, then f(K) is compact as well Real analysis

primitives `int, float, bool, str` Languages and parallel computing · Languages and parallel computing

prior distribution distribution of stimulus Computational neuroscience

prioritized sweeping prefer adjustments to states whose likely successors have just undergone a large adjustment in their own utility estimates (speeds things up) RL and decisions

proactive methods make assumptions about possible set of target distrs. Causal inference

probit distr \(P(buys\vert Cost=c) = \phi[(\mu - c)/\sigma]\) - integral of standard normal distr Graphical models

process program loaded into memory Operating systems · Operating systems

process contention scope competition for CPU takes place among threads belonging to same process Operating systems

process control block (PCB) task control block - repository for any info that varies process to process Operating systems

process scheduler selects available process for multi-tasking Operating systems

processes instances of a program executing on a machine Languages and parallel computing

processor affinity try to not switch too much Operating systems

profiling periodically samples instruction pointer to determine which code is being executed Operating systems

prognosis chance of recovery Disease

program execution load a program and run it Operating systems

program synthesis automatically find a program in an underlying programming language that satisfies some user intent Interpretability

progress things should be in critical section Operating systems

projection neurons long axons Brain basics

projective transformation homography Computer vision

propositionalization can convert first-order logic to propositional logic and do propositional inference Search and logic

prosopagnosia can't recognize faces - FFA, PPA (up to 1%) Vision

protection controlling access to resources Operating systems

proxy approximates model (ex. rule list, linear model) Interpretability

pseudounipolar born polar but become uni-polar Sensory input

psuedo-inverse Moore-Penrose inverse \(A^\dagger = (A^T A)^{-1} A^T\) Linear algebra

pupil hole (aperture) whose size controlled by iris Computer vision

pure demand paging never bring a page into memory until it is required Operating systems

pure interaction effects variance in the outcome which cannot be represented by any subset of features Interpretability

purkinje cells huge number of dendrite branches - dead planar allows good imaging Motor system

putamen output only to globus pallidus Motor system

pyriform flashes of smells Motor system

Q

Q-learning agent learns \(Q(s, a)\) - action-utility function = Q-function maps actions \(\to\) utility RL and decisions

q-q plot probability plot - easily check normality Data science

QCQP inequality constraints also convex Optimization

quantile regression use quantile loss to penalize models differently + get confidence intervals Uncertainty

query a designed representation of user's need Information retrieval and databases

R

radiance power in given direction / unit area / unit solid angle Computer vision

random forest for each split of each tree, choose from only m of the p possible features Classification

Rashomon lots of different good models, which explains best? Data science

rashomon curve empirical risk vs rashomon ratio Complexity

rashomon effect many different explanations exist for same phenomenon Complexity

rashomon elbow maximize rashomon ratio while minimizing risk Complexity

rashomon ratio ratio of the volume of the set of accurate models to the volume of the hypothesis space Interpretability · Complexity

Rashomon set class of well-performing models Interpretability · Complexity

rationalist morality is self-constructed by children on the basis of their experience with harm Psychology

ray parallelizing tasks, building distributed applications, one or more machines Languages and parallel computing

reattendance patient's return visit to ED within 72 hours ML in medicine

recall sensitivity = true-positive rate = TP / P = TP / (TP + FN) Evaluation

receptive field the things that make a neuron fire Computational neuroscience

receptor potential (small) due to the activation of sensory neurons by external stimuli Brain basics

reconstituted graph same nodes, includes all edges that were added Graphical models

recursive best-first search like standard best-first search but with linear space Search and logic

recursive estimation ... Graphical models

reduced parameterizations impose constraints on probability distributions (e.g. Gaussian) Graphical models

reduced-error pruning prune only if doesn't decrease error on validation set Classification

redundancy reduction pixels are correlated, want to compress info Vision

reference bit set whenever a page is used Operating systems

reflex agent learns \(Q(s)\) - policy that maps directly from states to actions RL and decisions

regression want \(p(y \vert x)\) Stat basics

regularized discriminant analysis shrink the separate covariance matrices towards a common matrix Classification

reinforcement learning use observed rewards to learn optimal policy for the environment RL and decisions

rejection learning allow models to reject (not make a prediction) when they are not confidently accurate (chow 1957, cortes et al. 2016) Uncertainty

rejection sampling sample in order and stop once evidence is violated Graphical models

relative entropy KL divergence - measures distance between 2 distributions Information theory

relaxed problem removes constraints and adds edges to the graph Search and logic

relevance relatedness between documents and need, this is vague Information retrieval and databases

remote procedure calls (RPCs) remote message-based communication Operating systems

representer point selection ... Interpretability

representer values decompose pre-activation single prediction of a DNN into linear combinations of training points - weights are the representer values Interpretability

request edge directed edge from process P to resource R means P has requested instance of resource type R Operating systems

residual + normalize after self-attention layer, often have residual connection to previous input, which gets added then normalized LLMs

resolution rule resolves different rules with each other - leads to complete inference procedure Search and logic

resource constraint atmost constraint - sum of variable must not exceed some limit Search and logic

restricted labeling given labels for some groundtruth factors (e.g. label "glasses", "gender" for all images) Disentanglement

restriction bias language bias - models that can't express entire space (e.g. linear) Stat basics

reticular feedforward = anticipate things before they happen Motor system

reticular formation feedforward, prepares digestion before we eat Motor system

retina ! Computer vision

Retinoids axial, affect skin Development

retinotopy radiation stain on retina maintained radiation image Computer vision

rhombomeres segments in brainstem made by hox gene patterns Development

risk \(E_{(x,y)\sim D}[\mathcal L(f(X), y) ]\) Evaluation · Philosophy

risk averse when utility of money is sublinear RL and decisions

risk premium value agent will accept in lieu of lottery = certainty equivalent= insurance premium RL and decisions

risk ratio relative risk = \(P(Y=1\vert T=1) / P(Y=1\vert T=0)\) Causal inference

risk-seeking supralinear RL and decisions

robust control theory give best outcome in the worst case over H RL and decisions

Roderigues formula converts: \(R = e^{\phi \hat{s}} = I + \sin [\phi] \: \hat{s} + (1 - \cos \phi) \hat{s}^2\) Computer vision

role second vector that embeds the structural role it fills Computational neuroscience

Rolle's thm f(a)=f(b) -> f'( c)=0 Real analysis

rotation orthogonal transformations with det = +1 Computer vision

rule post-pruning cost-complexity pruning Classification

rule-based architecture condition-action rules & database of facts Cognitive science

S

S1 cortex primary somatic sensory cortex - this is the knowledge of where was touched Sensory input

sample complexity how many training examples needed to converge Learning theory · Learning theory

sampling the process of taking individual values of a continuous-time signal Signal processing

sampling period time interval between samples Signal processing

sampling rate the number of samples taken per second (Hz) Signal processing

SAP Kumar et al. (2017) - how much does top latent factor match gt more than 2nd latent factor Disentanglement

sapir-whorf hypothesis understanding of the world is influenced by the language we speak Search and logic

satisfiability threshold conjecture for random clauses, probability of satisfiability goes to 0 or 1 based on ratio of clauses to symbols Search and logic

satisfiable true under some model Search and logic

SATPLAN how to make plans for future actions that solve the goal by propositional inference Search and logic

Sauer's lemma let \(d \geq 0, m \geq 1\), \(H\) hypothesis space, VC-dim(H) = d. Then, \(\Pi_H(m) \leq \phi (d,m)\) Learning theory

scheduler selects processes from queues Operating systems

scheduler activation kernel provides application with LWPs Operating systems

schema substring in which some of the positions can be left unspecified (ex. `246`) Search and logic

science organization of knowledge Philosophy

scope neglect the number of birds saved—the scope of the altruistic action—had little effect on willingness to pay (post) Cognitive science

score-based algorithms replace conditional indep. tests with godness of fit tests (e.g. BIC) Causal inference

screeplot eigenvalues in decreasing order, look for num dims with kink Unsupervised

SDP semi-definite program: \(\min \: c^T x\\s.t. \: x_1F_1+...+x_nF_n+G \preceq 0\\Ax=b\) ~ where \(F_1, ..., F_n \in S^k\) Optimization

search cost just time/memory Search and logic

second moment matrix need this to be high enough rank Computer vision

second-order / weak stationarity mean is constant for all t and, for any t and k, the covariance between \(X_t\) and \(X_{t+k}\) only depends on the lag difference k Time series

secondary 2d analysis of hydrogen bonds between rna parts (double-strands, hairpins, loops) - most of stabilizing free energy comes from here (unlike proteins, where tertiary is most important) Omics

sector slipping just renumbers to not index bad blocks Operating systems

security defends a system from attacks Operating systems

self-attention layer that lets word learn its relation to other layers LLMs

semantics study of the meaning of words / word combinations NLP · Cognitive science

semidecidable algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every nonentailed sentence Search and logic

sensitization associate two stimuli together: ! Brain basics

Sensory Cortex The sensory cortex, located in the front portion of the parietal lobe, receives information relayed from the spinal cord regarding the position of various body parts and how they are ... Brain basics

separating hyperplane thm C, D convex \(C \cap D =\emptyset \implies \exists a \neq 0, b \: s.t. \\ a^Tx \leq b \forall x \in C, \\a^Tx \geq b \forall x \in D\) Optimization

sequence could have each element be an address pointing to the next element Computational neuroscience

sequence modeling learn \(p(x)\), usually factorized as \(p(x_i\vert x_1,...,x_{i-1})\) LLMs

sequence-to-sequence learn \(p(z\vert x)\), e.g. transalation, speech-to-text, question answering LLMs

serotonin 5HT - happiness Brain basics

set can be represented with a sum (since the sum is similar to all the vectors) Computational neuroscience

shape relative distances of points on the object Computer vision

shared libraries all use same library Operating systems

shared memory shared region of memory is established Operating systems

shingles virus where you get stripes of sores - single DRG Sensory input

short-term / CPU scheduler selects from processes ready to execute and allocates CPU to one of them Operating systems

silhouette plots good clusters members are close to each other and far from other clustersf Data science

sim(u, v) A similarity function Unsupervised

simple cells sensitive to oriented lines Computer vision

simple constant-treatment model \(Y_{i t}=Y_{i t}(0)+T_{i t} \tau,\) for all \(i=1, \ldots, n, t=1, \ldots\) Causal inference

simplest sample network in topological order Graphical models

simplex conv{\(v_{0:k}\)} Optimization

simplicity too complex Philosophy

simpson's paradox yule-simpson paradox - trend appears in several different groups but disappears/reverses when groups are combined Causal inference

simulated annealing pick random move Search and logic

singular extension allow searching for certain specific moves that are always good at deeper depths RL and decisions

sion's minimax thm \(x \to f(x, y)\) ~ conditions Optimization

slit/robo receptor is slit Development

smooth pursuit can track things moving very fast Motor system

smoothing compute \(P(X_{k}\vert e_{1:t})\) for \(0 < k < t\) Graphical models

social determinism theory that social interactions and constructs alone determine individual behavior Fairness and STS

social intuitionism other people can alter intuitions Psychology

socially imposed in economic unequal societies (like today) Psychology

sockets endpoint for communication Operating systems

SOCP \(\min f^Tx \\ s.t. \: \vert \vert A_ix+b_i\vert \vert _2 \leq c_i^T + d_i \\ Fx=g\) Optimization

softmax takes vector z and returns vector of the same length Deep learning

sparse distributed representations (SDR) vectors with thousands of bits, mostly 0s Computational neuroscience

spasticity unorganized leg motions Motor system

spatial verification don't just use visual words, use structure of where the words are Computer vision

specificity true negative rate = TN / N = TN / (TN + FP) Evaluation

specular surfaces outgoing radiance direction obeys angle of incidence Computer vision

spike-triggered average stimulus averagin frames that correspond to spikes - yields receptive field Vision

spina bifida neural tube fails to seal Development

spines start of postsynaptic dendrite Brain basics

spinocerebellum clarke’s nucleus, knows stretch of every muscle, many proprioceptors go straight into here Motor system

spool buffer for device (ex. printer) that can't hold interleaved data Operating systems

stack algorithms set of pages in memory for n frames is always a subset of the set of pages that would be in memory with n + 1 frames Operating systems

staging describes where cancer is located and where it has spread Disease

starburst amacrine cells generate directional signals in the retina Vision

stat. significant p = 0.05 Testing

state-estimation keep track of belief state Search and logic

stereo correspondence stereo matching: given point in one image, find corresponding point in 2nd image Computer vision

stereo image rectification transform images so that image planes are parallel Computer vision

stereopsis perception of depth Computer vision

stochastic beam search pick best k with prob proportional to how good they are Search and logic

stochastic complexity noise + learnable info Complexity

strabismus lazy eye - poor coordination with one of the muscles Development

strange loop paradox - self-referential Cognitive science

stratification cross-tabulation - only compare samples when confounding variables have same value Causal inference

stratification at design stage stratify and run RCT for each stratum Causal inference

strict stationarity the distribution is the same across time Time series

strong alternative exactly one is true Optimization

strong duality \(d^ = p^\) ~ requires more than convexity Optimization

strongly k-consistent also (k-1) consistent, (k-2) consistent, ... 1-consistent Search and logic

stub tells how to load / locate library routine Operating systems

student's t \(U_{d+1} / \sqrt{d^{-1} \sum_1^d U_i^2}\) w/ d degress of freedom Testing

studentized statistic \(t=\frac{\hat \tau}{\sqrt{\frac{\hat S^2(T=1)}{n_1}+\frac{\hat S^2 (T=0)}{n_0}}}\) Causal inference

subgroup analysis identify subgroups with treatment effects far from the average Causal inference

substitutability can do algebra with preference eqns RL and decisions

subword-based break apart meaningful subparts of words, most popular (2022) NLP

succesor-state axiom \(F^{t+1} \iff \cgreen{ActionCausesF^t} \lor \cblue{(F^t \land -ActionCausesNotF^t )}\) Search and logic

sufficient set admissible set - set of factors to calculate causal effect of \(T\) on \(Y\) requires 2 conditions Causal inference

sum-product belief propagation - inference algorithm Graphical models

superior colliculus integrate visual, auditory, motor to get X,Y coordinate Development

supporting hyperplane thm {\(x\vert a^tx = a^t x_0\)} where \(x_0\) on boundary of convex C Optimization

supremum supA = least upper bound (similarly, infimum) Real analysis

switching Kalman filter multiple Kalman filters run in parallel and weighted sum of predictions is used Graphical models

symbol meaningful pattern that can be manipulated Cognitive science

symmetric clustering both run something Operating systems

symmetric multiprocessing (SMP) each processor performs all tasks Operating systems

sympathetic fight-or-flight (adrenaline) Motor system

synaptic potential (small) caused by activation of synapse Brain basics

syntax study of the grammatical structure of phrases/sentences NLP · Cognitive science

system calls provide an interface to os services Operating systems

system generation configuring os on a computer Operating systems

system programs system utilities Operating systems

T

t-test test if mean is nonzero Testing

task parallelism distribution tasks across cores Operating systems

technological determinism theory that assumes that a society's technology determines the development of its social structure and cultural values Fairness and STS

technology smaller or larger pieces or systems of hardware of a specific kind. Fairness and STS

temporal-difference learning adjust utility estimates towards local equilibrium for correct utilities RL and decisions

term logical expresision tha refers to an object Search and logic

term-by-term continuity thm let \(f_n\) be continuous functions defined on a set A \(\subseteq\) R and assume \(\sum f_n\) converges uniformly on A to a function f. Then, f is continuous on A. Real analysis

term-by-term differentiability thm let \(f_n\) be differentiable functions defined on an interval A, and assume \(\sum f'_n(x)\) converges uniformly to a limit g(x) on A. If there exists a point \(x_0 \in [a,b]\) where ... Real analysis

tethered hacking intervals take any model with small enough loss on the data Interpretability

texton histogram matching classify images Computer vision

texture non-countable stuff Computer vision

thermoceptors neurons that sense temperature Sensory input

Thm under latent ignorability, \(\mathrm{RR}_{x}^{\mathrm{obs}} \leq \frac{\mathrm{R} \mathrm{R}_{T U \mid x} \mathrm{RR}_{U Y \mid x}}{\mathrm{RR}_{T U \mid x}+\mathrm{RR}_{U Y \mid x}-1}\) Causal inference

thrashing process spends more time paging than executing Operating systems

thread basic unit of CPU utilization Operating systems

thread cancellation terminating target thread before it has completed Operating systems

thread library provides programmer with an API for creating/managing threads Operating systems

thread pool number of threads at startup that sit in a pool and wait for work Operating systems

thread-local storage when threads need separate copies of data Operating systems

threads multiple paths of execution within a single process Languages and parallel computing

Tikhonov \(\min \: \cblue{\vert \vert Ax-b\vert \vert _2^2} + \corange{\gamma \vert \vert x\vert \vert _2^2}\) Optimization

tiling train networks on the error of previous networks Deep learning

time sharing / multitasking multiple jobs switch so fast that both can be interacted with Operating systems

time-rescaling theorem tests how well we have captured influences on spiking (Brown et al 2001) Computational neuroscience

timnit gebru fairness team at google Fairness and STS

total cost search cost + path cost Search and logic

Tourette's too much dopamine Motor system

TraceInIdeal train with batch size 1, measure change in loss of \(x_{test}\) after taking a training step with \(x_{train}\) Interpretability

tract bundles of CNS axons Brain basics · Sensory input

transfer learning given unlabelled test data, match training/testing representations Causal inference

transference physical connection, e.g. continuous mechanism / conserved physical quantity Psychology

transform x if residuals generally normal and have constant variance Data science

transform y if relationship generally linear, but non-constant error variance Data science

transient failure model allows probability of sensor giving wrong value Graphical models

transient OS code not often used Operating systems

trap / exception software-generated interrupt Operating systems

treatment intervention, exposure, action Causal inference

tree undirected graph in which there is exactly one path between any pair of nodes Graphical models · Search and logic

tree decomposition view each subproblem as a mega-variable Search and logic

triangle \(\vert \vert x + y \vert \vert \leq \vert \vert x \vert \vert + \vert \vert y \vert \vert\) Stat basics

triangle inequality \(\vert a+b\vert \leq \vert a\vert + \vert b\vert\) (often use \|a-b\| = \|(a-c)+(c-b)\|) Real analysis

triangulation predict 3d points \((X_i, Y_i, Z_i)\) given pixels in multiple cameras \((x_i, y_i)\) and camera parameters \(R, t\) Computer vision

trophic support neuron survives on contact Development

tumor neoplasm - a mass formation from an uncontrolled growth of cells Disease

tuning curve neuron's response (ex. firing rate) as a function of stimulus Computational neuroscience

turing test is human mind deterministic { turing1950computing } Cognitive science

U

ultrametric distance tighter than triangle inequality Unsupervised

umap Uniform Manifold Approximation and Projection for Dimension Reduction Unsupervised

unary constraint restricts value of single variable Search and logic

unawareness don't show sensitive attributes Fairness and STS

unbounded buffer producer can keep producing indefinitely Operating systems

unconfounded children criterion if it is possible to block all backdoor paths from \(T\) to all of its children that are ancestors of \(Y\) with a single conditioning set \(S\), then \(P(Y=y\vert do(T=t))\) is identifiable (tian ... Causal inference

Unidirectional / masked self-attention mask scores from before a given word LLMs

unification finding substitutions that make different logical expressions look identical Search and logic · Philosophy

uniform-cost search always expand node with lowest path cost g(n) Search and logic

uninformed given no info about problem besides definition Search and logic

universal code one code such that no matter which \(x^n\) arrives, length is not much longer than the shortest length among all considered codes Information theory

universal instantiation we can infer any sentence obtained by substituting a ground term for the variable Search and logic

universal model prob. distr corresponding to universal codes Information theory

upcall kernel informs application about certain events Operating systems

urethelium inner layer that is able to stretch (has many layers) - this is where cancer originates Disease

user interface command-line interface and graphical user interface Operating systems

utility function lottery that satisfiers certain properties (e.g. transitivity) RL and decisions

utility nodes represent agent's utility function RL and decisions

utility-based agent learns \(U(S)\) - utility function on states RL and decisions

V

vagus nucleus digestive system, sexual function Motor system

valencies sometimes certain things in a class are allowed buothers are not NLP

validity tautalogy - true under all models Search and logic

value alignment putting in the right purpose AI futures

value function ordinal utility function - sometimes ranking, numbers not needed RL and decisions

value iteration calculates utility of each state and uses utilities to find optimal policy RL and decisions

value of perfect information VPI assume we can obtain exact evidence for a variable (ex. variable \(T=t\)) RL and decisions

Value-based estimate value function or \(Q\)-function of the optimal policy (no explicit policy) RL and decisions

vanishing gradients problem neurons in earlier layers learn more slowly than in later layers Deep learning

variable elimination dynamic programming (see elimination) Graphical models

variable-partition scheme OS keeps table of free mem Operating systems

variational inference formulate inference as optimization Graphical models

vergence crossing your eyes Motor system

verification does dnn satisfy a specification (e.g \(l_p\) robustness) Deep learning theory

version space set of all hypotheses consistent with the training examples Learning theory

vestibular cerebellum vestibular->cerebellum->vestibular Motor system

virtual memory fork not copy-on-write Operating systems

virtualization allows OS to run within another OS Operating systems

visceral pain don't know where the pain is Sensory input

Viterbi algorithm ... Graphical models

volatile lost when power off Operating systems

voltage-sensitive dyes would be great Vision

von Neumann architecture uses instruction register Operating systems

VOR vestibular ocular reflex - keeps eyes still Motor system

VPL everything accumulates here in the thalamus then goes to Sensory input

W

Wald estimator \(\frac{Cov(Y, I)}{Cov(T, I)}\) Causal inference

WAN wide-area network Operating systems

weak alternative at most one of 2 is true Optimization

weak duality \(d^ \leq p^\) Optimization

weighted linear model takes into account previous states weighted by f Computational neuroscience

weighted LS minimize \(\sum [w_i (y_i - x_i^T \theta)]^2\) Linear models

well-posedness want x to be unique for a given u Deep learning theory

white matter has more axons Brain basics

wide sense convex func \(f(x) = \inf \{ t \in \mathbb{R} \vert (x, t) \in F\}\) where \(F \subseteq \mathbb{R}^{n+1}\) Optimization

wide sense function can take on values \(\pm \infty\) Optimization

Wierstrass M-Test For each n in N, let \(f_n\) be a function defined on a set A \(\subseteq\) R, and let \(M_n > 0\) be a real number satisfying \(\vert f_n(x)\vert ≤ M_n\) for all x in A. If \(\sum M_n\) converges ... Real analysis

wilcoxon rank sum \(W = \sum_i T_i R_i\), where \(R_i = \#\{j : Y_j \leq Y_i \}\) is the rank of \(Y_i\) in the observed samples Causal inference

Wilson initial book sociobiology was vehemently opposed Psychology

windowed fourier transform chop signal into sections and analyze each section separately Signal processing

winzorizing replace suspect values with closest values that aren't Data science

Wnts skin, gut, hair Development

working-set model still based on locality Operating systems

worst-fit allocate largest hole (largest leftover hole) Operating systems

Z

zero-fill-on-demand zeroed out before being allocated Operating systems

Zeta \(\zeta(x) = \sum_1^\infty \frac{1}{x^2} \) Math basics

zombie process terminated but parent hasn't called wait() yet Operating systems

\(\,\)