1.1. transfer learning#

See also notes on 📌 causal inference for some close connections.

For neural-net specific transferring see 📌 adaption/transfer.

1.1.1. overviews#

transfer_taxonomy (from this paper)

1.1.2. domain adaptation algorithms#

Domain test bed available here, for generalizating to new domains (i.e. performing well on domains that differ from previous seen data)

  • Empirical Risk Minimization (ERM, Vapnik, 1998) - standard training

  • Invariant Risk Minimization (IRM, Arjovsky et al., 2019) - learns a feature representation such that the optimal linear classifier on top of that representation matches across domains.

  • distributional robust optimization

    • instead of minimizing training err, minimize maximum training err over different perturbations

    • Group Distributionally Robust Optimization (GroupDRO, Sagawa et al., 2020) - ERM + increase importance of domains with larger errors (see also papers from Sugiyama group e.g. 1, 2)

      • minimize error for worst group

    • Variance Risk Extrapolation (VREx, Krueger et al., 2020) - encourages robustness over affine combinations of training risks, by encouraging strict equality between training risks

  • Interdomain Mixup (Mixup, Yan et al., 2020) - ERM on linear interpolations of examples from random pairs of domains + their labels

  • Marginal Transfer Learning (MTL, Blanchard et al., 2011-2020) - augment original feature space with feature vector marginal distributions and then treat as a supervised learning problem

  • Meta Learning Domain Generalization (MLDG, Li et al., 2017) - use MAML to meta-learn how to generalize across domains

  • learning more diverse predictors

    • Representation Self-Challenging (RSC, Huang et al., 2020) - adds dropout-like regularization to important features, forcing model to depend on many features

    • Spectral Decoupling (SD, Pezeshki et al., 2020) - regularization which forces model to learn more predictive features, even when only a few suffice

  • embedding prior knowledge

    • Style Agnostic Networks (SagNet, Nam et al., 2020) - penalize style features (assumed to be spurious)

    • Penalizing explanations (Rieger et al. 2020) - penalize spurious features using prior knowledge

  • Domain adaptation under structural causal models (chen & buhlmann, 2020)

    • make clearer assumptions for domain adaptation to work

    • introduce CIRM, which works better when both covariates and labels are perturbed in target data

  • kernel approach (blanchard, lee & scott, 2011) - find an appropriate RKHS and optimize a regularized empirical risk over the space

  • In-N-Out (xie…lang, 2020) - if we have many features, rather than using them all as features, can use some as features and some as targets when we shift, to learn the domain shift

1.1.2.1. domain invariance#

key idea: want repr. to be invariant to domain label

1.1.2.2. feature learning#

1.1.2.3. dynamic selection#

Dynamic Selection (DS) refers to techniques in which, for a new test point, pre-trained classifiers are selected/combined from a pool at test time review paper (cruz et al. 2018), python package

  1. define region of competence

    1. clustering

    2. kNN - more refined than clustering

    3. decision space - e.g. a model’s classification boundary, internal splits in a model

    4. potential function - weight all the points (e.g. by their distance to the query point)

  2. criteria for selection

    1. individual scores: acc, prob. behavior, rank, meta-learning, complexity

    2. group: data handling, ambiguity, diversity

  3. combination

    1. non-trainable: mean, majority vote, product, median, etc.

    2. trainable: learn the combination of models

      1. related: in mixture of experts models + combination are trained jointly

    3. dynamic weighting: combine using local competence of base classifiers

    4. Oracle baseline - selects classifier predicts correct label, if such a classifier exists

1.1.2.4. test-time adaptation#

  • test-time adaptation

  • test-time learning with rotation prediction (sun et al. 2020) - at test-time, update parameters for self-supervised rotation prediction task then use for classification

    • masked autoencoders (gandelsman, sun, …, efros, 2022) - use reconstructed with masked autoencoder and improve performance on robustness tasks

    • test-time learning for Reading Comprehension (banerjee et al. 2021) - uses self-supervision to train models on synthetically generated question-answer pairs, and then infers answers to unseen human-authored questions for this context

    • TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive? (liu et al. 2021) - explore different test-time adaptation methods and combine Test-time feature alignment with Test-time contrastive learning

  • combining train-time and test-time adaptation

    • Adaptive Risk Minimization (ARM, Zhang et al., 2020) - combines groups at training time + batches at test-time

      • meta-train the model using simulated distribution shifts, which is enabled by the training groups, such that it exhibits strong post-adaptation performance on each shift

1.1.3. adv attacks#

1.1.3.1. nlp#

Attack Recipe Name Goal Function ConstraintsEnforced Transformation Search Method Main Idea
a2t Untargeted {Classification, Entailment} Percentage of words perturbed, Word embedding distance, DistilBERT sentence encoding cosine similarity, part-of-speech consistency Counter-fitted word embedding swap (or) BERT Masked Token Prediction Greedy-WIR (gradient) from "Towards Improving Adversarial Training of NLP Models" (Yoo et al., 2021)
alzantot Untargeted {Classification, Entailment} Percentage of words perturbed, Language Model perplexity, Word embedding distance Counter-fitted word embedding swap Genetic Algorithm from "Generating Natural Language Adversarial Examples" (Alzantot et al., 2018)
bae Untargeted Classification USE sentence encoding cosine similarity BERT Masked Token Prediction Greedy-WIR BERT masked language model transformation attack from ("BAE: BERT-based Adversarial Examples for Text Classification" (Garg & Ramakrishnan, 2019)).
bert-attack Untargeted Classification USE sentence encoding cosine similarity, Maximum number of words perturbed BERT Masked Token Prediction (with subword expansion) Greedy-WIR ("BERT-ATTACK: Adversarial Attack Against BERT Using BERT" (Li et al., 2020))
checklist {Untargeted, Targeted} Classification checklist distance contract, extend, and substitutes name entities Greedy-WIR Invariance testing implemented in CheckList . ("Beyond Accuracy: Behavioral Testing of NLP models with CheckList" (Ribeiro et al., 2020))
clare Untargeted {Classification, Entailment} USE sentence encoding cosine similarity RoBERTa Masked Prediction for token swap, insert and merge Greedy ["Contextualized Perturbation for Textual Adversarial Attack" (Li et al., 2020)](https://arxiv.org/abs/2009.07502))
deepwordbug {Untargeted, Targeted} Classification Levenshtein edit distance {Character Insertion, Character Deletion, Neighboring Character Swap, Character Substitution} Greedy-WIR Greedy replace-1 scoring and multi-transformation character-swap attack (["Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers" (Gao et al., 2018)](https://arxiv.org/abs/1801.04354)
fast-alzantot Untargeted {Classification, Entailment} Percentage of words perturbed, Language Model perplexity, Word embedding distance Counter-fitted word embedding swap Genetic Algorithm Modified, faster version of the Alzantot et al. genetic algorithm, from (["Certified Robustness to Adversarial Word Substitutions" (Jia et al., 2019)](https://arxiv.org/abs/1909.00986))
hotflip (word swap) Untargeted Classification Word Embedding Cosine Similarity, Part-of-speech match, Number of words perturbed Gradient-Based Word Swap Beam search (["HotFlip: White-Box Adversarial Examples for Text Classification" (Ebrahimi et al., 2017)](https://arxiv.org/abs/1712.06751))
iga Untargeted {Classification, Entailment} Percentage of words perturbed, Word embedding distance Counter-fitted word embedding swap Genetic Algorithm Improved genetic algorithm -based word substitution from (["Natural Language Adversarial Attacks and Defenses in Word Level (Wang et al., 2019)"](https://arxiv.org/abs/1909.06723)
input-reduction Input Reduction Word deletion Greedy-WIR Greedy attack with word importance ranking , Reducing the input while maintaining the prediction through word importance ranking (["Pathologies of Neural Models Make Interpretation Difficult" (Feng et al., 2018)](https://arxiv.org/pdf/1804.07781.pdf))
kuleshov Untargeted Classification Thought vector encoding cosine similarity, Language model similarity probability Counter-fitted word embedding swap Greedy word swap (["Adversarial Examples for Natural Language Classification Problems" (Kuleshov et al., 2018)](https://openreview.net/pdf?id=r1QZ3zbAZ))
pruthi Untargeted Classification Minimum word length, Maximum number of words perturbed {Neighboring Character Swap, Character Deletion, Character Insertion, Keyboard-Based Character Swap} Greedy search simulates common typos (["Combating Adversarial Misspellings with Robust Word Recognition" (Pruthi et al., 2019)](https://arxiv.org/abs/1905.11268)
pso Untargeted Classification HowNet Word Swap Particle Swarm Optimization (["Word-level Textual Adversarial Attacking as Combinatorial Optimization" (Zang et al., 2020)](https://www.aclweb.org/anthology/2020.acl-main.540/))
pwws Untargeted Classification WordNet-based synonym swap Greedy-WIR (saliency) Greedy attack with word importance ranking based on word saliency and synonym swap scores (["Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency" (Ren et al., 2019)](https://www.aclweb.org/anthology/P19-1103/))
textbugger : (black-box) Untargeted Classification USE sentence encoding cosine similarity {Character Insertion, Character Deletion, Neighboring Character Swap, Character Substitution} Greedy-WIR ([(["TextBugger: Generating Adversarial Text Against Real-world Applications" (Li et al., 2018)](https://arxiv.org/abs/1812.05271)).
textfooler Untargeted {Classification, Entailment} Word Embedding Distance, Part-of-speech match, USE sentence encoding cosine similarity Counter-fitted word embedding swap Greedy-WIR Greedy attack with word importance ranking (["Is Bert Really Robust?" (Jin et al., 2019)](https://arxiv.org/abs/1907.11932))