transfer learning

view markdown

---

See also notes on causal inference for some close connections.

• Domain-Adversarial Training of Neural Networks (ganin et al. 16) - want repr. to be invariant to domain label
  • 
  • exact same algorithm is used to learn fair representations, except domain label is replaced with sensitive attribute
• domain adaptation, given multiple training groups
  • group distributionally robust optimization
  • domain generalization
• domain adaptation using source/target, given all at once
  • importance weighting
  • learning invariant represetntations
• test-time adaptation
  • batch normalization
  • label shift estimation
  • rotation prediction (sun et al. 2020)
  • entropy minimization
• adaptive risk minimization - combines groups at training time + batches at test-time
  • meta-train the model using simulated distribution shifts, which is enabled by the training groups, such that it exhibits strong post-adaptation performance on each shift
• Domain adaptation under structural causal models (chen & buhlmann, 2020)
  • make clearer assumptions for domain adaptation to work
  • introduce CIRM, which works better when both covariates and labels are perturbed in target data