phd candidate at berkeley working (with the amazing prof. bin yu) on interpretable machine learning across domains 🧠💊️🦠
phd candidate at berkeley working (with the amazing prof. bin yu) on interpretable machine learning across domains 🧠💊️🦠
🔎 My research focuses on how we can build trustworthy machine-learning systems by making them interpretable. In my work, interpretability is grounded seriously via close collaboration with domain experts, e.g. medical doctors or cell biologists. These collaborations have given rise to useful methodology, roughly split into two areas: (1) building more effective transparent models and (2) improving the trustworthiness of black-box models. Going forward, I hope to help bridge the gap between transparent models and black-box models to improve real-world healthcare.
🌳 Whenever possible, building transparent models is the most effective route towards ensuring interpretability. Transparent models are interpretable by design, including models such as (concise) decision trees, rule lists, and linear models. My work in this area was largely motivated by the problem of clinical decision-rule development. Clinical decision rules (especially those used in emergency medicine), need to be extremely transparent so they can be readily audited and used by physicians making split-second decisions. To this end, we have developed methodology for enhancing decision trees. For example, replacing the standard CART algorithm with a novel greedy algorithm for tree-sums can substantially improve predictive performance without sacrificing predictive performance. Additionally, hierarchical regularization can improve the predictions of an already fitted model without altering its interpretability. Despite their effectiveness, transparent models such as these often get overlooked in favor of black-box models; to address this issue, we've spent a lot of time curating imodels, an open-source package for fitting state-of-the-art transparent models.
🌀 My second line of my work focuses on interpreting and improving black-box models, such as neural networks, for the cases when a transparent model simply can't predict well enough. Here, I work closely on real-world problems such as analyzing imaging data from cell biology and cosmology. Interpretability in these contexts demands more nuanced information than standard notions of "feature importance" common in the literature. As a result, we have developed methods to characterize and summarize the interactions in a neural network, particularly in transformed domains (such as the Fourier domain), where domain interpretations can be more natural. I'm particularly interested in how we can ensure that these interpretations are useful, either by using them to embed prior knowledge into a model or identify when it can be trusted.
🤝 There is a lot more work to do on bridging the gap between transparent models and black-box models in the real world. One promising avenue is distillation, whereby we can use a black-box model to build a better transparent model. For example, in one work we were able to distill state-of-the-art neural networks in cell-biology and cosmology into transparent wavelet models with <40 parameters. Despite this huge size reduction, these models actually improve prediction performance. By incorporating close domain knowledge into models and the way we approach problems, I believe interpretability can help unlock many benefits of machine-learning for improving healthcare and science.
| year | title | authors | tags | paper | code | misc |
|---|---|---|---|---|---|---|
| '22 | Interpretable deep learning for accurate molecular partner prediction in clathrin-mediated endocytosis | singh*, li*, et al. | 🔎🌀🦠 | in prep | ||
| '22 | Group Probability-Weighted Tree Sums for Interpretable Modeling of Heterogeneous Data | nasseri, singh, duncan, kornblith, & yu | 🔎🌳💊 | arXiv | ||
| '22 | developing reliable clinical decision rules: a case study in identifying blunt abdominal trauma in children | kornblith*, singh* et al. | 🔎🌳💊 | PLOS digital health | ||
| '22 | Fast Interpretable Greedy-Tree Sums (FIGS) | tan*, singh*, nasseri, agarwal, & yu | 🔎🌳 | arxiv | ||
| '22 | Hierarchical shrinkage: improving accuracy and interpretability of tree-based methods | agarwal*, tan*, ronen, singh, & yu | 🔎🌳 | icml (spotlight) | ||
| '22 | VeridicalFlow: a Python package for building trustworthy data science pipelines with PCS | duncan*, kapoor*, agarwal*, singh*, & yu | 💻🔍 | joss | ||
| '21 | imodels: a python package for fitting interpretable models | singh*, nasseri*, et al. | 💻🔍🌳 | joss | ||
| '21 | Adaptive wavelet distillation from neural networks through interpretations | ha, singh, et al. | 🔍🌀🌳 | neurips | ||
| '21 | Matched sample selection with GANs for mitigating attribute confounding | singh, balakrishnan, & perona | 🌀 | cvpr workshop | ||
| '21 | Revisiting complexity and the bias-variance tradeoff | dwivedi*, singh*, yu & wainwright | 🌀 | topml workshop | ||
| '20 | Curating a COVID-19 data repository and forecasting county-level death counts in the United States | altieri et al. | 🔎🦠 | hdsr | ||
| '20 | transformation importance with applications to cosmology | singh*, ha*, lanusse, boehm, liu & yu | 🔎🌀🌌 | iclr workshop (spotlight) | ||
| '20 | interpretations are useful: penalizing explanations to align neural networks with prior knowledge | rieger, singh, murdoch & yu | 🔎🌀 | icml | ||
| '19 | hierarchical interpretations for neural network predictions | Singh*, Murdoch*, & Yu | 🔍🌀 | ICLR | ||
| '19 | interpretable machine learning: definitions, methods, and applications | Murdoch*, Singh*, et al. | 🔍🌳🌀 | pnas | ||
| '19 | disentangled attribution curves for interpreting random forests and boosted trees | devlin, singh, murdoch & yu | 🔍🌳 | arxiv | ||
| '18 | large scale image segmentation with structured loss based deep learning for connectome reconstruction | Funke*, Tschopp*, et al. | 🧠🌀 | TPAMI | ||
| '18 | linearization of excitatory synaptic integration at no extra cost | Morel, Singh, & Levy | 🧠 | J Comp Neuro | ||
| '17 | a consensus layer V pyramidal neuron can sustain interpulse-interval coding | Singh & Levy | 🧠 | Plos One | ||
| '17 | a constrained, weighted-l1 minimization approach for joint discovery of heterogeneous neural connectivity graphs | Singh, Wang, & Qi | 🧠 | neurips Workshop | , |
Some posts in ml research and organizing different papers.
Resources including teaching slides, research overview slides, and coding projects. There's also some dumb stuff here.
A rough set of notes which may serve as useful reference for people in machine learning / neuroscience. Here's a jupyter-book version.
phd research in interpretable ml
developed deep learning models for pathology
developed methods for causal benchmarking of fairness in computer vision models
developed methods for interpreting medical machine-learning models
Worked on unsupervised pretraining of CNNs for semantic segmentation
Contributed to development of novel weighted-L1, multi-task Gaussian graphical models
Contributed to enhanced ML implementations for neural image segmentation
Simulated stochastic neurons to determine mutual information, variability, energy efficiency, and threshold
Simulated extracellular neural recordings via Neurocube Matlab scripts
Developed web app for simultaneous task coordination + Android app to increase the data storage capacity of QR codes.
I've been lucky to be advised by / collaborate with many amazing people
It has been my pleasure to help advise some incredible undergrads at berkeley -- check them out!
