Beyond Accuracy: Behavioral Testing of NLP Models with Sameer Singh
EPISODE 406
|
SEPTEMBER
3,
2020
Watch
Follow
Share
About this Episode
Today we're joined by Sameer Singh, an assistant professor in the department of computer science at UC Irvine.
Sameer's work centers on large-scale and interpretable machine learning applied to information extraction and natural language processing. We caught up with Sameer right after he was awarded the best paper award at ACL 2020 for his work on Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In our conversation, we explore CheckLists, the task-agnostic methodology for testing NLP models introduced in the paper. We also discuss how well we understand the cause of pitfalls or failure modes in deep learning models, Sameer's thoughts on embodied AI, and his work on the now famous LIME paper, which he co-authored alongside Carlos Guestrin.
About the Guest
Sameer Singh
UC Irvine
Connect with Sameer
Resources
- Paper: Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
- Paper: "Why Should I Trust You?": Explaining the Predictions of Any Classifier (LIME)
- Paper: Universal Adversarial Triggers for Attacking and Analyzing NLP
- Paper: AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
- #007 - Explaining the Predictions of Machine Learning Models with Carlos Guestrin
- Project Page: Universal Adversarial Triggers for Attacking and Analyzing NLP
- Semantically Equivalent Adversarial Rules for Debugging NLP Models
- #387 - AI for High-Stakes Decision Making with Hima Lakkaraju
Related Topics
Related Episodes
613
