Today we continue the 2020 AI Rewind series, joined by friend of the show Sameer Singh, an Assistant Professor in the Department of Computer Science at UC Irvine.
Subscribe: iTunes / Google Play / Spotify / RSS
We last spoke with Sameer at our Natural Language Processing office hours back at TWIMLfest, and was the perfect person to help us break down 2020 in NLP. Sameer tackles the review in 4 main categories, Massive Language Modeling, Fundamental Problems with Language Models, Practical Vulnerabilities with Language Models, and Evaluation.
We also explore the impact of GPT-3 and Transformer models, the intersection of vision and language models, and the injection of causal thinking and modeling into language models, and much more.
To follow along with the 2020 AI Rewind Series, head over to the series page!
Thanks to our Sponsor!
Thanks to our friends at Pachyderm for sponsoring the show!
At the end of the day, real-world machine learning is all about the data. You already know this, but manually cleaning and transforming data can be exhausting, inconsistent, and error-prone, and is not the path towards getting your models into production, especially when your data, models, and code are constantly changing.
This is where Pachyderm can help. Pachyderm is an easy-to-use data science platform that lets you productionalize your machine learning tasks into fully-automated, end-to-end workflows, regardless of language or framework. Pachyderm provides Git-like data versioning and lineage that lets you automatically track every data change and final output result.
Right now, TWIML listeners can enjoy 20% off of Pachyderm Hub and build production-grade data science workflows in minutes, without ever having to configure a single piece of infrastructure.
Imagine being able to automate your entire data science workflow and reproduce any result from any point in time — in seconds, and with complete confidence.
Head over to pachyderm.com/TWIML to learn more and take advantage of this limited time offer.
Connect with Sameer!
- Language Models are Few-Shot Learners (GPT-3)
- Scaling Laws for Neural Language Models
- Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
- Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez
- Longformer: The Long-Document Transformer
- Big Bird: Transformers for Longer Sequences
- Reformer: The Efficient Transformer
- Linformer: Self-Attention with Linear Complexity
- Rethinking Attention with Performers
- Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
- Is Linguistics Missing from NLP Research? w/ Emily M. Bender
- Experience Grounds Language
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- Weight Poisoning Attacks on Pre-trained Models
- Customizing Triggers with Concealed Data Poisoning
- “You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
- How Can We Accelerate Progress Towards Human-like Linguistic Generalization?
- Beyond Accuracy: Behavioral Testing of NLP models with CheckList
- Beyond Accuracy: Behavioral Testing of NLP Models with Sameer Singh
- BLiMP: The Benchmark of Linguistic Minimal Pairs for English
- Evaluating Models’ Local Decision Boundaries via Contrast Sets
- WinoGrande: An Adversarial Winograd Schema Challenge at Scale
- Check out our TWIML Presents: series page!
- Register for the TWIML Newsletter
- Check out the official TWIMLcon:AI Platform video packages here!
- Download our latest eBook, The Definitive Guide to AI Platforms!
“More On That Later” by Lee Rosevere licensed under CC By 4.0