AI Rewind 2021: Trends in NLP with John Bohannon
EPISODE 550
|
JANUARY
6,
2022
Watch
Follow
Share
About this Episode
Today we’re joined by friend of the show John Bohannon, the director of science at Primer AI, to help us showcase all of the great achievements and accomplishments in NLP in 2021! In our conversation, John shares his two major takeaways from last year, 1) NLP as we know it has changed, and we’re back into the incremental phase of the science, and 2) NLP is “eating” the rest of machine learning. We explore the implications of these two major themes across the discipline, as well as best papers, up and coming startups, great things that did happen, and even a few bad things that didn’t. Finally, we explore what 2022 and beyond will look like for NLP, from multilingual NLP to use cases for the influx of large auto-regressive language models like GPT-3 and others, as well as ethical implications that are reverberating across domains and the changes that have been ushered in in that vein.
About the Guest
John Bohannon
Primer
Resources
- EleutherAI
- Data set: The Pile
- Blog: The Ultimate Performance Metric in NLP
- DALL·E
- Article: Alien Dreams: An Emerging Art Scene
- Paper: GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
- Article: Alibaba Announces 10 Billion Parameter Multi-Modal AI M6
- Paper: Florence: A New Foundation Model for Computer Vision
- Paper: FLAVA: A Foundational Language And Vision Alignment Model
- Blog: ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
- Blog: Understanding Google’s Switch Transformer
- Blog: Google GLaM Vs DeepMind Gopher: Who Wins The Large Language Model Race
- Paper: Show Your Work: Scratchpads for Intermediate Computation with Language Models
- Paper: BSTC: A Large-Scale Chinese-English Speech Translation Dataset
- Paper: Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning
- Blog: GPT-3 Scared You? Meet Wu Dao 2.0: A Monster of 1.75 Trillion Parameters
- Paper: KLUE: Korean Language Understanding Evaluation
- Article: Naver trained a ‘GPT-3-like’ Korean language model
- Article: AI Can Write in English. Now It's Learning Other Languages
- Blog: XLS-R: Self-supervised speech processing for 128 languages
- Blog: The first-ever multilingual model to win WMT, beating out bilingual models
- Blog: Why we must rethink AI benchmarks
- Article: BigScience Research Workshop Releases AI Language Model T0
- Github: EleutherAI / lm-evaluation-harness
- Blog: Introducing Dynabench: Rethinking the way we benchmark AI
- Article: Teaching artificial intelligence right from wrong: New tool from AI2 aims to model ethical judgments
- Paper: Visually Grounded Reasoning across Languages and Cultures
- Blog: Our promises to you: Axios Bill of Rights
- Blog: The AI Company Helping the Pentagon Assess Disinfo Campaigns
- Blog: Learning to Summarize with Human Feedback
- Tweet: A Pony AI story
- Article: Deep Talk: A year’s worth of daily questions generated by a machine
- Blog: AI Dungeon-maker Latitude raises $3.3M to build games with ‘infinite’ story possibilities
- Blog: AI Dungeon’s creators are launching an experimental AI-powered game platform
- TWIMLfest Office Hours: NLP
- Taming arXiv with Natural Language Processing w/ John Bohannon - #136
- Deep Learning for NLP: From the Trenches w/ Charlene Chambliss - #433

