Environmental Impact of Large-Scale NLP Model Training with Emma Strubell

800 800 This Week in Machine Learning & AI

Today we’re joined by Emma Strubell, currently a visiting scientist at Facebook AI Research.

Emma’s focus is on NLP and bringing state of the art NLP systems to practitioners by developing efficient and robust machine learning models. Her paper, Energy and Policy Considerations for Deep Learning in NLP, hones in on one of the biggest topics of the generation: environmental impact. Training neural networks has resulted in an impressive increase in accuracy, however the computational resources required to train these models is staggering – and carbon footprints are only getting bigger. In this episode, we discuss Emma’s research methods, how companies are reacting to environmental concerns, and  how we as an industry can be doing better.

About Emma

From the Interview

TWIMLcon Update

TWIMLcon: AI Platforms will feature live “keynote interview” podcast recordings with accomplished guests in the industry. Read on to learn about a few of the ones we just announced:

 Andrew Ng. I can’t think of anyone who’s done more to bring new practitioners into the fields of machine learning and deep learning than Andrew and we’re so excited to be opening the event with him. He’ll be sharing what he’s learned helping many businesses with machine learning and AI, and also speak with us about where he sees the field going.

Hussein Mehanna. Hussein is the Head of AI/ML Cruise, the self-driving car company. Before Cruise, Hussein helped build Google’s Cloud ML Platform and Facebook’s FBLearner platform. He’ll be sharing some of the lessons he’s learned building ML platforms from scratch at some of the most advanced companies in the space, and applying these lessons with much smaller teams.

Fran Bell. Fran is the director of data science responsible for Data Science Platforms at Uber. Fran leads a team building use-case focused ML platforms supporting areas like Forecasting, Anomaly Detection, Segmentation, NLP & Conversational AI, and more. Her platforms sit on top of Uber’s Michelangelo, putting her in a unique position to speak with us about how both low-level and higher-level ML platforms can drive data scientist and developer productivity.

Beyond keynote interviews like these, we’ve got a bunch of interesting speakers lined up to share their successes and failures helping their organizations build and productionalize ML and deep learning models. Stay tuned for more details! http://twimlcon.com/

Meetup Update!

Many of you are aware that we’ve been hosting a couple of paper-reading meetups in conjunction with the podcast. I’m excited to share that Matt Kenney, Duke staff researcher and long-time listener and friend of the show, has stepped up to help take this group to the next level. The paper reading meetup will now be meeting every other Sunday at 1 PM Eastern Time to dissect the latest and greatest academic research papers in ML and AI. If you want to take your understanding of the field to the next level, check twimlai.com/meetup for more upcoming community events.

We’ve also got a couple of study groups currently running, one working through the fast.ai Deep Learning from the Foundations course, another on fast.ai [Natural Language Processing] (https://www.fast.ai/2019/07/08/fastai-nlp/), and another working through the Stanford cs224n Deep Learning for Natural Language Processing course. These study groups just started and will be working on these courses through October and November, so it’s not too late to join. Sign up on the meetup page at twimlai.com/meetup.

Check it out

“More On That Later” by Lee Rosevere licensed under CC By 4.0

Leave a Reply

Your email address will not be published.