The Unreasonable Effectiveness of the Forget Gate with Jos van der Westhuizen

EPISODE 240
LISTEN
Banner Image: Jos van der Westhuizen - Podcast Interview

Join our list for notifications and early access to events

About this Episode

Today we're joined by Jos Van Der Westhuizen, PhD student in Engineering at Cambridge University. Jos' research focuses on applying LSTMs, or Long Short-Term Memory neural networks, to biological data for various tasks. In our conversation, we discuss his paper "The unreasonable effectiveness of the forget gate," in which he explores the various "gates" that make up an LSTM module and the general impact of getting rid of gates on the computational intensity of training the networks. Jos eventually determines that leaving only the forget-gate results in an unreasonably effective network, and we discuss why. Jos also gives us some great LSTM related resources, including references to Jurgen Schmidhuber, whose research group invented the LSTM, and who I spoke to back in Talk #44.

Thanks to our sponsor PegaSystems

Thanks to our friends at Pega for their support of the podcast and their sponsorship of today's show. Pega is a low-code platform for AI-powered decisioning and workflow automation. Its scalable architecture helps the world's leading organizations work smarter, unify experiences, and adapt instantly - so they're always ready for what's next. Check out the latest from Pega at their annual conference PegaWorld inspire, which will focus on how to address constantly shifting perspectives in an ever-evolving world, including guidance, strategies, and powerful tools to achieve resiliency in the face of rapid change. The event will be held virtually on May 24th for the Americas and Europe. And again, on May 25th for Asia Pacific. Agenda and registration details can be found at pegaworld.com.

 

Pegasystems Logo

More from TWIML

Leave a Reply

Your email address will not be published. Required fields are marked *