Scaling TensorFlow at LinkedIn with Jonathan Hung

Today we're joined by Jonathan Hung, Sr. Software Engineer at LinkedIn, who we caught up with at TensorFlow World last week. Jonathan's presentation at the event focused on LinkedIn's efforts scaling Tensorflow. In our conversation, we discuss his work as part of the Hadoop infrastructure team, experimenting on Hadoop with various frameworks, and their motivation for using TensorFlow on their pre-existing Hadoop clusters infrastructure. Jonathan also details TonY, or TensorFlow on Yard, LinkedIn's framework that natively runs deep learning jobs on Hadoop, its relationship with Pro-ML, LinkedIn's internal AI Platform, which we've discussed on earlier episodes of the podcast. Finally, we discuss how far LinkedIn's Hadoop infrastructure has come since 2017, and their foray into using Kubernetes for research.