Productionizing machine learning models in an organization is a difficult challenge for several reasons. From a technical perspective, it requires tooling to handle tasks such as model deployment, monitoring, and retraining. Talent-wise, these tasks require practitioners to possess technical skills in software engineering and DevOps. Coupled with a rapidly changing landscape and shortage of established best practices, operationalizing models is no small feat. Kubernetes provides machine learning practitioners the ability to deploy their model training and inference processes, scale deployed models vertically and horizontally, and can be extended to cover use-cases including model monitoring and A/B testing. The goal of this presentation is to discuss how Kubernetes can be leveraged to train, deploy, and monitor models in production settings. Throughout the talk we’ll reveal the technical and organizational lessons learned from using Kubernetes to productionize machine learning workloads at 2U.
