About
Capital One
Capital One, a leading US retail bank, has recently deployed Seldon Core+ to support machine learning deployment across a number of business areas. With millions of customers and users of their mobile banking app, a robust, scalable and flexible approach to deployment of ML models was needed. Capital One had three key goals they wanted to achieve through Seldon’s model serving platform.
What was the challenge?
Steve Evangelista, Director of Product Management, and MLOps Lead Sumit Daryani noticed month-long lag times in their ML deployment pipeline. It was clear the Data Science teams were heavily reliant on Engineering to test, deploy or upgrade models. As a result, updates to existing models often came with a requirement to redeploy entire applications and individually integrating using the source code. Scaling up projects were only possible by using more developer resource and people power, further placing demand on the overstretched teams.
How did ThEy do it?
Using Seldon as the underlying infrastructure, the team set out to create a ‘Model as a Service’ platform (MaaS) using ML-based real-time decisioning that could do the heavy lifting in packaging and containerizing models for developers. The platform would form the building blocks for a number of internal applications, features, models and rules, and would allow data scientists, analysts and engineering groups to collaborate efficiently. By operationalising model deployment, data scientists could then deploy and safely test their models without placing a burden on tech engineering teams.
“With our Model as a Service’ platform (MaaS) running on Seldon, we’ve gone from it taking months to minutes to deploy or update models.”
Steve Evangelista
Director of Product Management
How Seldon WorkeD
“Seldon’s self-service installation process was simple and many of the things we needed came out of the box. Firstly, we were able to decouple the underlying models embedded in the streaming applications and serve them as REST/gRPC endpoints.”
Seldon’s components allowed Sumit to wrap a polyglot of ML models (H2O, scikit-learn, Tensorflow) into containers, then fit everything together to represent a service graph that could be seamlessly deployed into their platform. The entire application is wrapped around a service that goes through their CI/CD process to ensure Capital One as an organisation have met the versioning, registering, testing and governance requirements. Model invocation goes through another service that intercepts the model call for auditing, logging, monitoring before invoking the model being inferenced with Seldon Core+.
“We have been able to achieve zero downtime during machine image refreshes, so model executions are uninterrupted in a high throughput streaming environment through techniques like pod disruption budgets, liveness / readiness probes, graceful termination and connection draining of in-flight requests.”
What's next for Steve and sumit?
Within months, several use cases were bringing value to the business and Capital One’s customers, from fraud, marketing, finance and customer service. Versioning, vulnerability scanning, containerising, deployment, testing and promoting to production is all taken care of in this rapid process.
Seldon is now an integral part of Capital One’s MLOps stack, and in being so the data science team are now able to test, update and deploy models far quicker, meaning the process takes minutes rather than months as before. Lastly, Seldon has enabled organic adoption across Capital One as teams were able to deploy models to the system independent of the project owners.
“Our machine learning platform has powered gains across a number of our business domains. The use of Seldon therein has helped drive efficiencies in our machine learning processes and operational costs, without compromising on our data science needs.”
Download the PDF Version
Submit your email here and we will send a copy of the PDF right to your inbox.