Here at Seldon, we’re immensely proud of the work we’re been doing on the KFServing project alongside other contributors from Google, Microsoft, Bloomberg and IBM — the official Kubeflow 1.0 release is a milestone worth celebrating.
Kubeflow is Google’s solution for deploying machine learning stacks on Kubernetes and was built to address two major issues with machine learning projects: the need for integrated, end-to-end workflows and the need to make deployments of machine learning systems simple, manageable and scalable. Largely, Seldon’s role in the KFServing project has been working on serverless machine learning inference.
As a core PM on the project, the components Seldon has created for KFServing includes the core functionality of ‘Model Explanations’. Thanks to the hard work of our technical team, now a model and an associated explainer can be launched to allow for explanation requests for particular model predictions. Explainers from Seldon’s Alibi Explain open source library can be configured alongside the running inference model.
The Seldon team have also added payload logging so requests and response payloads can be sent to processors and log storage asynchronously. This allows crucial machine learning governance capabilities such as Outlier Detection, Adversarial Detection and Concept Drift to be applied to the incoming requests to the core model being served for inference. State-of-the-art implementations of these are available in the Alibi Detect open-source library from Seldon.
We’re still hard at work collaborating with the team at Google and with the numerous other contributors for the next Kubeflow release to continuously improve the product. Our team understand that saving infrastructure costs is a key component to deploying machine learning components at scale. We are now helping KFServing look at model server sharing both for GPU and non-GPU use cases. We are also undertaking further work to integrate the full suite of Alibi components into KFServing to help it become a production-ready solution.
KFServing is already fully integrated as a model server in Seldon Deploy, our enterprise product for ML team workflows and governance. Seldon Core 1.0 is stable, remains a part of Kubeflow and Seldon Deploy, continues to be actively developed within the roadmap ahead, and will be fully supported for the long term. To learn about the differences between Kubeflow and Seldon Core, check out this overview on the Kubeflow docs.
Collaborative efforts like this in the tech community are important and it has been a fantastic experience working towards this release. To discuss KFServing, Alibi, or any other of Seldon’s industry-leading products, email us or join us on Slack.