The release of Seldon Core 1.5.0 sees updates of Drift, Outlier and Explanations with Alibi, as well as default inference endpoints for REST and gRPC by default. Read the full summary below, or view the project on Github.
Inference endpoints for REST and gRPC by default
We now expose both REST and gRPC endpoints for all inference graphs by default. This means the graph definitions no longer need to contain
endpoint.type. This has been made possible by updating our python wrapper to allow both REST and gRPC endpoints to be exposed along with all the prepackaged servers. So in the past you would have a resource like below for a gRPC model:
apiVersion: machinelearning.seldon.io/v1 kind: SeldonDeployment metadata: name: grpc-seldon spec: name: grpcseldon protocol: seldon transport: grpc predictors: - componentSpecs: - spec: containers: - image: seldonio/mock_classifier_grpc:1.3 name: classifier graph: name: classifier type: MODEL endpoint: type: GRPC name: model replicas: 1
From 1.5 the same resource would look like:
apiVersion: machinelearning.seldon.io/v1 kind: SeldonDeployment metadata: name: example-seldon spec: protocol: seldon predictors: - componentSpecs: - spec: containers: - image: seldonio/mock_classifier:1.5.0 name: classifier graph: name: classifier type: MODEL name: model replicas: 1
mock_classifier:1.5.0 has been wrapped with the 1.5.0 python wrapper. For models wrapped with older versions of Seldon Core the REST or gRPC endpoint will still continue to function. See upgrading for details.
Istio and Amabssador configurations have been updated to allow both REST and gRPC configurations.
Updated Drift, Outlier and Explanations with Alibi
Our CIFAR10 outlier detection example and CIFAR10 drift detection example have been updated and now requires KNative 0.18 with a compatible istio (tested on 1.7.3). This utilizes the v1 resources of KNative Eventing to show off asnychronous outlier and drift detection on images.
Our explanations examples using Alibi Explain have been updated to the latest 0.4.3 release.
- Our batch processor has been updated to allow feedback requests to be sent to allow accuracy and other metrics to be tracked for deployed models.
- Our KEDA autoscaling example has been updated to use the stable v1 release of KEDA.
Interested in finding out more about Seldon’s products? Get a deploy demo today
Clive is Chief Technology Officer at Seldon and has more than 25 years of experience in IT and technology. Since its launch in 2014, Clive has led the creation of the company’s core product offering. Clive is a regular speaker at industry-leading conferences where he relays the story of Seldon’s technology, and has contributed to many ML open source projects, including Kubeflow. His PhD research was on Natural Language Processing and he previously worked successfully for early speech recognition pioneers.