Kubeflow & ModelMesh-Serving Compatibility

[muratyarali]

Validation Checklist

• Is this a Kubeflow issue?
• Are you posting in the right repository ?
• Did you follow the Kubeflow installation guideline ?
• Is the issue report properly structured and detailed with version numbers?
• Is this for Kubeflow development ?
• Would you like to work on this issue?
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

1.9

Describe your issue

I deployed ModelMesh-Serving in a Kubeflow K8S cluster (kubeflow version: v1.9.1-RC.2) and created an InferenceService (ISVC) with modelmesh as follows:

kubectl apply -f - <<EOF
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: example-sklearn-isvc
  namespace: modelmesh-serving
  annotations:
    serving.kserve.io/deploymentMode: ModelMesh
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
EOF

Annotation is used to select the model mesh for deploying an ISVC. When I create an ISVC with KServe (without annotation), I can view it on the KServe Endpoints tab in the Kubeflow UI, and clicking on it reveals the details.

However, when I create the ISVC with ModelMesh-Serving, I can see it listed in the KServe Endpoints tab, but clicking on it does not display the details and instead shows an error:

[404] The requested resource could not be found in the API Server https://www.example.com/kserve-endpoints/api/namespaces/modelmesh-serving/revisions/undefined

The Kubeflow-UI is looking for resources at /kserve-endpoints/api/namespaces/modelmesh-serving/revisions/ which don't exist for ModelMesh.

Can this be fixed or extended on the Kubeflow or ModelMesh-Serving side?

Steps to reproduce the issue

1. I checked this with Kubeflow
2. I checked this ModelMesh-Serving

Put here any screenshots or videos (optional)

No response

[juliusvonkohout]

Probably yes, but what we need is someone willing to create PRs to fix it.