Procedure

1. Configure the Kubernetes role-based access control (RBAC) and credentials that the connector uses to access the cluster.

   The connector requires a Kubernetes ServiceAccount with least-privilege, read-only access to the KServe and OpenShift Container Platform resources that it discovers. Create the following resources, replacing the namespace (ai-rhdh) as needed for your RHDH deployment:

   1. A ServiceAccount for the connector. For example:

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: rhdh-kserve-kubeflow-connector
        namespace: ai-rhdh

   2. A ClusterRole and ClusterRoleBinding that grant read-only access to InferenceService and LLMInferenceService resources, OpenShift Container Platform Route resources, and ServiceAccount resources. For example:

      # Example ClusterRole
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: rhdh-kserve-kubeflow-connector
      rules:
        - apiGroups: ["serving.kserve.io"]
          resources: ["inferenceservices", "llminferenceservices"]
          verbs: ["get", "list", "watch"]
        - apiGroups: ["route.openshift.io"]
          resources: ["routes"]
          verbs: ["get", "list", "watch"]
        - apiGroups: [""]
          resources: ["serviceaccounts"]
          verbs: ["get", "list", "watch"]

      # Example ClusterRoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: rhdh-kserve-kubeflow-connector
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: rhdh-kserve-kubeflow-connector
      subjects:
        - kind: ServiceAccount
          name: rhdh-kserve-kubeflow-connector
          namespace: ai-rhdh

      Note

      The connector reads OpenShift Container Platform Route resources only when you do not configure a Kubeflow Model Catalog URL and the connector must discover the route automatically. It reads ServiceAccount resources to detect whether a model server requires authentication.

   3. A Secret of type kubernetes.io/service-account-token for the ServiceAccount. For example:

      apiVersion: v1
      kind: Secret
      metadata:
        name: rhdh-kserve-kubeflow-connector-token
        namespace: ai-rhdh
        annotations:
          kubernetes.io/service-account.name: rhdh-kserve-kubeflow-connector
      type: kubernetes.io/service-account-token

2. Update your RHDH dynamic plugin configuration to install the connector and its companion plugins.

   In your RHDH dynamic plugins configuration, add the following plugins:

   plugins:
     - disabled: false
       package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-kserve-kubeflow-connector-backend:<tag>
     - disabled: false
       package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-catalog-backend-module-ai-model-server:<tag>
     - disabled: false
       package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-catalog-backend-module-model-catalog:<tag>
     - disabled: false
       package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-catalog-techdoc-url-reader-backend:<tag>

   where:

   <tag>

   Enter your RHDH version of Backstage and the plugin version, in the format bs_<backstage-version>__<plugin-version> (note the double underscore delimiter). To find these versions, complete the following steps:

3. Find your Backstage version in the RHDH release notes preface.

4. Locate the plugin version in the Dynamic Plugins Reference guide. For example, for a RHDH version based on Backstage <backstage-version>, use the format bs_<backstage-version>__<plugin-version>.

   Tip

   To ensure environment stability, use a SHA256 digest instead of a version tag. See Determining SHA256 Digests.

   where:

   kserve-kubeflow-connector-backend

   The connector backend plugin. It watches KServe InferenceService and LLMInferenceService resources and exposes the REST routes that the other plugins consume.

   catalog-backend-module-ai-model-server

   The catalog module that registers the AiModelServerAPI entity kind so that the catalog accepts the entities that the connector generates.

   catalog-backend-module-model-catalog

   The entity provider that calls the connector REST route and emits AiModelServerAPI catalog entities.

   catalog-techdoc-url-reader-backend

   The TechDocs URL reader that resolves model-card content through the connector for TechDocs generation.

5. Allow the AiModelServerAPI entity kind in your catalog rules by adding it to your app-config.yaml file:

   catalog:
     rules:
       - allow:
           [
             Component,
             System,
             API,
             AiModelServerAPI,
             Resource,
             Location,
             User,
             Group,
           ]

Procedure

1. Configure cluster access by using one of the following modes.

   • Mode 1: Reuse the Backstage Kubernetes plugin cluster

     If the Backstage Kubernetes plugin is already configured with access to the cluster that hosts KServe, point the connector at that cluster by using the kubernetesPluginRef field. Add the following configuration to your app-config.yaml file:

     kubernetes:
       serviceLocatorMethod:
         type: 'multiTenant'
       clusterLocatorMethods:
         - type: 'config'
           clusters:
             - name: my-k8s-cluster
               url: '${K8S_CLUSTER_URL}'
               serviceAccountToken: '${K8S_SA_TOKEN}'
               authProvider: serviceAccount
               skipTLSVerify: false
               caData: '${K8S_CA_DATA:-}'
     catalog:
       providers:
         modelCatalog:
           kserve-kubeflow-connector:
             cluster-1:
               name: my-k8s-cluster
               kubernetesPluginRef: my-k8s-cluster
               default-owner: '${OWNER:-default-owner}'
               default-lifecycle: '${LIFECYCLE:-production}'

     For more information about configuring cluster access for the Backstage Kubernetes plugin, see the Configuring Red Hat Developer Hub guide.

   • Mode 2: Direct cluster connection

     If the Backstage Kubernetes plugin is not installed, or it does not define the target cluster, supply the connection fields directly on the connector cluster entry. Add the following configuration to your app-config.yaml file:

     catalog:
       providers:
         modelCatalog:
           kserve-kubeflow-connector:
             cluster-1:
               name: my-k8s-cluster
               url: '${K8S_CLUSTER_URL:-}'
               serviceAccountToken: '${K8S_SA_TOKEN:-}'
               skipTLSVerify: false
               caData: '${K8S_CA_DATA:-}'
               default-owner: '${OWNER:-default-owner}'
               default-lifecycle: '${LIFECYCLE:-production}'

   where:

   cluster-1

   A unique key that identifies the cluster entry. You can define multiple cluster entries.

   name

   The display name of the cluster.

   kubernetesPluginRef

   In Mode 1, the name of the cluster that is defined under kubernetes.clusterLocatorMethods.

   url

   In Mode 2, the URL of the cluster API server.

   serviceAccountToken

   In Mode 2, the ServiceAccount token that the connector uses to authenticate to the cluster.

   skipTLSVerify

   Whether to skip TLS verification of the cluster API server certificate. Set this to false in production.

   caData

   The base64-encoded PEM certificate authority (CA) bundle for the cluster API server. For more information, see Configure TLS for the connector.

   default-owner

   The default owner that is applied to generated entities when an InferenceService or LLMInferenceService does not specify an owner.

   default-lifecycle

   The default lifecycle that is applied to generated entities when an InferenceService or LLMInferenceService does not specify a lifecycle.

2. Optional: To import model cards from the Kubeflow Model Catalog, configure the model catalog URL by adding the kubeflow-model-catalog-url field to the cluster entry:

   catalog:
     providers:
       modelCatalog:
         kserve-kubeflow-connector:
           cluster-1:
             name: my-k8s-cluster
             kubernetesPluginRef: my-k8s-cluster
             default-owner: my-team
             default-lifecycle: production
             kubeflow-model-catalog-url: https://model-catalog.apps.example.com

   where:

   kubeflow-model-catalog-url

   The URL of the Kubeflow Model Catalog service. When you omit this field, the connector attempts to discover the model catalog route automatically on the cluster.

   Verification

   • Restart your RHDH instance and confirm that model servers from your cluster appear in the RHDH Catalog as AiModelServerAPI entities.

Procedure

1. Retrieve the OpenAPI specification from the running endpoint of the model server. Use a tool such as curl to fetch the specification from the /openapi.json path, and include a Bearer token if the model requires authentication:

   $ curl -k -H "Authorization: Bearer $MODEL_API_KEY" https://$MODEL_ROOT_URL_INCLUDING_PORT/openapi.json | jq > open-api.json

2. Define a Backstage API entity that embeds the OpenAPI specification, and store it in a catalog-info.yaml file in your Git repository. Set the spec.definition field to the contents of the open-api.json file:

   apiVersion: backstage.io/v1alpha1
   kind: API
   metadata:
     name: my-model-api
     description: OpenAPI specification for my model server
   spec:
     type: openapi
     lifecycle: production
     owner: my-team
     definition: |
       <contents of open-api.json>

3. Import the catalog-info.yaml file into the RHDH Catalog as a location so that the API entity is registered. For more information about registering catalog locations, see the Configuring Red Hat Developer Hub guide.

4. Annotate the InferenceService or LLMInferenceService for the model server with the rhdh.io/api-entity-ref annotation, and set its value to the entity reference of the API entity that you registered:

   apiVersion: serving.kserve.io/v1beta1
   kind: InferenceService
   metadata:
     name: my-model-server
     namespace: my-models
     annotations:
       rhdh.io/api-entity-ref: api:default/my-model-api
   spec:
     predictor:
       model:
         modelFormat:
           name: pytorch

Verification

1. Restart your RHDH instance, or wait for the connector to run on its configured schedule.
2. In the RHDH Catalog, open the generated AiModelServerAPI entity for the model server.
3. Confirm that the Definition tab displays the interactive API documentation from the referenced API entity.

Procedure

1. Remove the location, storage-rest, and rhoai-normalizer sidecar containers from your RHDH deployment:

   1. If you installed RHDH by using the Operator, remove the sidecar containers from your RHDH custom resource (CR).
   2. If you installed RHDH by using the Helm chart, remove the sidecar containers from the Deployment specification.

2. Remove the configuration that was specific to the sidecar deployment:

   1. The provider configuration that pointed to the local sidecar, for example, the modelCatalog provider with baseUrl: http://localhost:9090.
   2. The sidecar-specific ServiceAccount, ClusterRole, ClusterRoleBinding, Role, RoleBinding, and token Secret.
   3. The RoleBinding in the Kubeflow Model Registry namespace that granted the connector read access to the model registry.
   4. The bac-import-model config map that cached the model metadata.
3. Install the standalone connector and its companion plugins, and create the new Kubernetes RBAC resources. For more information, see Install the OpenShift AI Connector for Red Hat Developer Hub.
4. Add the new connector configuration, including the cluster access mode and, optionally, the Kubeflow Model Catalog URL. For more information, see Configure the OpenShift AI Connector for Red Hat Developer Hub.