Procedure

1. Create an opaque Kubernetes Secret containing your operational credentials and query validation keys before applying the Backstage CR. Refer to the following key definitions for required environment variables:

   Important

   The Secret supplies only provider credentials and endpoints. To enable an inference provider, define it as an entry under the inference.providers section of the lightspeed-stack.yaml file. Adding a credential key to the Secret does not enable a provider on its own.

   Key	Description
   VLLM_URL	Specifies the target API endpoint URL for vLLM. The URL must end in /v1, for example, https://<api_endpoint>/v1.
   VLLM_API_KEY	Stores the authorization token for your vLLM platform.
   OPENAI_API_KEY	Stores the authorization secret key for OpenAI.
   VERTEX_AI_PROJECT	Specifies your Google Cloud project ID.
   VERTEX_AI_LOCATION	Specifies your target Google Cloud region.
   GOOGLE_APPLICATION_CREDENTIALS	Specifies the in-container file path of your mounted Google Cloud service account credentials JSON file.
   ENABLE_VALIDATION	Enables the question-validity shield when set to question_validity. Leave the variable unset, or set it to disabled, to keep validation disabled. A nonempty value such as true does not enable the shield.
   VALIDATION_PROVIDER	Specifies the type of the inference provider that runs the classification, for example, openai or vllm. A provider of this type must be enabled in inference.providers.
   VALIDATION_MODEL_NAME	Specifies the model that classifies each prompt, for example, gpt-4o-mini.

   The following code shows an example configuration Secret for vLLM with validation:

   apiVersion: v1
   kind: Secret
   metadata:
     name: intelligent-assistant-auth-secrets
   type: Opaque
   stringData:
     VLLM_URL: "https://<api_endpoint>/v1"
     VLLM_API_KEY: "<api_key>"
     ENABLE_VALIDATION: question_validity
     VALIDATION_PROVIDER: "vllm"
     VALIDATION_MODEL_NAME: "llama3.1"

   Note

   The VALIDATION_PROVIDER value is the inference provider type, such as vllm or openai. A provider of this type must be enabled in the inference.providers section of the lightspeed-stack.yaml file.

2. Enable at least one inference provider by defining it under the inference.providers section of the lightspeed-stack.yaml file. Creating the Secret alone does not enable a provider. For the provider entry format and the Secret keys that each provider requires, see the inference provider configuration reference for Developer Hub Intelligent Assistant.

3. Map your secret inside the extraEnvs section of the Backstage CR to complete container provisioning:

   apiVersion: rhdh.redhat.com/v1alpha5
   kind: Backstage
   metadata:
     name: intelligent-assistant-rhdh
   spec:
     application:
       extraEnvs:
         secrets:
           - name: intelligent-assistant-auth-secrets
             containers:
               - lightspeed-core

4. Optional: To protect settings such as Model Context Protocol (MCP) server additions from being overwritten during reconciliation loops, define a custom ConfigMap mapping in the extraFiles section of the CR:

       extraFiles:
         configMaps:
           - name: "my-custom-config"
             mountPath: /app-root
             key: lightspeed-stack.yaml
             containers:
               - lightspeed-core

5. Configure access rights by updating the RBAC policy inside your Backstage CR:

   To grant non-administrator teams access to the virtual assistant, append permission lines to the rbac-policies.csv section, replacing <team> with your target team name:

   p, role:default/<team>, intelligent-assistant.chat, use, allow 1

   <1>

   Grants full use of the Developer Hub Intelligent Assistant chat feature, including opening the chat window, sending messages, managing chat history, and saving and reusing chat prompts.

   For the complete list of AI feature permissions, including Notebooks, MCP tools, and skills, see AI feature permissions.

6. Apply the updated custom resource manifest to your cluster:

   $ oc apply -f <backstage_cr_file>.yaml

Verification

1. Log in to your console instance.
2. Verify that the Intelligent Assistant button appears on the home page.
3. Select the Intelligent Assistant button and confirm that the chat window initializes successfully.

Procedure

1. Create a Kubernetes Secret to store your provider credentials by adding the required keys to your Secret based on your provider requirements:

   Important

   The Secret supplies only provider credentials and endpoints. To enable an inference provider, define it as an entry under the inference.providers section of the lightspeed-stack.yaml file. Adding a credential key to the Secret does not enable a provider on its own.

   Key	Description
   VLLM_URL	Specifies the target API endpoint URL for vLLM. The URL must end in /v1, for example, https://<api_endpoint>/v1.
   VLLM_API_KEY	Stores the authorization token for your vLLM platform.
   OPENAI_API_KEY	Stores the authorization secret key for OpenAI.
   VERTEX_AI_PROJECT	Specifies your Google Cloud project ID.
   VERTEX_AI_LOCATION	Specifies your target Google Cloud region.
   GOOGLE_APPLICATION_CREDENTIALS	Specifies the in-container file path of your mounted Google Cloud service account credentials JSON file.
   ENABLE_VALIDATION	Enables the question-validity shield when set to question_validity. Leave the variable unset, or set it to disabled, to keep validation disabled. A nonempty value such as true does not enable the shield.
   VALIDATION_PROVIDER	Specifies the type of the inference provider that runs the classification, for example, openai or vllm. A provider of this type must be enabled in inference.providers.
   VALIDATION_MODEL_NAME	Specifies the model that classifies each prompt, for example, gpt-4o-mini.

   Note

   Vertex AI requires custom architecture mapping and has received limited testing.

   Tip

   To filter and reject off-topic user queries, you can optionally configure query validation within this Secret by defining the ENABLE_VALIDATION, VALIDATION_PROVIDER, and VALIDATION_MODEL_NAME keys.

2. Reference your Secret inside the values.yaml file:

   intelligentAssistant:
     existingSecret: "my-custom-secret"

3. Enable at least one inference provider by defining it under the inference.providers section of the lightspeed-stack.yaml file. Creating the Secret alone does not enable a provider. For the provider entry format and the Secret keys that each provider requires, see the inference provider configuration reference for Developer Hub Intelligent Assistant.

4. Optional: Configuration files such as lightspeed-stack.yaml and rhdh-profile.py are managed by the Helm deployment and are overwritten on helm upgrade runs. To protect changes to a file, create a custom config map and reference it in the values.yaml file:

   Important

   Provide only the name and key of your existing ConfigMap. The chart manages the mount path and file configuration.

   intelligentAssistant:
     config:
       stack:
         existingConfigMap:
           name: my-custom-stack
           key: lightspeed-stack.yaml

5. Configure access rights by updating your RBAC definitions:

   To grant non-administrator teams access to the virtual assistant, append permission lines to the rbac-policies.csv section, replacing <team> with your target team name:

   p, role:default/<team>, intelligent-assistant.chat, use, allow 1

   <1>

   Grants full use of the Developer Hub Intelligent Assistant chat feature, including opening the chat window, sending messages, managing chat history, and saving and reusing chat prompts.

   For the complete list of AI feature permissions, including Notebooks, MCP tools, and skills, see AI feature permissions.

6. Run the helm upgrade command to apply your configurations to the cluster.

Verification

1. Log in to your console instance.
2. Verify that the Intelligent Assistant button appears on the home page.
3. Select the Intelligent Assistant button and confirm that the chat window initializes successfully.

Procedure

1. Open your Backstage custom resource (CR) YAML file.

2. In the spec section, set the enabled flag to false for the intelligent-assistant flavour. This disables the chat interface and prevents the Operator from injecting unconfigured sidecar containers:

   apiVersion: rhdh.redhat.com/v1alpha5
   kind: Backstage
   metadata:
     name: intelligent-assistant-disabled
   spec:
     flavours:
       - name: intelligent-assistant
         enabled: false

3. Apply the updated custom resource manifest to your cluster:

   oc apply -f <backstage_cr_file>.yaml

Verification

1. Log in to your console instance.
2. Verify that the Intelligent Assistant button no longer appears on the home page.

Procedure

1. Open your Helm values.yaml file.

2. Update the global.lightspeed.enabled parameter to false to disable the chat interface:

   global:
     lightspeed:
       enabled: false

3. Run the helm upgrade command to apply the configuration change to your cluster.

Verification

1. Log in to your console instance.
2. Verify that the Intelligent Assistant button no longer appears on the home page.

Procedure

1. In your custom configuration file, such as lightspeed-stack.yaml, modify the user_data_collection block to configure your data preferences:

   • To enable feedback collection, set the feedback_enabled parameter to true:

     user_data_collection:
       feedback_enabled: true
       feedback_storage: "/tmp/data/feedback"
       transcripts_enabled: true
       transcripts_storage: "/tmp/data/transcripts"

   • To disable feedback collection, set the feedback_enabled parameter to false:

     user_data_collection:
       feedback_enabled: false
       feedback_storage: "/tmp/data/feedback"
       transcripts_enabled: true
       transcripts_storage: "/tmp/data/transcripts"

     Note

     Do not modify the feedback_storage or transcripts_storage data paths when disabling feedback. Altering these path strings prevents the service from locating existing historical logs.

2. Apply the updated configuration file changes to your cluster by running your platform’s standard deployment or upgrade sequence.

Procedure

1. In your app-config.yaml file, add or modify the systemPrompt parameter under the intelligent-assistant section, specifying your custom instruction string:

   intelligent-assistant:
     # ... other intelligent-assistant configurations
     systemPrompt: "You are a helpful assistant focused on Red Hat Developer Hub development."

2. Save the file.
3. Restart the RHDH service to apply the updated system prompt configuration.

Procedure

1. In your custom configuration file, such as lightspeed-stack.yaml, modify the conversation_cache block to specify your storage configuration:

   • To enable persistent storage, add your PostgreSQL database credentials and endpoint properties:

     conversation_cache:
       type: "postgres"
       postgres:
         host: _<your_database_host>_
         port: _<your_database_port>_
         db: _<your_database_name>_
         user: _<your_user_name>"_
         password: _<postgres_password>_

     • To retain the default non-persistent SQLite setup, verify that the parameters match the following paths:

       conversation_cache:
         type: "sqlite"
         sqlite:
           db_path: '/tmp/cache.db'

2. Restart the LCORE service to apply your new database configuration.

   Note

   Saved chat prompts are stored on the same backend as chat history. When you configure persistent storage, RHDH retains both chat history and saved prompts across system restarts.

Procedure

1. In your app-config.yaml file, add or modify the rateLimit block under the intelligent-assistant section:

   intelligent-assistant:
     rateLimit:
       expensive:
         max: 25
       general:
         max: 200

   expensive.max

   Maximum requests per minute per user for expensive endpoints such as LLM inference. Set to 0 to disable rate limiting for this tier.

   general.max

   Maximum requests per minute per user for general endpoints. Set to 0 to disable rate limiting for this tier.

   The following example shows a tightened configuration for a small deployment with limited LLM resources:

   lightspeed:
     rateLimit:
       expensive:
         max: 10
       general:
         max: 50

2. Save the file.
3. Restart the RHDH service to apply the updated configuration.

Verification

1. Log in to RHDH and submit requests to the Intelligent Assistant chat interface that exceed the configured expensive limit.
2. Confirm that the server returns an HTTP 429 Too Many Requests response with a Retry-After header indicating when to retry.

Procedure

1. Enable the notebook feature and define your model by adding the following configuration to your app-config.yaml file:

   intelligent-assistant:
     notebooks:
       enabled: true
       queryDefaults:
         model: ${NOTEBOOKS_QUERY_MODEL} # Use the exact model name
         provider_id: ${NOTEBOOKS_QUERY_PROVIDER_ID}

   Note

   If the model name is wrong, an error message is displayed in the logs and the user interface.

2. Grant user access through role-based access control (RBAC) policies by defining permissions in your rbac-policy-csv file:

   1. Add the permission policies:

      p, role:default/<your_team_name>, intelligent-assistant.notebooks, use, allow

   2. Assign the role to specific users:

      g, user:default/<your_user_name>, role:default/<your_team_name>

3. Apply the updated configuration and restart the service.

Verification

1. Log in to RHDH using an account assigned to the RBAC role defined in the configuration.
2. Confirm that the Notebooks tab is visible next to the Chat tab in the primary navigation bar.
3. Click the Notebooks tab and ensure the My Notebooks dashboard loads without error messages.

Procedure

1. In your lightspeed-stack.yaml config map, point the notebooks vector store provider at a path on the persistent mount. The provider stays faiss, so the FAISS index continues to persist as an on-disk file:

   vector_store:
     default_provider: notebooks
     providers:
       - id: notebooks
         type: faiss
         embedding_model: nomic-ai/nomic-embed-text-v1.5
         embedding_dimension: 768
         config:
           path: /var/lib/lightspeed-data/vector_db/notebooks/faiss_store.db

2. Update your deployment manifest to create the directory on the volume and mount the PVC into the Intelligent Assistant container. The config map mounts are already defined by the deployment and are unchanged:

   spec:
     template:
       spec:
         securityContext:
           fsGroup: 1000
         initContainers:
           - name: init-notebooks-storage
             image: registry.access.redhat.com/ubi9/ubi-minimal
             command:
               - sh
               - -c
               - mkdir -p /var/lib/lightspeed-data/vector_db/notebooks
             volumeMounts:
               - name: intelligent-assistant-notebooks
                 mountPath: /var/lib/lightspeed-data
         containers:
           - name: lightspeed-core
             volumeMounts:
               - name: intelligent-assistant-notebooks
                 mountPath: /var/lib/lightspeed-data
         volumes:
           - name: intelligent-assistant-notebooks
             persistentVolumeClaim:
               claimName: intelligent-assistant-notebooks-pvc

3. Apply the updated configuration and restart the service.

Verification

1. In Red Hat Developer Hub, create a Notebook and upload a test document.
2. Send a message to the virtual assistant and verify that the response is based on the document.

3. Restart the pod:

   $ oc delete pod <pod_name>

4. After the pod recovers, refresh the My Notebooks dashboard.
5. Verify that the Notebook and the uploaded file are still accessible.

Procedure

1. To enable query validation, add the following keys to the Secret that LCORE reads:

   stringData:
     ENABLE_VALIDATION: question_validity
     VALIDATION_PROVIDER: <provider_type>
     VALIDATION_MODEL_NAME: <model_name>

   where:

   ENABLE_VALIDATION

   Specifies whether the shield is enabled. You must set the value to question_validity. A nonempty value such as true does not enable the shield in RHDH 2.1.

   VALIDATION_PROVIDER

   Specifies the type of the inference provider that runs the classification, for example, openai or vllm. A provider of this type must be enabled in inference.providers.

   VALIDATION_MODEL_NAME

   Specifies the model that classifies each prompt. If the provider defines allowed_models, include this model in that list.

   For example, to run classification on an openai provider with the gpt-4o-mini model, set the following values:

   stringData:
     ENABLE_VALIDATION: question_validity
     VALIDATION_PROVIDER: openai
     VALIDATION_MODEL_NAME: gpt-4o-mini

2. To keep query validation disabled, leave ENABLE_VALIDATION unset, or set it to disabled:

   stringData:
     ENABLE_VALIDATION: __disabled__

   Important

   Running without query validation increases the risk of off-topic or harmful model output. Disable validation only in secure development environments.

3. Apply the updated Secret, and then restart the LCORE service so that the container reads the new environment variables.

Verification

1. Open the Intelligent Assistant and submit a development question, for example, How do I create a software template in RHDH?. Confirm that the assistant answers the question.
2. Submit a clearly unrelated question, for example, What is a good recipe for pizza?. Confirm that the assistant rejects the question when validation is enabled.

Procedure

1. In the lightspeed-stack.yaml file, add the allowed_models field under extra for each inference provider, and list the models that you want to expose to users:

   inference:
     providers:
       - type: openai
         id: openai
         api_key_env: OPENAI_API_KEY
         extra:
           allowed_models:
             - gpt-4o-mini

   where:

   allowed_models

   The list of models that appear in the model selection dropdown menu for that provider. Models that you omit from this list are hidden from users.

2. Apply the modified lightspeed-stack.yaml file by using a custom ConfigMap:

   For Operator-based deployments, map the ConfigMap in the extraFiles section of the Backstage custom resource.

   For Helm-based deployments, reference the ConfigMap in the values.yaml file.

3. Apply the updated configuration to your cluster, and restart the deployment if it does not trigger an automatic rollout.

Verification

1. Log in to your RHDH instance.
2. Open the chat dialog and select the model selection dropdown menu.
3. Confirm that the dropdown menu lists only the allowed models.

Procedure

1. Click the Open intelligent assistant floating action button (FAB) at the lower right of the screen to open the chat overlay.

   

2. Optional: Configure the interface display and server settings:

   • Click the Chatbot options icon (⋮) in the header to open the options menu.

     
   • In the Chatbot options menu, toggle Enable pinned chats or Disable pinned chats to show or hide the pinned chats. The system enables this option by default.
   • Available only if Model Context Protocol (MCP) is configured: In the Chatbot options menu, click MCP and Prompt Settings to manage Model Context Protocol connections.

   • In the Chatbot options menu, under Display mode, select any of the following views:

     • Overlay: A floating window is displayed over the current page content.

       

     • Dock to window: A panel attaches to the right side of the screen. Activating this mode automatically closes the Quick start panel if it is already open.

       

     • Fullscreen: A dedicated page opens for intensive chat sessions. This mode displays a revised header containing the Developer Hub Intelligent Assistant icon and a horizontal tab bar, which replaces the previous main menu. Bookmark the URL in your browser to save a direct link to the chat interface.

       

3. Start a chat or load an earlier session:

   • Enter a prompt: Type a query in the Enter a prompt for the intelligent assistant chat field and press Enter.
   • Use a sample: Click a prompt tile.

   • Change the AI model: Select a model from the model selector dropdown menu inside the prompt bar.

     

   • Attach a file: Click the (+) icon on the left of the prompt bar to upload a .yaml, .json, or .txt file. Descriptive text clarifies the function of the icon.

     
   • Use voice: Click the Use microphone icon. The microphone and send buttons are located on the right side of the prompt bar.

   • Control AI generation: Use the control buttons on the right side of the prompt bar. Click the Send (>) button to submit queries, or click the Stop button to halt AI generation.

     
     
   • Resume a chat: Select a title from the Chats list.

4. Organize your chat history:

   • Start a new topic: Click New chat to reset the assistant’s context. When the history panel is collapsed, click the New chat icon to create a new chat.

     
     
   • Search history: Enter a keyword in the Search field.

   • Rename a session: Click Options next to a chat title, select Rename, enter a new name in the Rename chat? dialog, and click Rename.

     
   • Pin a chat: Click Options next to a chat title and select Pin. The chat moves to the Pinned group.

   • Sort chats: Click the Sort control and choose a sorting criterion, such as Date (Newest first).

     

   • Delete a chat: Click Options next to a chat title and select Delete.

     
   • Collapse or expand a category: Click a category header, such as Pinned, Saved prompts, or Recent, to collapse or expand that section of the chat history sidebar.

   • Expand or collapse the panel: Click the Expand/collapse icon to toggle the history panel.

     
     
     Note

     The expanded panel is resizable up to a defined maximum width.

5. Optional: To hide the interface, if you are in the Overlay or Dock to window mode, click the Close intelligent assistant FAB to hide the window. If you are in Fullscreen mode, revert to the other modes and click the Close intelligent assistant FAB. The system preserves your active query and history.

Verification

1. The main window displays the active chat or selected history.
2. The chat history list reflects renamed, pinned, or deleted entries.

Procedure

1. Click the Open intelligent assistant floating action button (FAB) at the lower right of the screen to open the chat overlay.

2. Create a saved prompt:

   1. Click the Chatbot options icon (⋮) in the header and select MCP and Prompt Settings.

      

   2. Click the Saved prompts tab.

      
   3. Click + New prompt.

   4. Enter a Title of 128 characters or fewer and a Prompt of 5,000 characters or fewer.

      

   5. Click Save.

      Note

      By default, you can save up to 100 prompts. When you reach the limit, delete an existing prompt before you create a new one.

3. Reuse a saved prompt from the Saved prompts section of the chat history sidebar or from the Saved prompts tab:

   

   • To insert the prompt into the message field so that you can review or edit it before sending, click the prompt, or select Apply in input box from the prompt’s options menu.

     

   • To insert and submit the prompt immediately, select Send directly from the prompt’s options menu.

     
   Note

   When the sidebar lists more than five saved prompts, click Show all to display the remaining prompts, or click Show less to collapse the list.

4. Optional: To remove a saved prompt, select Delete from the prompt’s options menu, and then confirm the deletion. Deleting a saved prompt is permanent.

   

5. Optional: To open the Saved prompts settings directly, click the settings (gear) icon next to the Saved prompts section in the chat history sidebar.

   
6. Optional: To show or hide the Saved prompts feature, click the Chatbot options icon (⋮) in the header and toggle Enable saved prompts or Disable saved prompts. Saved prompts are enabled by default. When you disable the feature, RHDH hides the Saved prompts section and excludes your saved prompts from the default prompts offered in a new chat.

Verification

1. The Saved prompts tab displays the prompt count and your new prompt.
2. Clicking or sending a saved prompt applies its content in the message field.
3. Deleted prompts are removed from both the sidebar and the Saved prompts tab.

Procedure

1. Clone the evaluation repository and navigate to the directory:

   git clone https://github.com/lightspeed-core/lightspeed-evaluation
   cd lightspeed-evaluation

2. Synchronize the environment and install dependencies:

   uv sync

3. Configure the environment variables for the judge LLM. You can create a .env file in the root directory or export the keys directly to your terminal.

   • If you use Gemini, you must set the Gemini API key:

     export GEMINI_API_KEY="your-google-api-key"

   • If you use OpenAI, you must set the OpenAI API key:

     export OPENAI_API_KEY="your-key"

4. Optional: If you test with a live service, set your Developer Hub Intelligent Assistant service API key:

   export API_KEY="your-lightspeed-service-key"

Procedure

1. Download pre-generated data sets: Use this method to test the performance of specific RHDH releases. These data sets are generated using Ragas testset generation for RAG.

   1. In your terminal, navigate to the /dataset folder in the evaluation repository.
   2. Locate the .evaluation_dataset_yaml files. These files are pre-configured for the evaluation tool.

   3. To test a historical release, switch to the corresponding branch.

      For example, to access the Red Hat Developer Hub 1.8 data set, switch to the 1.8 branch.

      Important

      The main branch contains work-in-progress (WIP) data sets. Avoid using this branch for stable evaluations.

2. Generate custom data sets: Use this method to create a new test set from your own technical documentation.

   1. Generate a diverse set of question-and-answer (Q&A) pairs by following the Ragas test data generation documentation.
   2. Ensure your Q&A pairs match the required format by reviewing the evaluation data structure configuration.

Procedure

1. Download the system.yaml configuration template from the repository.

2. Configure the parameters in the system.yaml file based on your evaluation mode:

   Field	Description
   llm	Defines the judge LLM that scores the responses, such as gemini-2.5-pro.
   api.enabled	Set to false for static mode to use pre-filled data. Set to true for dynamic mode to call a live service.
   api.api_base	(Required for dynamic mode only) Provide the URL of your Developer Hub Intelligent Assistant service.
   api.endpoint_type	Specify the service configuration type: streaming or query.

3. Execute the evaluation by using the lightspeed-eval command:

   lightspeed-eval \
     --system-config config/system.yaml \
     --eval-data config/evaluation_data.yaml \
     --output-dir ./my_evaluation_results

Procedure

1. In the root of the repository, navigate to the version-specific folder within the /evaluation-result directory.

2. Open the following files to evaluate performance:

   • Model Pass Rate: Compare the overall performance between different LLMs.
   • Topic Pass Rate: Identify performance trends and gaps within specific documentation areas.