Procedure

1. Optional: To disable the chat interface and prevent the Operator from injecting unconfigured sidecar containers, set the lightspeed.enabled flag to false in the spec section of your Backstage CR YAML file:

   spec:
     application:
       lightspeed:
         enabled: false

2. Create an opaque Kubernetes Secret containing your operational credentials and query safety guardrails before applying the Backstage CR. Refer to the following key definitions for required environment variables:

   Important

   To disable an inference provider or configuration feature, you must leave the corresponding ENABLE_* variable completely unset. Setting an ENABLE_* variable to false does not disable the component because the underlying system checks only whether the variable is defined.

1. Map your secret inside the extraEnvs section of the Backstage CR to complete container provisioning:

   apiVersion: rhdh.redhat.com/v1alpha5
   kind: Backstage
   metadata:
     name: lightspeed-rhdh
   spec:
     application:
       extraEnvs:
         secrets:
           - name: lightspeed-auth-secrets
             containers:
               - lightspeed-core

2. Optional: To protect settings such as Model Context Protocol (MCP) server additions from being overwritten during reconciliation loops, define a custom ConfigMap mapping in the extraFiles section of the CR:

       extraFiles:
         configMaps:
           - name: "my-custom-config"
             mountPath: /app-root
             key: lightspeed-stack.yaml
             containers:
               - lightspeed-core

3. Configure access rights by updating the RBAC policy inside your Backstage CR:

   To grant non-administrator teams access to the virtual assistant, append permission lines to the rbac-policies.csv section, replacing <team> with your target team name:

1. Apply the updated custom resource manifest to your cluster:

   oc apply -f <backstage_cr_file>.yaml

Verification

1. Log in to your console instance.
2. Verify that the Open Lightspeed floating action button (FAB) appears on the home page.
3. Select the FAB and confirm that the chat window initializes successfully.

Procedure

1. Optional: To disable the chat interface, update the global.lightspeed.enabled parameter to false in your Helm values.yaml file:

   global:
     lightspeed:
       enabled: false

2. Create a manual Kubernetes Secret to store your provider credentials.

   Note

   By default, the Helm installation creates a temporary Kubernetes Secret containing keys for various LLM providers. On subsequent helm upgrade cycles, the system overwrites this default Secret. You must create a manual Kubernetes Secret to persist your credentials.

   Add the required keys to your secret based on your provider requirements:

   Important

   To disable an inference provider or configuration feature, you must leave the corresponding ENABLE_* variable completely unset. Setting an ENABLE_* variable to false does not disable the component because the underlying system checks only whether the variable is defined.

1. Optional: To filter and reject off-topic user queries, add validation guardrails to your manual Secret:

   • Set ENABLE_VALIDATION to "true".
   • Set VALIDATION_PROVIDER to your enabled validation provider (for example, openai).
   • Set VALIDATION_MODEL_NAME to your specific verification model (for example, gpt-4o-mini).

2. Reference your manual secret inside the values.yaml file:

   global:
     lightspeed:
       secret:
         create: false
         name: "my-custom-secret"

3. Optional: To protect configuration files like lightspeed-stack.yaml, config.yaml, or rhdh-profile.py from being overwritten during updates, you must use a custom ConfigMap:

   Important

   You must only modify the create and nameOverride fields. Keep the default mount paths and file configurations unchanged.

   global:
     lightspeed:
       configMaps:
         - name: stack
           create: false
           nameOverride: "my-custom-stack"
           mountPath: /app-root/lightspeed-stack.yaml
           subPath: lightspeed-stack.yaml
           sourceFile: lightspeed-stack.yaml
           optional: false

4. Configure access rights by updating your RBAC definitions:

   To grant non-administrator teams access to the virtual assistant, append permission lines to the rbac-policies.csv section, replacing <team> with your target team name:

1. Run the helm upgrade command to apply your configurations to the cluster.

Verification

1. Log in to your console instance.
2. Verify that the Open Lightspeed floating action button (FAB) appears on the home page.
3. Select the FAB and confirm that the chat window initializes successfully.

Procedure

1. Extract or identify the image digests for the Developer Lightspeed for RHDH sidecar and initialization container images.

2. Mirror the images directly to your internal mirror registry by running the skopeo copy command:

Procedure

1. Open your custom configuration file, such as lightspeed-stack.yaml.

2. Modify the user_data_collection block to configure your data preferences:

   • To enable feedback collection, set the feedback_enabled parameter to true:

     user_data_collection:
       feedback_enabled: true
       feedback_storage: "/tmp/data/feedback"
       transcripts_enabled: true
       transcripts_storage: "/tmp/data/transcripts"

   • To disable feedback collection, set the feedback_enabled parameter to false:

     user_data_collection:
       feedback_enabled: false
       feedback_storage: "/tmp/data/feedback"
       transcripts_enabled: true
       transcripts_storage: "/tmp/data/transcripts"

1. Apply the updated configuration file changes to your cluster by running your platform’s standard deployment or upgrade sequence.

Procedure

1. Open your RHDH configuration file, typically named app-config.yaml.

2. Add or modify the systemPrompt parameter under the lightspeed section, specifying your custom instruction string:

   lightspeed:
     # ... other lightspeed configurations
     systemPrompt: "You are a helpful assistant focused on Red Hat Developer Hub development."

3. Save the file.
4. Restart the RHDH service to apply the updated system prompt configuration.

Procedure

1. Open your custom configuration file, typically named lightspeed-stack.yaml.

2. Modify the conversation_cache block to specify your storage configuration:

   • To enable persistent storage, add your PostgreSQL database credentials and endpoint properties:

     conversation_cache:
       type: "postgres"
       postgres:
         host: _<your_database_host>_
         port: _<your_database_port>_
         db: _<your_database_name>_
         user: _<your_user_name>"_
         password: _<postgres_password>_

     • To retain the default non-persistent SQLite setup, verify that the parameters match the following paths:

       conversation_cache:
         type: "sqlite"
         sqlite:
           db_path: '/tmp/cache.db'

3. Restart the LCORE service to apply your new database configuration.

Procedure

1. Enable the notebook feature and define your model by adding the following configuration to your app-config.yaml file:

   lightspeed:
     notebooks:
       enabled: true
       queryDefaults:
         model: ${NOTEBOOKS_QUERY_MODEL} # Use the exact model name
         provider_id: ${NOTEBOOKS_QUERY_PROVIDER_ID}

   Note

   If the model name is incorrect, an error message appears in the logs and the user interface.

2. Grant user access through role-based access control (RBAC) policies by defining permissions in your rbac-policy-csv file:

   1. Add the permission policy:

      p, role:default/_<your_team_name>_, lightspeed.notebooks.use, update, allow

   2. Assign the role to specific users:

      g, user:default/_<your_user_name>_, role:default/_<your_team_name>_

3. Apply the updated configuration and restart the service.

Verification

1. Log in to RHDH using an account assigned to the RBAC role defined in the configuration.
2. Confirm that the Notebooks tab is visible next to the Chat tab in the primary navigation bar.
3. Click the Notebooks tab and ensure the My Notebooks dashboard loads without error messages.

Procedure

1. Enable the notebook feature and define your model by adding the following configuration to your app-config.yaml file:

   lightspeed:
     notebooks:
       enabled: true
       queryDefaults:
         model: ${NOTEBOOKS_QUERY_MODEL} # Use the exact model name
         provider_id: ${NOTEBOOKS_QUERY_PROVIDER_ID}

   Note

   If the model name is incorrect, an error message appears in the logs and the user interface.

2. Grant user access through role-based access control (RBAC) policies by defining permissions in your rbac-policy-csv file:

   1. Add the permission policy:

      p, role:default/_<your_team_name>_, lightspeed.notebooks.use, update, allow

   2. Assign the role to specific users:

      g, user:default/_<your_user_name>_, role:default/_<your_team_name>_

3. Apply the updated configuration and restart the service.

Verification

1. Log in to RHDH using an account assigned to the RBAC role defined in the configuration.
2. Confirm that the Notebooks tab is visible next to the Chat tab in the primary navigation bar.
3. Click the Notebooks tab and ensure the My Notebooks dashboard loads without error messages.

Procedure

1. Update your custom config map llama-stack-configs/config.yaml file to point the kv_notebooks storage backend to your persistent mount point:

   spec:
     initContainers:
       - name: init-notebooks-dir
         # ... complete init container
     containers:
       - name: lightspeed-core
         image: quay.io/lightspeed-core/lightspeed-stack:0.5.1
         ports:
           - containerPort: 8080
         volumeMounts:
           - name: notebooks-storage
             mountPath: /var/lib/lightspeed-data
           - name: config  # ← Added all ConfigMap mounts
             mountPath: /app-root/config.yaml
             subPath: config.yaml
           - name: lightspeed-config
             mountPath: /app-root/lightspeed-stack.yaml
             subPath: lightspeed-stack.yaml
           - name: profile
             mountPath: /app-root/rhdh-profile.py
             subPath: rhdh-profile.py
         livenessProbe:  # ← Added health checks
           httpGet:
             path: /readiness
             port: 8080
         readinessProbe:
           httpGet:
             path: /readiness
             port: 8080
     volumes:  # ← Added all volume definitions
       - name: notebooks-storage
         persistentVolumeClaim:
           claimName: lightspeed-notebooks-pvc
       - name: config
         configMap:
           name: llama-stack-config
       - name: lightspeed-config
         configMap:
           name: lightspeed-core-config
       - name: profile
         configMap:
           name: rhdh-profile

2. Update your deployment manifest to include the init container, volume mounts, and volume definitions:

   spec:
     template:
       spec:
         initContainers:
           - name: init-notebooks-storage
             image: registry.access.redhat.com/ubi9/ubi-minimal
             command: ["sh", "-c", "mkdir -p /var/lib/lightspeed-data/notebooks && chmod -R 777 /var/lib/lightspeed-data/notebooks"]
             volumeMounts:
               - name: lightspeed-notebooks
                 mountPath: /var/lib/lightspeed-data
         containers:
           - name: lightspeed-stack
             image: quay.io/lightspeed-core/lightspeed-stack:0.5.1
             ports:
               - containerPort: 8080
             volumeMounts:
               - name: lightspeed-notebooks
                 mountPath: /var/lib/lightspeed-data
               - name: config
                 mountPath: /app-root/config.yaml
                 subPath: config.yaml
             livenessProbe:
               httpGet:
                 path: /readiness
                 port: 8080
             readinessProbe:
               httpGet:
                 path: /readiness
                 port: 8080
         volumes:
           - name: lightspeed-notebooks
             persistentVolumeClaim:
               claimName: lightspeed-notebooks-pvc
           - name: config
             configMap:
               name: llama-stack-config

3. Apply the updated configuration and restart the service.

Verification

1. In Red Hat Developer Hub, create a Notebook and upload a test document.
2. Send a message to the virtual assistant and verify that the response is based on the document.

3. Restart the pod:

   $ oc delete pod <pod_name>

4. After the pod recovers, refresh the My Notebooks dashboard.
5. Verify that the Notebook and the uploaded file are still accessible.

Procedure

1. Add the following YAML file as a config map to your namespace:

   version: 2
   image_name: redhat-ai-dev-llama-stack-no-guard
   apis:
     - agents
     - inference
     - safety
     - tool_runtime
     - vector_io
     - files
   container_image:
   external_providers_dir:
   providers:
     agents:
       - config:
           persistence:
             agent_state:
               namespace: agents
               backend: kv_default
             responses:
               table_name: responses
               backend: sql_default
         provider_id: meta-reference
         provider_type: inline::meta-reference
     inference:
       - provider_id: ${env.ENABLE_VLLM:+vllm}
         provider_type: remote::vllm
         config:
           url: ${env.VLLM_URL:=}
           api_token: ${env.VLLM_API_KEY:=}
           max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
           tls_verify: ${env.VLLM_TLS_VERIFY:=true}
       - provider_id: ${env.ENABLE_OLLAMA:+ollama}
         provider_type: remote::ollama
         config:
           url: ${env.OLLAMA_URL:=http://localhost:11434}
       - provider_id: ${env.ENABLE_OPENAI:+openai}
         provider_type: remote::openai
         config:
           api_key: ${env.OPENAI_API_KEY:=}
       - provider_id: ${env.ENABLE_VERTEX_AI:+vertexai}
         provider_type: remote::vertexai
         config:
           project: ${env.VERTEX_AI_PROJECT:=}
           location: ${env.VERTEX_AI_LOCATION:=us-central1}
       - provider_id: sentence-transformers
         provider_type: inline::sentence-transformers
         config: {}
     tool_runtime:
       - provider_id: model-context-protocol
         provider_type: remote::model-context-protocol
         config: {}
       - provider_id: rag-runtime
         provider_type: inline::rag-runtime
         config: {}
     vector_io:
       - provider_id: faiss
         provider_type: inline::faiss
         config:
           persistence:
             namespace: vector_io::faiss
             backend: faiss_kv
     files:
       - provider_id: localfs
         provider_type: inline::localfs
         config:
           storage_dir: /tmp/llama-stack-files
           metadata_store:
             table_name: files_metadata
             backend: sql_files
   storage:
     backends:
       kv_default:
         type: kv_sqlite
         db_path: /tmp/kvstore.db
       sql_default:
         type: sql_sqlite
         db_path: /tmp/sql_store.db
       sql_files:
         type: sql_sqlite
         db_path: /rag-content/vector_db/rhdh_product_docs/1.9/files_metadata.db
       faiss_kv:
         type: kv_sqlite
         db_path: /rag-content/vector_db/rhdh_product_docs/1.9/faiss_store.db
     stores:
       metadata:
         namespace: registry
         backend: faiss_kv
       inference:
         table_name: inference_store
         backend: sql_default
         max_write_queue_size: 10000
         num_writers: 4
       conversations:
         table_name: openai_conversations
         backend: sql_default
   registered_resources:
     models:
       - model_id: sentence-transformers/all-mpnet-base-v2
         metadata:
           embedding_dimension: 768
         model_type: embedding
         provider_id: sentence-transformers
         provider_model_id: /rag-content/embeddings_model
     tool_groups:
       - provider_id: rag-runtime
         toolgroup_id: builtin::rag
     vector_dbs:
       - vector_db_id: rhdh-product-docs-1_8
         embedding_model: sentence-transformers/all-mpnet-base-v2
         embedding_dimension: 768
         provider_id: faiss
   server:
     auth:
     host:
     port: 8321
     quota:
     tls_cafile:
     tls_certfile:
     tls_keyfile:

2. Mount the config map to your Llama Stack container at /app-root/run.yaml to make sure it overrides the default image file:

   name: llama-stack
   volumeMounts:
   - mountPath: /app-root/run.yaml
     subPath: run.yaml
     name: llama-stack-config

3. Configure the required volume:

   volumes:
   - name: llama-stack-config
     configMap:
       name: llama-stack-config

   where:

   llama-stack-config

   The config map where you added the new no-guard configuration file.

4. Restart the deployment if it does not trigger an automatic rollout.

Procedure

1. Click the Open Lightspeed floating action button (FAB) at the lower right of the screen to open the chat overlay.

2. Optional: Configure the interface display and server settings:

   • Click the Chatbot options icon (⋮) to view chat history or start a new chat.

   • Click the Display mode icon and select any of the following views:

     

     • Overlay: A floating window is displayed over the current page content.

       

     • Dock to window: A panel attaches to the right side of the screen. Activating this mode automatically closes the quick start panel if it is already open.

       

     • Fullscreen: A dedicated page opens for intensive chat sessions. You can bookmark this URL for direct access.

       
     • Optional: Toggle Enable pinned chats/Disable pinned chats to enable or hide the pinned chats. The system enables this option by default.
     • Available only if MCP is configured: MCP settings: Manage Model Context Protocol connections.

3. Start a chat or load an earlier session:

   • Enter a prompt: Type a query in the Enter a prompt for Lightspeed chat field and press Enter.
   • Use a sample: Click a prompt tile.

   • Attach a file: Click Attach to upload a .yaml, .json, or .txt file.

     1. Click the file name to open the preview model.

     2. View or edit the content of the file:

        
   • Use voice: Click the Use microphone icon.
   • Resume a chat: Select a title from the Recent list.

4. Organize your chat history:

   • Start a new topic: Click New chat to reset the assistant’s context.
   • Search history: Enter a keyword in the Search field.

   • Rename a session: Click Options next to a chat title, select Rename, and enter a new name.

     
   • Pin a chat:: Click Options next to a chat title and select Pin. The chat moves to the Pinned group.

   • Sort chats:: Click Sort control and choose a sorting criteria, such as Date (Newest first).

     

   • Delete a chat: Click Options next to a chat title and select Delete.

     
   • Hide the chat history section and reduce visual noise: Select the Close icon (x) next to New chat.
   • Restore access to your pinned chat: Select the Chat history menu icon.
5. Optional: To hide the interface, if you are in the Overlay or Dock to window mode, click the Close Lightspeed icon (X) to hide the window. If you are in Fullscreen mode, revert to the other modes and click the Close Lightspeed icon (X). The system preserves your active query and history.
6. Optional: In Fullscreen mode, bookmark the URL in your browser to save a direct link to the chat interface.

Verification

1. The main window displays the active chat or selected history.
2. The chat history list reflects renamed, pinned, or deleted entries.

Procedure

1. Clone the evaluation repository and navigate to the directory:

   git clone https://github.com/lightspeed-core/lightspeed-evaluation
   cd lightspeed-evaluation

2. Synchronize the environment and install dependencies:

   uv sync

3. Configure the environment variables for the judge LLM. You can create a .env file in the root directory or export the keys directly to your terminal.

   • If you use Gemini, you must set the Gemini API key:

     export GEMINI_API_KEY="your-google-api-key"

   • If you use OpenAI, you must set the OpenAI API key:

     export OPENAI_API_KEY="your-key"

4. Optional: If you test with a live service, set your Developer Lightspeed for RHDH service API key:

   export API_KEY="your-lightspeed-service-key"

Procedure

1. Download pre-generated datasets: Use this method to test the performance of specific RHDH releases. These datasets are generated using Ragas testset generation for RAG.

   1. In your terminal, navigate to the /dataset folder in the evaluation repository.
   2. Locate the .evaluation_dataset_yaml files. These files are pre-configured for the evaluation tool.

   3. To test a historical release, switch to the corresponding branch.

      For example, to access the Red Hat Developer Hub 1.8 dataset, switch to the 1.8 branch.

      Important

      The main branch contains work-in-progress (WIP) datasets. Avoid using this branch for stable evaluations.

2. Generate custom datasets: Use this method to create a new test set from your own technical documentation.

   1. Generate a diverse set of question-and-answer (Q&A) pairs by following the Ragas test data generation documentation.
   2. Ensure your Q&A pairs match the required format by reviewing the evaluation data structure configuration.

Procedure

1. Download the system.yaml configuration template from the repository.

2. Configure the parameters in the system.yaml file based on your evaluation mode:

   Field	Description
   llm	Defines the judge LLM that scores the responses, such as gemini-2.5-pro.
   api.enabled	Set to false for static mode to use pre-filled data. Set to true for dynamic mode to call a live service.
   api.api_base	(Required for dynamic mode only) Provide the URL of your Developer Lightspeed for RHDH service.
   api.endpoint_type	Specify the service configuration type: streaming or query.

3. Execute the evaluation by using the lightspeed-eval command:

   lightspeed-eval \
     --system-config config/system.yaml \
     --eval-data config/evaluation_data.yaml \
     --output-dir ./my_evaluation_results

Procedure

1. In the root of the repository, navigate to the version-specific folder within the /evaluation-result directory.

2. Open the following files to evaluate performance:

   • Model Pass Rate: Compare the overall performance between different LLMs.
   • Topic Pass Rate: Identify performance trends and gaps within specific documentation areas.