Interacting with Red Hat Developer Lightspeed for Red Hat Developer Hub
Leverage Artificial Intelligence (AI)-driven expertise of the Red Hat Developer Lightspeed for Red Hat Developer Hub (Developer Lightspeed for RHDH) virtual assistant to help you use Red Hat Developer Hub (RHDH)
Abstract
- 1. Interacting with Red Hat Developer Lightspeed for Red Hat Developer Hub
- 1.1. About Developer Lightspeed
- 1.2. Supported architecture for Red Hat Developer Lightspeed for Red Hat Developer Hub
- 1.3. Retrieval Augmented Generation embeddings
- 1.4. Installing and configuring Red Hat Developer Lightspeed for Red Hat Developer Hub
- 1.5. Customizing Developer Lightspeed for RHDH
- 1.6. Using Developer Lightspeed for RHDH
- 1.7. Appendix: LLM requirements
- 1.8. Appendix About user data security
1. Interacting with Red Hat Developer Lightspeed for Red Hat Developer Hub
1.1. About Developer Lightspeed
This section describes Developer Preview features in the Red Hat Developer Lightspeed for Red Hat Developer Hub plugin. Developer Preview features are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Developer Preview features for production or business-critical workloads. Developer Preview features provide early access to functionality in advance of possible inclusion in a Red Hat product offering. Customers can use these features to test functionality and provide feedback during the development process. Developer Preview features might not have any documentation, are subject to change or removal at any time, and have received limited testing. Red Hat might provide ways to submit feedback on Developer Preview features without an associated SLA.
For more information about the support scope of Red Hat Developer Preview features, see Developer Preview Support Scope.
This early access program enables customers to share feedback on the user experience, features, capabilities, and any issues encountered. Your input helps ensure that Developer Lightspeed better meets your needs when it is officially released and made generally available.
Red Hat Developer Lightspeed for Red Hat Developer Hub (Developer Lightspeed for RHDH) is a virtual assistant powered by generative Artificial Intelligence (AI) designed for Red Hat Developer Hub(RHDH). The assistant offers in-depth insights into RHDH, including its wide range of capabilities. You can interact with this assistant to explore and learn more about RHDH in greater detail.
Developer Lightspeed for RHDH provides a natural language interface within the RHDH console, helping you easily find information about the product, understand its features, and get answers to your questions.
You can experience Developer Lightspeed for RHDH Developer Preview by installing the Developer Lightspeed for Red Hat Developer Hub plugin within an existing RHDH instance. Alternatively, if you prefer to test it locally first, you can try Developer Lightspeed for RHDH using RHDH Local.

1.2. Supported architecture for Red Hat Developer Lightspeed for Red Hat Developer Hub
Developer Lightspeed for RHDH is available as a plugin on all platforms that host RHDH, and it requires the use of Road-Core Service (RCS) as a sidecar container. For more information on the different available platforms to install RHDH, see Install.
Currently, the provided RCS image is built for x86 platforms. To use other platforms (for example, arm64), ensure that you enable emulation.
1.2.1. About Road-Core Service
The Road-Core Service (RCS) acts as an intermediary and service layer for interfacing with LLM providers. RCS handles LLM provider setup, authentication, and includes functionalities such as question validation, feedback, and Retrieval Augmented Generation (RAG). The Developer Lightspeed for RHDH plugin within RHDH communicates with the RCS sidecar to send prompts and receives responses from the configured LLM service. The RCS sidecar is used to centralize the LLM interaction logic and configuration alongside your RHDH instance.
Red Hat Developer Lightspeed for Red Hat Developer Hub is a Developer Preview release. You must manually deploy the Road-Core Service as a sidecar container, and then install the Developer Lightspeed for RHDH plugin on your RHDH instance.
1.3. Retrieval Augmented Generation embeddings
The Red Hat Developer Hub documentation set has been added to the Road-Core Service as a RAG embedding.
1.4. Installing and configuring Red Hat Developer Lightspeed for Red Hat Developer Hub
You must install and configure both the Developer Lightspeed for RHDH and the RCS sidecar container manually.
Prerequisites
- You are logged into your OpenShift Container Platform account.
You have an RHDH instance installed either of the following ways:
Procedure
Create the RCS ConfigMap.
- In the OpenShift Container Platform web console, go to your RHDH instance and select the ConfigMaps tab.
- Click Create ConfigMaps.
From the Create ConfigMap page, select the YAML view option in Configure via, and edit the file as shown in the following example:
kind: ConfigMap apiVersion: v1 metadata: name: rcsconfig namespace: _<namespace>_ # Enter your namespace (For example, `rhdh`) data: rcsconfig.yaml: | llm_providers: - name: dummy type: openai url: 'https://dummy.com' models: - name: dummymodel ols_config: reference_content: product_docs_index_path: "./vector_db/rhdh_product_docs/1.7" product_docs_index_id: rhdh-product-docs-1_7 embeddings_model_path: "./embeddings_model" conversation_cache: type: memory memory: max_entries: 1000 authentication_config: module: noop default_provider: dummy default_model: dummymodel query_validation_method: llm user_data_collection: feedback_disabled: false feedback_storage: /app-root/tmp/data/feedback dev_config: enable_dev_ui: false disable_auth: false disable_tls: true enable_system_prompt_override: true user_data_collector_config: ingress_url: 'https://example.ingress.com/upload' user_agent: example-agent
Optional: Configure the number of workers that scale the REST API by specifying the following example to the
ols_config.max_workers
parameter in the RCS ConfigMap.ols_config: max_workers: _<number_of_workers>_
- Click Create.
Create the Developer Lightspeed for RHDH ConfigMap.
NoteCreate a dedicated Developer Lightspeed for RHDH ConfigMap instead of adding an additional section to your existing RHDH custom application configuration file (for example,
lightspeed-app-config.yaml
). Creating two files prevents the entire RHDH ConfigMap from being loaded into RCS.- In the OpenShift Container Platform web console, go to your RHDH instance and select the ConfigMaps tab.
- Click Create ConfigMap.
From the Create ConfigMap page, select the YAML view option in Configure via, and add the following example:
kind: ConfigMap apiVersion: v1 metadata: name: lightspeed-app-config namespace: <__namespace__> # Enter your RHDH instance namespace data: app-config.yaml: |- backend: csp: upgrade-insecure-requests: false img-src: - "'self'" - "data:" - https://img.freepik.com - https://cdn.dribbble.com - https://avatars.githubusercontent.com # This is to load GitHub avatars in the UI script-src: - "'self'" - https://cdn.jsdelivr.net lightspeed: # REQUIRED: Configure LLM servers with OpenAI API compatibility servers: - id: ${LIGHTSPEED_SERVER_ID} url: ${LIGHTSPEED_SERVER_URL} token: ${LIGHTSPEED_SERVER_TOKEN} # OPTIONAL: Enable/disable question validation (default: true) # When enabled, restricts questions to RHDH-related topics for better security questionValidation: true # OPTIONAL: Custom users prompts displayed to users # If not provided, the plugin uses built-in default prompts prompts: - title: 'Getting Started with Red Hat Developer Hub' message: Can you guide me through the first steps to start using Developer Hub as a developer, like exploring the Software Catalog and adding my service? # OPTIONAL: Port for lightspeed service (default: 8080) # servicePort: ${LIGHTSPEED_SERVICE_PORT} # OPTIONAL: Override default RHDH system prompt # systemPrompt: "You are a helpful assistant focused on Red Hat Developer Hub development."
Create Developer Lightspeed for RHDH secret file.
- In the OpenShift Container Platform web console, go to Secrets.
- Click Create > Key/value secret.
In the Create key/value secret page, select the YAML view option in Configure via, and add the following example:
kind: Secret apiVersion: v1 metadata: name: lightspeed-secrets namespace: _<namespace>_ # Enter your rhdh instance namespace stringData: LLM_SERVER_ID: _<server_id>_ # Enter your server ID (for example, `ollama` or `granite`) LLM_SERVER_TOKEN: _<token>_ # Enter your server token value LLM_SERVER_URL: _<server_url>_ # Enter your server URL type: Opaque
- Click Create.
To your existing dynamic plugins ConfigMap (for example,
dynamic-plugins-rhdh.yaml
), add the Developer Lightspeed for RHDH plugin image as shown in the following example:includes: - dynamic-plugins.default.yaml plugins: - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-lightspeed:bs_1.39.1__0.5.7!red-hat-developer-hub-backstage-plugin-lightspeed disabled: false pluginConfig: lightspeed: # OPTIONAL: Custom users prompts displayed to users # If not provided, the plugin uses built-in default prompts prompts: - title: 'Getting Started with Red Hat Developer Hub' message: Can you guide me through the first steps to start using Developer Hub as a developer, like exploring the Software Catalog and adding my service? dynamicPlugins: frontend: red-hat-developer-hub.backstage-plugin-lightspeed: appIcons: - name: LightspeedIcon module: LightspeedPlugin importName: LightspeedIcon dynamicRoutes: - path: /lightspeed importName: LightspeedPage module: LightspeedPlugin menuItem: icon: LightspeedIcon text: Lightspeed - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-lightspeed-backend:bs_1.39.1__0.5.7!red-hat-developer-hub-backstage-plugin-lightspeed-backend disabled: false pluginConfig: lightspeed: # REQUIRED: Configure LLM servers with OpenAI API compatibility servers: - id: ${LLM_SERVER_ID} url: ${LLM_SERVER_URL} token: ${LLM_SERVER_TOKEN} # OPTIONAL: Port for lightspeed service (default: 8080) # servicePort: ${LIGHTSPEED_SERVICE_PORT}
Update your deployment configuration based on your installation method:
For an Operator-installed RHDH instance, update your Backstage custom resource (CR).
In the
spec.application.appConfig.configMaps
section, add the Developer Lightspeed for RHDH custom app configuration as shown in the following example:appConfig: configMaps: - name: lightspeed-app-config mountPath: /opt/app-root/src
Update the
extraVolumes
specification to include the RCS ConfigMap as shown in the following example:volumes: - configMap: name: rcsconfig name: rcsconfig
Update the
volumeMounts
specification to mount the RCS ConfigMap as shown in the following example:volumeMounts: - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig subPath: rcsconfig.yaml - mountPath: /app-root/config/app-config-rhdh.yaml name: lightspeed-app-config subPath: app-config.yaml
Add the Developer Lightspeed for RHDH Secret file as shown in the following example:
envFrom: - secretRef: name: lightspeed-secrets
In your
deployment.patch.spec.template.spec.containers.env
section, set the RCS environment variables as shown in the following example:- name: PROJECT value: rhdh - name: RCS_CONFIG_FILE value: /app-root/config/rcsconfig.yaml - name: RHDH_CONFIG_FILE value: /app-root/config/app-config-rhdh.yaml
NoteYour RHDH container is typically already present in your CR. You are adding the second container definition
road-core-sidecar
as the RCS sidecar.Click Save. The Pods are automatically restarted.
Example of a Backstage CR with the RCS container
apiVersion: rhdh.redhat.com/v1alpha3 kind: Backstage metadata: name: backstage namespace: _<namespace>_ # your RHDH instance namespace spec: application: appConfig: configMaps: # Adding the Developer Lightspeed custom app config file - name: lightspeed-app-config mountPath: /opt/app-root/src dynamicPluginsConfigMapName: dynamic-plugins-rhdh extraEnvs: # Adding the Developer Lightspeed secrets file secrets: - name: lightspeed-secrets replicas: 1 extraFiles: mounthPath: /opt/app-root/src replicas: 1 route: enabled: true database: enableLocalDb: true deployment: patch: spec: template: spec: containers: - env: - name: PROJECT value: rhdh # Mounting the RCS sidecar to your RHDH instance - name: RCS_CONFIG_FILE value: /app-root/config/rcsconfig.yaml # Your existing RHDH ConfigMap - name: RHDH_CONFIG_FILE value: /app-root/config/app-config-rhdh.yaml envFrom: - secretRef: name: lightspeed-secrets image: 'quay.io/redhat-ai-dev/road-core-service:rcs-06302025-rhdh-1.7' name: road-core-sidecar ports: - containerPort: 8080 name: rcs-backend protocol: TCP volumeMounts: # Mounting the RCS sidecar to your RHDH instance - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig subPath: rcsconfig.yaml # Mounting the Lightspeed app config file to your RCS container - mountPath: /app-root/config/app-config-rhdh.yaml name: lightspeed-app-config subPath: app-config.yaml volumes: - configMap: name: rcsconfig name: rcsconfig
For a Helm-installed RHDH instance, update your Helm chart.
Add your dynamic plugins configuration in the`global.dynamic` property as shown in the following example:
global: dynamic: includes: - dynamic-plugins.default.yaml plugins: - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-lightspeed:bs_1.39.1__0.5.7!red-hat-developer-hub-backstage-plugin-lightspeed disabled: false pluginConfig: lightspeed: # OPTIONAL: Custom users prompts displayed to users # If not provided, the plugin uses built-in default prompts prompts: - title: 'Getting Started with Red Hat Developer Hub' message: Can you guide me through the first steps to start using Developer Hub as a developer, like exploring the Software Catalog and adding my service? dynamicPlugins: frontend: red-hat-developer-hub.backstage-plugin-lightspeed: appIcons: - name: LightspeedIcon module: LightspeedPlugin importName: LightspeedIcon dynamicRoutes: - path: /lightspeed importName: LightspeedPage module: LightspeedPlugin menuItem: icon: LightspeedIcon text: Lightspeed - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-lightspeed-backend:bs_1.39.1__0.5.7!red-hat-developer-hub-backstage-plugin-lightspeed-backend disabled: false pluginConfig: lightspeed: # REQUIRED: Configure LLM servers with OpenAI API compatibility servers: - id: ${LLM_SERVER_ID} url: ${LLM_SERVER_URL} token: ${LLM_SERVER_TOKEN} # OPTIONAL: Port for lightspeed service (default: 8080) # servicePort: ${LIGHTSPEED_SERVICE_PORT}
Add your Developer Lightspeed for RHDH custom app config file as shown in the following example:
extraAppConfig: - configMapRef: lightspeed-app-config filename: app-config.yaml
Update the
extraVolumes
section to include the RCS ConfigMap as shown in the following example:extraVolumes: - configMap: name: rcsconfig name: rcsconfig
Update the
extraVolumeMounts
section to mount the RCS ConfigMap as shown in the following example:extraVolumeMounts: - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig
Add the Developer Lightspeed for RHDH Secret file as shown in the following example:
extraEnvVarsSecrets: - lightspeed-secrets
Add the RCS image as shown in the following example:
extraContainers: - env: - name: PROJECT value: rhdh - name: RCS_CONFIG_FILE value: /app-root/config/rcsconfig.yaml - name: RHDH_CONFIG_FILE value: /app-root/config/lightspeed-app-config.yaml envFrom: - secretRef: name: lightspeed-secrets image: 'quay.io/redhat-ai-dev/road-core-service:rcs-06302025-rhdh-1.7' name: road-core-sidecar ports: - containerPort: 8080 name: rcs-backend protocol: TCP volumeMounts: - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig subPath: rcsconfig.yaml - mountPath: /app-root/config/lightspeed-app-config.yaml name: lightspeed-app-config subPath: app-config.yaml
NoteYour RHDH container is typically already present in your Helm chart. You are adding the second container definition
road-core-sidecar
as the RCS sidecar.- Click Save.
Click Helm upgrade.
Example of a Helm chart with the RCS container
global: ... upstream: backstage: appConfig: ... args: ... extraAppConfig: - configMapRef: lightspeed-app-config filename: app-config.yaml extraContainers: - env: - name: PROJECT value: rhdh - name: RCS_CONFIG_FILE value: /app-root/config/rcsconfig.yaml - name: RHDH_CONFIG_FILE value: /app-root/config/lightspeed-app-config.yaml envFrom: - secretRef: name: lightspeed-secrets image: 'quay.io/redhat-ai-dev/road-core-service:rcs-06302025-rhdh-1.7' name: road-core-sidecar ports: - containerPort: 8080 name: rcs-backend protocol: TCP volumeMounts: - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig subPath: rcsconfig.yaml - mountPath: /app-root/config/lightspeed-app-config.yaml name: lightspeed-app-config subPath: lightspeed-app-config.yaml extraEnvVars: ... extraEnvVarsSecrets: - lightspeed-secrets extraVolumeMounts: - mountPath: /app-root/config/rcsconfig.yaml name: rcsconfig extraVolumes: - configMap: name: rcsconfig name: rcsconfig ... image: ... initContainers: ...
Define permissions and roles for your users who are not administrators by completing the following steps:
Configure the required RBAC permission by defining an
rbac-policies.csv
file as shown in the following example:p, role:default/_<your_team>_, lightspeed.chat.read, read, allow p, role:default/_<your_team>_, lightspeed.chat.create, create, allow p, role:default/_<your_team>_, lightspeed.chat.delete, delete, allow g, user:default/_<your_user>_, role:default/_<your_team>_
-
Upload your
rbac-policies.csv
andrbac-conditional-policies.yaml
files to anrbac-policies
config map in your OpenShift Container Platform project containing RHDH. Update your Backstage custom resource to mount in the RHDH filesystem your files from the
rbac-policies
ConfigMap:apiVersion: rhdh.redhat.com/v1alpha3 kind: Backstage spec: application: extraFiles: mountPath: /opt/app-root/src configMaps: - name: rbac-policies
For detailed information, see Managing authorizations by using external files.
Verification
- Log in to your RHDH instance.
- In your Red Hat Developer Hub navigation menu, you are able to see and access the Lightspeed menu item. Clicking this menu takes you to the Developer Lightspeed for RHDH screen.

1.5. Customizing Developer Lightspeed for RHDH
You can customize Developer Lightspeed for RHDH functionalities, such as, question validation, gathering feedback, and storing chat history in PostgreSQL.
1.5.1. Using Question Validation in Developer Lightspeed for RHDH
Developer Lightspeed for RHDH utilizes Question Validation to validate the query to check if it relates to Red Hat Developer Hub.
When a user asks a question that falls outside of these restricted topics, Developer Lightspeed for RHDH provides a general response to notify the user that the question is out of scope. If you want to disable the Question Validation feature to allow for broader questioning, you can set questionValidation
to false
in your Developer Lightspeed for RHDH app config file.
Procedure
To disable the Question Validation feature, in your Developer Lightspeed for RHDH
app-config.yaml
file, add the following example:lightspeed: # ... other lightspeed configurations questionValidation: false
1.5.2. Gathering feedback in Developer Lightspeed for RHDH
Feedback collection is an optional feature configured on the RCS. This feature gathers user feedback by providing thumbs-up/down ratings and text comments directly from the chat window. RCS gathers the feedback, along with the user’s query and the response of the model, and stores it as a JSON file within the local file system of the Pod for later collection and analysis by the platform administrator. This can be useful for assessing model performance and improving your users' experience. The collected feedback is stored in the cluster where RHDH and RCS are deployed, and as such, is only accessible by the platform administrators for that cluster. For users that intend to have their data removed, they must request their respective platform administrator to perform that action as Red Hat does not collect (or have access to) any of this data.
Procedure
To enable or disable feedback, in your RCS configuration file, add the following settings:
llm_providers: ....... ols_config: ...... user_data_collection: feedback_disabled: <true/false> feedback_storage: "/app-root/tmp/data/feedback"
1.5.3. Updating the system prompt in Developer Lightspeed for RHDH
You can override the default system prompt that Developer Lightspeed for RHDH uses to better frame queries to your LLM. Customizing the system prompt allows you to refine the context, personality, and instructions that the LLM receives, improving the relevance and accuracy of the responses it creates for your specific environment.
Procedure
To set a custom system prompt, in your Developer Lightspeed for RHDH app config file, add or modify the
lightspeed.systemPrompt
key and set its value to your preferred prompt string as shown in the following example:lightspeed: # ... other lightspeed configurations systemPrompt: "You are a helpful assistant focused on {product} development."
Set systemPrompt
to prefix all queries sent by Developer Lightspeed for RHDH to the LLM with this instruction, guiding the model to generate more tailored responses.
1.5.4. Customizing the chat history storage in Developer Lightspeed for RHDH
By default, the RCS service stores chat history using an in-memory database. This means that if you restart the Pod containing the server, the chat history is lost. You can manually configure Developer Lightspeed for RHDH to store the chat history persistently as a long-term backup with PostgreSQL by any of the following methods:
- RHDH Operator
RHDH Helm chart
WarningIf you configure Developer Lightspeed for RHDH to store chat history using PostgreSQL, prompts and responses are recorded and can be reviewed by your platform administrators. If any of your user’s chat history contains any private, sensitive, or confidential information, this might have data privacy and security implications that you need to assess. For users that wish to have their chat data removed, they must request their respective platform administrator to perform this action. Red Hat does not collect (or have access to) any of this chat history data.
Procedure
When you are using Developer Lightspeed for RHDH on an Operator-installed RHDH instance, in your RHDH instance ConfigMap, update the
conversation-cache
field as shown in the following example:conversation_cache: type: postgres postgres: host: _<your_database_host>_ port: _<your_database_port>_ dbname: _<your_database_name>_ user: _<your_user_name>_ password_path: postgres_password.txt ca_cert_path: postgres_cert.crt ssl_mode: "require"
When you are using Developer Lightspeed for RHDH on a Helm-installed RHDH instance, in your RHDH instance
values.yaml
file, update theconversation-cache
field as shown in the following example:conversation_cache: type: postgres postgres: host: _<your_database_host>_ port: _<your_database_port>_ dbname: _<your_database_name>_ user: _<your_user_name>_ password_path: postgres_password.txt ca_cert_path: postgres_cert.crt
1.6. Using Developer Lightspeed for RHDH
Red Hat Developer Lightspeed for Red Hat Developer Hub is designed to support you when performing various tasks during your development workflow.
With Question Validation
enabled, you can ask Developer Lightspeed for RHDH the following types of questions:
- “Tell me about Red Hat Developer Hub.”
- “What are the benefits of RHDH?”
- “Can I use RHDH on an OpenShift Container Platform?”
- “How do I install plugins on Red Hat Developer Hub?”
With Question Validation
disabled, the scope of prompts you can put to Developer Lightspeed for RHDH is much broader. This allows Developer Lightspeed for RHDH to support you in a much more varied range of work situations as described in the following examples:
- “Analyze this log for me…”
- “Suggest libraries and frameworks I can use to build Event Driven Architecture microservices.”
- “I’m not familiar with this language, so explain to me what this code snippet is doing…”
- “Create a Kubernetes deployment for this service…”
- “Create a test plan for the following scenarios and conditions…”
- “Create a Jira record that describes the following feature…”
- “Draft the end-user documentation describing how to use the following cli command…”
1.6.1. Using Developer Lightspeed for RHDH to start a chat for the first time
You can start a chat with Developer Lightspeed for RHDH for quick answers on a number of topics depending on your settings. You can manually start a chat with the Developer Lightspeed for RHDH or use the following sample prompts we have provided to help you get started:
- Getting Started with Backstage
- Deploy with Tekton
- Create an OpenShift Deployment
Prerequisites
- You have the Developer Lightspeed for RHDH plugin configured in your RHDH instance.
Procedure
- In your RHDH navigation menu, click Lightspeed.
You can start a chat in either of the following ways:
To manually start a chat, in the Send a message text box, you can do any of the following tasks:
- Type your query and press Enter.
To attach a file in the chat, click the Attach icon or drag and drop the file in your chat.
NoteThe following file types are supported:
yaml
,json
,txt
, andxml
.- Click Open.
- To start a chat using the existing prompts, in the Developer Lightspeed for RHDH virtual assistant interface, click any of the relevant prompt tiles.
1.6.2. Using Developer Lightspeed for RHDH to create new chats after the first chat
After you have started an initial chat with the Developer Lightspeed for RHDH, you can begin a chat on a new topic at any time. Even if you log out and log back in, your previous chats are still available in your chat history for you to view.
Prerequisites
- You have the Developer Lightspeed for RHDH plugin configured in your RHDH instance.
Procedure
- In your RHDH navigation menu, click Lightspeed.
- In the Developer Lightspeed for RHDH virtual assistant interface, click New chat.
1.6.3. Using Developer Lightspeed for RHDH to view chat history
Your chats with Developer Lightspeed for RHDH are automatically saved in your RHDH instance. You can easily revisit your chat history at any time, switch between chats, and revisit any previous chats. Each chat remains active, enabling you to go back to any of your previous chats and continue from where you left off.
Prerequisites
- You have the Developer Lightspeed for RHDH plugin configured in your RHDH instance.
Procedure
- In your RHDH navigation menu, click Lightspeed. Developer Lightspeed for RHDH opens with your previous chat.
In the Developer Lightspeed for RHDH virtual assistant interface, do any of the following tasks:
- Select a chat title to open and view the full chat.
- In Search previous chats…, enter the text that you want to find from the earlier chats.
1.6.4. Using Developer Lightspeed for RHDH to delete a chat
Prerequisites
- You have the Developer Lightspeed for RHDH plugin configured in your RHDH instance.
Procedure
- In your RHDH navigation menu, click Lightspeed. Developer Lightspeed for RHDH opens with your previous chat.
- In Developer Lightspeed for RHDH, select the vertical ellipsis for the chat title of the chat that you want to delete.
- Select Delete.
1.7. Appendix: LLM requirements
1.7.1. Large language model (LLM) requirements
Developer Lightspeed for RHDH follows a Bring Your Own Model approach. This model means that to function, Developer Lightspeed for RHDH requires access to a large language model (LLM) which you must provide. An LLM is a type of generative AI that interprets natural language and generates human-like text or audio responses. When an LLM is used as a virtual assistant, the LLM can interpret questions and provide answers in a conversational manner.
LLMs are usually provided by a service or server. Since Developer Lightspeed for RHDH does not provide an LLM for you, you must configure your preferred LLM provider during installation. You can use Developer Lightspeed for RHDH with a number of LLM providers that offer the OpenAI API interface including the following LLMS:
- OpenAI (cloud-based inference service)
- Red Hat OpenShift AI (enterprise model builder & inference server)
- Red Hat Enterprise Linux AI (enterprise inference server)
- Ollama (popular desktop inference server)
- vLLM (popular enterprise inference server)
1.7.2. OpenAI
OpenAI offers a range of generative AI models, such as GPT 5, which can be used to provide inference services for applications like Developer Lightspeed for RHDH.
To use OpenAI with Developer Lightspeed for RHDH, you need access to the OpenAI API platform. For more information, see the OpenAI developer platform documentation.
1.7.3. Ollama
Ollama is a powerful and easy-to-use open-source project that simplifies the process of running large language models (LLMs) locally on your computer. It provides a simple command-line interface for downloading, managing, and running a wide variety of open-source models, such as Llama 3, Mistral, and many others, all without requiring a dedicated server or cloud service. By abstracting away the complex setup and dependencies, Ollama makes it accessible for developers, researchers, and enthusiasts to experiment with, build on, and integrate state-of-the-art LLMs into their applications directly from their personal machines.
The open source Ollama server in container form provides a convenient local testbed for LLM models that is very accessible and easily controlled.
1.7.4. vLLM
vLLM is an open-source, high-throughput serving engine for large language models (LLMs) that significantly improves upon traditional serving systems. It achieves this by introducing several key optimizations to reduce memory usage and eliminate redundant computations. vLLM prominently increases the number of concurrent requests an LLM can handle, making it a powerful tool for deploying and scaling LLM-based applications.
1.8. Appendix About user data security
1.8.1. About data use
Developer Lightspeed for RHDH is a virtual assistant you interact with using natural language. Using the Developer Lightspeed for RHDH interface, you send chat messages that Developer Lightspeed for RHDH transforms and sends to the large language model (LLM) provider you have configured for your environment. These messages could potentially contain information provided by your users about themselves, your cluster, cluster resources, or other aspects of your business or working environment.
Developer Lightspeed for RHDH has limited capabilities to filter or redact the information you provide to the LLM. Do not enter information into Developer Lightspeed for RHDH that you do not want to send to the LLM provider. To remind end users not to share private or confidential information, Developer Lightspeed for RHDH begins each new chat with an 'Important' message asking them not to “include personal or sensitive information” in their chat messages.
1.8.2. About feedback collection
Developer Lightspeed for RHDH collects feedback from users who engage with the feedback feature in the virtual assistant interface. If a user submits feedback, the feedback score (thumbs up or down), text feedback (if entered), the user query, and the LLM provider response are stored locally in the file system of the Pod. Red Hat does not have access to the collected feedback data.
1.8.3. About Bring Your Own Model
Developer Lightspeed for RHDH does not provide its own inference services, but uses a Bring Your Own Model approach. This means that you can configure the Road-Core Service to talk to the inference server or service of your choice. This also means that you are responsible for ensuring that the configured service meets your particular company policies and legal requirements, including any applicable terms with the third-party model provider. The only technical requirements for inference services are:
- The service must conform to the OpenAI API specification.
- The service must be configured correctly following the installation and configuration instructions. There are many commercial and open source inference services that support the OpenAI API specification for chat completions. The cost, performance, and security of these services can differ and it is up to you to choose, through evaluation and testing, the inference service that best meets your company’s needs.
1.8.4. Your responsibility
All of the information your users share in their questions and responses with Developer Lightspeed for RHDH are shared with the LLM inference service you configured. You are responsible for ensuring compliance with your company’s policies regarding the sharing of data with your chosen inference service.