Understanding and visualizing Red Hat Developer Hub project health using Scorecards
Setting up, configuring, and managing customizable Project Health Scorecards
Abstract
- 1. Understanding and visualizing Red Hat Developer Hub project health using Scorecards
- 1.1. Understand Scorecard plugins in Red Hat Developer Hub
- 1.2. Setting up Scorecards to monitor your Red Hat Developer Hub project health
- 1.3. Configuring Scorecards to view metrics in your Red Hat Developer Hub instance
- 1.4. Managing metric thresholds in your Scorecard plugin
- 1.5. Viewing Scorecard metrics in Red Hat Developer Hub
1. Understanding and visualizing Red Hat Developer Hub project health using Scorecards
1.1. Understand Scorecard plugins in Red Hat Developer Hub
This section describes Developer Preview features in the Red Hat Developer Lightspeed for Red Hat Developer Hub plugin. Developer Preview features are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Developer Preview features for production or business-critical workloads. Developer Preview features provide early access to functionality in advance of possible inclusion in a Red Hat product offering. Customers can use these features to test functionality and provide feedback during the development process. Developer Preview features might not have any documentation, are subject to change or removal at any time, and have received limited testing. Red Hat might provide ways to submit feedback on Developer Preview features without an associated SLA.
For more information about the support scope of Red Hat Developer Preview features, see Developer Preview Support Scope.
The Scorecard plugin offers comprehensive, customizable insights into the health, security posture, and compliance of your components, services, and APIs. It centralizes Key Performance Indicators (KPIs) within Red Hat Developer Hub (RHDH), so you do not have to consult multiple external systems.

1.1.1. Supported metric providers in Red Hat Developer Hub
The Scorecard plugin collects metrics from third-party data sources using metric providers. The Scorecard node plugin provides the scorecardMetricsExtensionPoint extension point that is used to connect your backend plugin module that exports custom metrics via metric providers to the Scorecard backend plugin.
The following metric providers are available:
| Provider | Metric ID | Title | Description | Type |
|---|---|---|---|---|
|
GitHub |
|
GitHub open PRs |
Count of open Pull Requests in GitHub |
number |
|
Jira |
|
Jira open issues |
The number of opened issues in Jira |
number |
1.2. Setting up Scorecards to monitor your Red Hat Developer Hub project health
1.2.1. Installing the Scorecard plugin in your Red Hat Developer Hub instance
You must manually install and enable the plugin in your Red Hat Developer Hub instance.
Prerequisites
- You have installed your RHDH instance.
- You have provisioned your custom dynamic plugins config map.
Procedure
Add the following configuration in your
Red Hat Developer Hub dynamic-plugin-config.yamlfile:plugins: - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard:bs_1.42.5__1.0.0!red-hat-developer-hub-backstage-plugin-scorecard disabled: false pluginConfig: dynamicPlugins: frontend: red-hat-developer-hub.backstage-plugin-scorecard: entityTabs: - path: '/scorecard' title: Scorecard mountPoint: entity.page.scorecard mountPoints: - mountPoint: entity.page.scorecard/cards importName: EntityScorecardContent config: layout: gridColumn: 1 / -1 if: allOf: - isKind: component - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard-backend:bs_1.42.5__1.0.0!red-hat-developer-hub-backstage-plugin-scorecard-backend disabled: false - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard-backend-module-github:bs_1.42.5__1.0.0!red-hat-developer-hub-backstage-plugin-scorecard-backend-module-github disabled: false - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira:bs_1.42.5__1.0.1!red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira disabled: false pluginConfig: jira: baseUrl: ${JIRA_BASE_URL} token: ${JIRA_TOKEN} product: datacenterwhere:
baseUrl- Enter the Jira URL.
token- Enter the Jira token.
1.2.2. Authenticating and managing Scorecard plugins to control who sees the metrics
As an administrator, you can secure the connection to your metric providers by managing user access to the Scorecards using the Role-Based Access Control (RBAC) permissions. For users to view the Scorecard metrics, you must provide users with read access to the Scorecard metrics. You can configure these permissions either by modifying your RBAC csv file directly or by using the RBAC Web UI.
Procedure
Set the required RBAC permissions using any of the following methods:
To use the csv file, in your csv file, you can directly add the following code:
g, user:default/<YOUR_USERNAME>, role:default/scorecard-viewer p, role:default/scorecard-viewer, scorecard.metric.read, read, allow p, role:default/scorecard-viewer, catalog.entity.read, read, allow
To use the RBAC Web UI, complete the following steps:
- In your Red Hat Developer Hub menu, go to Administration > RBAC.
- Select or create the Role to which you want to grant access to the Scorecards.
- To enable read access for the Scorecard plugin, in Add permission policies, select Scorecard from the plugins dropdown.
Expand Scorecard, select policy with the following details, and click Next:
-
Name:
scorecard.metric.read -
Permission:
read
-
Name:

1.3. Configuring Scorecards to view metrics in your Red Hat Developer Hub instance
You can configure one of the following Scorecards in your RHDH instance using metrics and scoring capabilities from various data sources for software components in the RHDH catalog:
- GitHub Scorecards
- Jira Scorecards
1.3.1. Configuring GitHub Scorecards to view GitHub metrics in your Red Hat Developer Hub instance
You can view GitHub metrics for software components registered in the RHDH catalog. To allow the RHDH to access the GitHub API, you must create a GitHub App and configure the necessary integration.
Prerequisites
- You have installed your RHDH instance.
- You have installed the Scorecard images.
- You must have permissions in GitHub to create and manage a GitHub App.
- You must have added a custom RHDH application configuration and have enough permissions to change it.
Procedure
Create and configure a GitHub App.
- Create the GitHub App with Read-only permissions for Contents to allow reading repositories.
- Generate a client secret and a private key.
- Install the app on the desired account or organization.
Save the following credential values:
-
App ID -
Client ID -
Client secret -
Private key
-
Add GitHub credentials to your RHDH secrets.
Set the following key/value pairs as environment variables in your secrets store:
-
GITHUB_INTEGRATION_APP_ID -
GITHUB_INTEGRATION_CLIENT_ID -
GITHUB_INTEGRATION_CLIENT_SECRET -
GITHUB_INTEGRATION_HOST_DOMAIN(e.g.,github.com) -
GITHUB_INTEGRATION_PRIVATE_KEY_FILE
-
Configure the GitHub integration in your
RHDH app-config.yaml.Add the
integrations.githubsection to your customRHDH app-config.yamlconfiguration file using the environment variables:integrations: github: - host: ${GITHUB_INTEGRATION_HOST_DOMAIN} apps: - appId: ${GITHUB_INTEGRATION_APP_ID} clientId: ${GITHUB_INTEGRATION_CLIENT_ID} clientSecret: ${GITHUB_INTEGRATION_CLIENT_SECRET} privateKey: | ${GITHUB_INTEGRATION_PRIVATE_KEY_FILE}Annotate the component entity in
catalog-info.yaml.Link a software component to the GitHub data source by adding the required annotations to its
catalog-info.yamlfile as shown in the following example:apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: my-service annotations: # Required: GitHub project slug in format "owner/repository" github.com/project-slug: myorg/my-service # Required: Entity source location backstage.io/source-location: url:https://github.com/myorg/my-service spec: type: service lifecycle: production owner: _<your_team_name>_Where:
annotations:github.com/project-slug-
The GitHub repository format, for example,
owner/repository. annotations:backstage.io/source-location-
The entity source location format, for example,
url:https://github.com/owner/repository. spec:owner- Your team name.
You must add the team entity to the Catalog to ensure the provided permissions are applicable.
(Optional) Configure metric threshold values.
To customize the thresholds for the GitHub Open Pull Requests (
github.open_prs) metric, add the following section to yourapp-config.yamlfile:scorecard: plugins: github: open_prs: thresholds: rules: - key: success expression: '<10' - key: warning expression: '10-50' - key: error expression: '>50'scorecard:plugins:github:open_prs:thresholds- Lists the default threshold values for the GitHub open PRs metric.
To read more information about how to customize the threshold values, see Thresholds in Scorecard plugins.
1.3.2. Configuring Jira Scorecards to view GitHub metrics in your Red Hat Developer Hub instance
To view project tracking data from Atlassian Jira in your RHDH instance, configure Jira Scorecards. The following Jira versions are supported:
- Jira Cloud: API version 3
- Jira Data Center: API version 2
Prerequisites
- You must have administrator privileges for Jira and RHDH.
- You have installed your RHDH instance.
- You have installed the Scorecard images.
- You must have added a custom RHDH application configuration and have enough permissions to change it.
Procedure
Create a Jira configuration token using one of the following methods, depending on your Jira product:
Jira Cloud: Create a personal token. You must then create a
Base64-encodedstring using the following plain text format:your-atlassian-email:your-jira-api-token.echo -n 'your-atlassian-email:your-jira-api-token' | base64
- Jira datacenter: Create and use a Personal Access Token (PAT) in your Jira datacenter account. For more information, see the Atlassian documentation.
Configure the Jira Scorecard plugin in your
RHDH dynamic-plugins-config.yamlusing either a direct setup or a proxy setup.Use a direct setup: Add the following code to your
RHDH dynamic-plugins-config.yaml:plugins: - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira:pr_1499__0.1.0!red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira disabled: false pluginConfig: jira: baseUrl: ${JIRA_BASE_URL} token: ${JIRA_TOKEN} product: datacenter # Change to 'cloud' if using Jira Cloudwhere:
baseUrl- Enter the Jira URL.
token- Enter the Jira token (Base64 string for Cloud, PAT for Data Center).
product-
Enter the supported product:
cloudordatacenter.
Use a proxy setup: In your
RHDH dynamic-plugins-config.yaml, add the following code:plugins: - package: oci://ghcr.io/redhat-developer/rhdh-plugin-export-overlays/red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira:pr_1499__0.1.0!red-hat-developer-hub-backstage-plugin-scorecard-backend-module-jira disabled: false pluginConfig: jira: proxyPath: /jira/api product: datacenter # Change to 'cloud' if using Jira CloudIn your
RHDH app-config.yamlfile, add the following proxy settings:proxy: endpoints: '/jira/api': target: ${JIRA_BASE_URL} headers: Accept: 'application/json' Content-Type: 'application/json' X-Atlassian-Token: 'no-check' Authorization: ${JIRA_TOKEN} # Must be configured in your environmentwhere:
AuthorizationSet any of the following header value:
-
For Cloud:
Basic YourCreatedAboveToken -
For Data Center:
Bearer YourJiraToken
-
For Cloud:
Annotate the component entity in the
catalog-info.yaml.In the
catalog-info.yamlfile for the entity you want to configure, link the component to the Jira data source by adding the necessary annotations:apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: my-service annotations: jira/project-key: PROJECT jira/component: Component jira/label: UI jira/team: 9d3ea319-fb5b-4621-9dab-05fe502283e jira/custom-filter: 'reporter = "abc@xyz.com" AND resolution is not EMPTY' spec: type: website lifecycle: experimental owner: guests system: examples providesApis: [example-grpc-api]where:
jira/project-key- Required: Enter the Jira project key.
jira/component- Optional: Enter the Jira component name.
jira/label- Optional: Enter the Jira label.
jira/team- Optional: Enter the Jira team ID (not the team title).
jira/custom-filter- Optional: Enter a custom Jira Query Language (JQL) filter.
(Optional) Configure Jira Open Issues metric thresholds.
To customize the thresholds for the Jira Open Issues (
jira.open_issues) metric, add the following to yourRHDH app-config.yamlfile:scorecard: plugins: jira: open_issues: thresholds: rules: - key: success expression: '<10' - key: warning expression: '10-50' - key: error expression: '>50'where
scorecard:plugins:jira:open_issues:thresholds- Lists the default threshold values for the Jira open Issues metric.
(Optional) Define global or custom mandatory filters.
You can apply global mandatory filters or define custom filters that entities can override. Add the following to your
RHDH app-config.yaml:scorecard: plugins: jira: open_issues: options: mandatoryFilter: Type = Task AND Resolution = Resolved customFilter: priority in ("Critical", "Blocker")where:
mandatoryFilter-
Optional: Replaces the default filter (
type = Bug and resolution = Unresolved). customFilter-
Optional: Specifies a global custom filter. The entity annotation
jira/custom-filteroverrides this value.
For more information about how to customize the threshold values, see Thresholds in Scorecard plugins.
1.4. Managing metric thresholds in your Scorecard plugin
The Scorecard plugin uses thresholds to translate raw metric values into visual health indicators (Success, Warning, or Error). Configuring these thresholds is the primary way platform engineers and developers define what constitutes a healthy component.
1.4.1. How Thresholds work
A threshold defines a condition or an expression that determines which visual category a metric value belongs to.
- Evaluation Order: Threshold rules are evaluated in the order they are defined. The first matching rule is applied to the metric value.
-
Allowed Categories: Only three keys are supported:
success,warning, anderror. - Best Practice: Always order rules from the most restrictive to the least restrictive to ensure logical assignment.
1.4.2. Supported Threshold expressions in Red Hat Developer Hub
Threshold expressions are highly flexible and support standard mathematical and logical operators based on the data type of the metric.
| Metric Type | Operator | Example Expression | Job Performed |
|---|---|---|---|
|
Number |
|
|
Assigns category if the value is greater than 40. |
|
Number |
|
|
Assigns category if the value is between 80 and 100 (inclusive). |
|
Boolean |
|
|
Assigns category if the value is exactly true. |
1.4.3. Threshold rules to metrics for your Scorecard plugin
Thresholds are merged based on a strict priority order, allowing you to set defaults while granting specific entities the ability to override rules.
| Priority | Configuration Method | Location | Job Performed |
|---|---|---|---|
|
1 (Highest) |
Entity Annotations |
|
Override specific rules for an individual component. |
|
2 (Medium) |
App Configuration |
|
Set global rules that override provider defaults. |
|
3 (Lowest) |
Provider Defaults |
Backend Plugin Code |
Baseline rules defined by the metric source developer. |
1.4.4. Set global standards for metric thresholds
To override the default thresholds set by a metric provider, you must update your RHDH app-config.yaml file under the configuration path of the specific metric.
The custom configuration completely replaces the provider’s defaults. If you omit a threshold category (such as success), that category is not assigned.
The following configuration defines a success threshold as 10 or fewer issues, which applies to all components using the jira.open_issues metric unless explicitly overridden by an entity annotation.
+
scorecard:
plugins:
jira:
open_issues:
thresholds:
rules:
- key: success
expression: '<10' # fewer than 10 open issues is Success
- key: warning
expression: '10-50' # Between 10 and 50 is a Warning
- key: error
expression: '>50' # More than 50 open issues is an Error1.4.5. Threshold rules to metrics for your Scorecard plugin
You can override individual threshold rules for a specific component by using annotations in its catalog-info.yaml file. These annotations are merged with the existing global rules defined in your RHDH app-config.yaml file.
Annotation Format Reference
The annotation key follows a strict structure: scorecard.io/{providerId}.thresholds.rules.{thresholdKey}: '{expression}'
+
| Element | Description | Example Annotation | Example Value |
|---|---|---|---|
|
|
Unique identifier of the metric |
|
|
|
|
The overridden category |
|
|
|
|
The new condition for the rule |
|
|
Example: Entity Annotation Override
The following example describes the entity overriding only the warning and error rules for the jira.open_issues metric. The success rule falls back to the value defined in the global RHDH app-config.yaml file.
+
# catalog-info.yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: critical-production-service
annotations:
# Overrides the global 'warning' rule (e.g., changing '11-50' to '10-15')
scorecard.io/jira.open_issues.thresholds.rules.warning: '10-15'
# Overrides the global 'error' rule (e.g., changing '>50' to '>15')
scorecard.io/jira.open_issues.thresholds.rules.error: '>15'
# The 'success' rule (<10) remains defined by the global app-config
spec:
type: service1.4.6. Best practices for threshold rules
Threshold rule evaluation is order-dependent. You must follow logical ordering to avoid unintended category assignments, as the system stops evaluation once a value matches the first rule.
1.4.6.1. Follow logical ordering
Rules must be sequenced logically. Order rules from the most strict (smallest range) to the least strict (largest range) or ensure all ranges are mutually exclusive.
1.4.6.1.1. Problematic Rule Order Example
If rules are ordered incorrectly, a less restrictive rule can prevent stricter rules from being evaluated:
-
warning: <50: Any value less than 50 triggers the warning rule and stops evaluation. -
success: <10: This rule is not evaluated because all values less than 10 already matched the preceding warning rule.
1.4.6.1.2. Correct Ordering Example
Order the rules from the most restrictive value range to the least restrictive to ensure logical flow.
+
# Correct Example: Order from most restrictive (Success) to least restrictive (Error)
rules:
- key: success
expression: '<10' # Only values below 10
- key: warning
expression: '10-50' # Only values between 10 and 50
- key: error
expression: '>50' # All remaining values above 501.5. Viewing Scorecard metrics in Red Hat Developer Hub
As a developer, you can view the GitHub and Jira metrics using the Scorecards in your RHDH instance.
Prerequisites
- You have installed your RHDH instance.
- You have either the GitHub or Jira Scorecard (or both) configured in your RHDH instance.
- You must have permissions to read the Scorecard metrics.
Procedure
- In your RHDH navigation menu, go to Catalog.
- Select the software component (catalog entity) that has Scorecard metrics configured.
- On the component Service page, go to the Scorecard tab.
- To view the desired metrics, select the corresponding tile.