Interpreting and improving the AI trust score of an asset
The AI Trust Score husers evaluate the maturity, safety, and operational health of AI assets. As a data steward, you can consult the AI trust score to identify specific gaps and take steps to increase the maturity and trustworthiness of the asset.
By default, the AI Trust Score widget is included on the asset pages of the following AI-related assets:
- AI Agent
- AI Use Case
- AI Base Model
- AI Model Version
- All subtypes of these asset types
If the AI Trust Score widget does not appear on relevant asset pages, contact you Collibra administrator. Administrators can enable the AI trust score and edit various settings to ensure your AI trust scores reflect what matters most to your organization. For complete information, go to Edit the AI trust score settings.
Improve the score of AI Use Case assets
The following example image shows the AI Trust Score widget that is included on the asset pages of your AI assets.
For AI Use Case assets, the AI trust score is derived from the individual scores of 5 themes. The importance, or weight, of each theme in determining the overall score is specified in the AI Trust Score settings.
The following table describes how the scoring is derived for each of the 5 themes, and offers suggestions for improving the overall score for the AI use case.
| Theme | Scoring criteria | Suggestions for improving scores |
|---|---|---|
|
Data Assets
|
A relation exists between the AI Use case asset and at least one Data Asset.
|
If such a relation does not exist, you can manually add them. For complete information, go to Manually link AI-related assets. |
|
Documentation
|
|
If no Owner is assigned for the asset, you can assign one or more users or user groups. When assigning an Owner, referred to as a "responsibility", you typically assign the Owner for the community or domain level, and the asset then inherits the responsibility. For complete information, go to View and edit responsibilities. |
|
If no description exists for the asset, you can enter one in the Description field. | |
|
If such a relation does not exist, you can manually add them. For complete information, go to Manually link AI-related assets. | |
|
The contribution from this theme is derived as follows:
|
||
|
Lifecycle progress
|
The criterion for this theme is the advancement of the AI Use Case asset through its lifecycle, as determined in the Lifecycle tab in the global assignment of AI Use Case asset type. The score increases incrementally as the AI Use Case progresses through its lifecycle. For example, if 5 lifecycle statuses are configured for the AI Use Case asset type, the score increases by 20% for every successive stage reached. |
On an AI Use Case asset page, the Lifecycle tracker allows you to track the maturity and evolution of the AI use case. As a use case evolves, the Status button allows you to advance the use case to successive lifecycle stages. Lifecycle transitions should be driven by the maturity and readiness of the AI use case. Advancing stages prematurely to artificially inflate the AI trust score undermines the integrity of the governance framework. |
|
Linked Technology Assets
|
The criteria for this theme is met if a relation exists between the AI Use case asset and at least one Technology Asset. The contribution from this theme is the average trust score of all Technology Assets that are related to the AI use case. For example, if the AI use case has a relation to 3 Technology Assets, of which the AI trust scores are 20, 20, and 80, the contribution from this theme will be (20 + 20 + 80) / 3 = 40. |
Review the linked Technology Assets and apply the suggested actions outlined in this topic to improve their respective AI trust scores. |
|
Risk rating
|
The criterion for this theme is the value of the "overall risk rating" of the AI use case, as prompted in the out-of-the-box Risks and Safeguards assessment. The contribution from this theme is derived as follows:
|
Conduct a Risks and Safeguards assessment for the AI use case and ensure that a value - Low, Medium, or High - is selected for the "overall risk rating" question. |
Improve the score of AI-related Technology Assets
For AI-related Technology Assets, the AI trust score is derived from the individual scores of 3 themes. The importance, or weight, of each theme in determining the overall score is specified in the AI Trust Score settings.
The following table describes how the scoring is derived for each of the 3 themes, and offers suggestions for improving the overall score for the AI use case.
| Theme | Scoring criteria | Suggestions for improving scores |
|---|---|---|
|
Data integrity
|
|
If such a relation does not exist, you can manually add them. For complete information, go to Manually link AI-related assets. |
|
Review the related input data asset and check whether there is a value (True or False) for the "Personally Identifiable Information" attribute type. If there isn't, review the metadata in the asset for PII, and then enter a value based on your findings. | |
|
||
| Documentation |
Same criteria as described for AI Use Case assets, except that the relations-based criteria focuses on the link between the Technology Asset and an AI Use Case asset. |
|
|
Lifecycle progress
|
Same criteria as described for AI Use Case assets. | |