Reference data is data that is used to classify, categorize, structure, or constrain other data. Typically, they are static or slowly changing over time, for example, units of measurement and country codes.
Reference data often includes a known code set, which consists of code values that rarely change. As the name suggests, reference data is designed to be referenced by various other data to create a standard vocabulary and structure across diverse systems and data sources. Some examples of reference data are country codes, language codes, product codes, and account identifiers.
The Reference Data product aims at a systematic approach to manage reference data, including code sets and code values. For example, you can define relations between Code Set assets and the Column assets for which they are the allowed values, or between Code Value assets and the Business Assets that they represent. Additionally, you can define complex mappings between them in order to enable crosswalks from one information system to another, considering the differences in the code sets through time.
With all the reference data gathered in a single place, you can build an organization-wide understanding of how your data is organized, classified, and collected.
Open Reference Data
You can open Reference Data if you have the Product Rights > Reference Data Manager global permission. To open Reference Data, on the main toolbar, click
>
Reference Data.
If the Enable landing pages for applications setting is enabled in your Collibra environment, you see a landing page for this application. This landing page is a preconfigured page to help you find information and resources faster. Admins can update this page via the Page Editor.
Reference Data tabs
Reference Data has the following tabs.
| Tab | Description |
|---|---|
| Code sets and values | Contains all Code Value and Code Set assets. |
| Hierarchies | Contains all Hierarchies domains. |
Reference Data lifecycle
Reference data is relatively easy to govern because it concerns predictable data. Often, the code sets are related to the assets in Business Glossary. The process of managing reference data in Collibra generally involves the following phases.
0.1. Create
Gather all existing reference data content, analyze it, and enter the relevant parts in Collibra as Code Set and Code Value assets. We recommend that you use a specific Codelist domain for each code set.
You can create the assets manually, but it is usually easier to use the import feature to add many assets at once.
To describe the code set completely, you can create relations between the Code Set and Code Value assets, as well as other relevant assets.
|
Relation type |
Head assets |
Tail assets |
Description |
|---|---|---|---|
|
Code Value is part of / contains Code Set |
Code Value assets |
Code Set asset |
Relations of this type link the Code Value assets to the corresponding Code Set asset. |
|
Business Term has code / is code for Code Value |
Business Term asset |
Code Value asset |
Relations of this type link Business Term assets to Code Value assets to provide more information about the meaning of the Code Value asset. |
|
Data Element allowed value set / applies to Code Set |
Column asset |
Code Set asset |
Relations of this type describe which code set is used to restrict the possible values of a column. |
|
Data Element allowed value / allowed value for Code Value |
Column asset |
Code Value asset |
Relations of this type describe the actual code values that are used in a column. |
The outcome consists of Code Set and Code Value assets, organized in different Codelist domains. The assets can have relations to other assets and still have the Candidate status.
0.2. Complete
Create responsibilities by assigning users or user groups to roles for the respective Codelist domains:
- Data stewards improve the bulk import and prepare it for review. They also hold the ultimate decision-making authority in the approval process.
- Subject Matter Experts review the correctness of the assets.
- Stakeholders comment on the assets and validate the correctness.
Use the Approval and Simple Approval workflows to update and approve the Code Set and Code Value assets. The outcome consists of Code Set and Code Value assets with the Approved status.
0.3. Map
Data stewards map code values and crosswalks between corresponding Code Value assets. A Crosswalk asset may have additional attributes to describe the transformation logic. Often, this transformation logic is hidden or implicit.
Initially, the Crosswalk assets have Candidate status. Therefore, they should also be reviewed and approved through the Approval and Simple Approval workflows.
0.4. Publish and trace
After you have created the required assets and added the required relations, you can use diagrams to trace the lineage. The approved code values can also be provided to business users in different ways:
- You can export them to an XLSX or CSV file by using Collibra workflows. However, this file will be attached to a community or domain within Collibra.
- You can use the Collibra APIs to pull information from Collibra through external orchestrators, ETL tools, or programming languages.
To indicate that the code sets are published, you can create a status, such as Published.
0.5. Use and maintain
Finally, business users use the published code sets in their own applications, for example, in reporting software. Typically, there will be inconsistencies or incompleteness in the code sets. These issues can be reported, which starts a workflow to fix the issue.