Creating and maintaining data contracts and their manifest files from Collibra
From Collibra, you can:
- Create and update a Data Contract.
- Generate a manifest file based on the Collibra content.
- Upload a new version of the manifest file.
- Download a manifest file.
- Delete a manifest file.
- Set a manifest file as the active one.
You can also use the Collibra Data Contracts CLI or APIs. For more information, go to Creating and maintaining data contracts and their manifest files through CLI and API
Steps overview
- Review the preflight checks.
- Create / manage the data contract.
- Verify the data contract information in Collibra.
You can do this by navigating to the Data Contract asset type. You can also get a global view on all the data contracts in an environment through the Data Contract Registry.
Review the preflight checks
Before you create and maintain data contracts, check the following:
- Your environment has been enabled for Data Products and Data Contracts.
- To initialize and access a Data Contract from the Output ports viewer widget, make sure that the widget is part of your Data Product asset page layout and that your Data Product is connected to a Data Product Port via the 'is output port' relation. A Data Product Port asset can have only one linked Data Contract asset.
- If you want the Data Contract to include a Schema section mentioning tables and columns when generating the Data Contract from Collibra, the Data Product Port must be linked to a physical Table, View, or Schema that contains one or more Tables or Views, and the Table or View should contain columns.
- You have the required permissions:
Action Required global permission Required resource permission Initialize a new Data Contract Product Rights > Data Contract - View permission on the Data Product Port asset.
- Asset > Attribute > Update on the Data Product Port asset.
- Asset > Attribute > Add on the Data Contract asset.
- Asset > Attribute > Update on the Data Contract asset.
When creating a new Data Contract asset:
- View permission on the domain.
- Asset > Add on the domain.
- Asset > Attribute > Add on the Data Product Port asset.
When adding or generating a manifest for an existing Data Contract without any manifests yet:
- View permission on the Data Contract asset.
Updating an existing Data Contract asset - View permission on the Data Contract asset.
- Asset > Attribute > Update on the Data Product Port asset.
- Asset > Attribute > Add on the Data Contract asset.
- Asset > Attribute > Remove on the Data Contract asset.
Upload a new Data Contract Manifest Product Rights > Data Contract - View permission on the Data Contract asset.
- Asset > Attribute > Add on the Data Contract asset.
- Asset > Attribute > Update on the Data Contract asset.
Generate a new Data Contract Manifest Product Rights > Data Contract - View permission on the Data Contract asset.
- Asset > Attribute > Add on the Data Contract asset.
- Asset > Attribute > Update on the Data Contract asset.
Set a specific Data Contract Manifest version active Product Rights > Data Contract - View permission on the Data Contract asset.
- Asset > Attribute > Update on the Data Contract asset.
- Asset > Attribute > Add on the Data Contract asset.
Delete a specific Data Contract Manifest Product Rights > Data Contract - View permission on the Data Contract asset.
- Asset > Attribute > Remove on the Data Contract asset.
Delete a Data Contract Product Rights > Data Contract - View permission on the Data Contract asset.
- Asset > Remove on the Data Contract asset.
Access the Data Contract Registry view Product Rights > Data Contract Only Data Contract assets for which you have View permission are shown. View a Data Contract View permission on the asset View a Data Contract Manifest View permission on the asset Download a Data Contract Manifest View permission on the asset Note Manifests created via the knowledge graph generation will include all Tables and Columns related to the Data Product Port, regardless of your personal View permissions on these assets.
Create a data contract
Prerequisites
You have reviewed the preflight checks.
Steps
- Navigate to the Data Product asset page.
- Scroll down to the section that shows the Output ports viewer widget.
- Click the card of the Data Product Port you want to add a data contract to.
The Preview panel opens showing tabs:- Data product port shows the attributes and schema information in the product port.
- Data contract shows the data contract details if available.
- Click the Data contract tab.
If no data contract has been initiated yet, buttons are shown which allow you to create a new contract. - Choose one of the following methods to create a data contract:
- Knowledge graph generation allows you to generate a data contract based on the available information in the Data Product, Data Product Port, and linked Tables.
- Click Generate a contract manifest.
The Generate manifest dialog box appears. - Complete the fields as follows:
Field Description Name The name you want to give to the Data Contract asset. Domain The domain in which you want to add the Data Contract. We advise to add it to the same domain as the Data Product. Initial version The version of the manifest. By default the version is set to 0.0.1. Manifest ID The Manifest ID field defines how you want to structure the ID of the manifest file in the Data Contract asset. You can select the following options: - Use data contract asset UUID as manifest ID
- Use custom manifest ID
Download manifest Select the Download manifest checkbox to download the generated manifest file. - Click Generate or Generate and download.
The name depends on your selection.
A new Data Contract asset and manifest file are created. The manifest file is automatically completed with the Tables, Columns, and Descriptions from the Data Product Port asset.
If you selected to download the file, the file is downloaded automatically to your computer.
- Click Generate a contract manifest.
- File Upload allows you to create a file outside Collibra and use this file as the manifest file for the Data Contract asset of the Data Product Port.
- Click Upload a contract manifest.
The Upload manifest dialog box appears. - Click Upload a file or drop it into the Upload a file section.
The file gets uploaded. - Complete the fields as follows:
Field Description Name The name you want to give to the Data Contract asset. Domain The domain in which you want to add the Data Contract. We advise to add it to the same domain as the Data Product. Initial version The version of the manifest. By default the version is set to 0.0.1. Manifest ID The Manifest ID field defines how you want to structure the ID of the manifest file in the Data Contract asset. The manifest ID defined in the uploaded file is automatically picked up.
- Click Upload.
A new Data Contract asset is created. The uploaded manifest file is automatically set to the latest version of the Data Contract.
- Click Upload a contract manifest.
- CLI allows you to get the correct command to use in the CLI to create a Data Data Contract and manifest file.
Click Copy the code and follow the instructions in the CLI information to start working on the data contract. For information, go to Creating and maintaining data contracts and their manifest files through CLI and API.
- Knowledge graph generation allows you to generate a data contract based on the available information in the Data Product, Data Product Port, and linked Tables.
Manage a data contract via Collibra
In Collibra, you can manage the Data Contract Manifest files from the Data Contract asset page. You can, for example, replace a manifest with a new version by downloading the existing version, updating it locally, and uploading it again.
Prerequisites
- You have reviewed the preflight checks.
- A data contract is available.
Upload a new data contract version
- Navigate to a Data Contract asset page and click Contract Manifest.
The active data contract manifest is shown. - From the More icon, select Upload a new version.
The Upload new manifest version dialog box appears. - Click Upload a file or drop the file into the upload area.
- Click Upload.
The new version is uploaded. Depending on your selections, the file is now the active version.
| Field | Description |
|---|---|
| New version to upload |
The version of the manifest. For a new manifest, by default, the version is set to 0.0.1. During the upload process, Collibra checks whether the provided version number already exists for the data contract. It doesn't look at the file name, nor the version number mentioned inside the file. If you don't force an overwrite through the Force overwrite if version already exists checkbox, Collibra shows an error message if the version already exists. |
| Set as active manifest version | Select the Set as active manifest version checkbox if you want the new file to become the active version. This means that the previous active version will be deactivated. |
| Force overwrite if version already exists |
Select the Force overwrite if version already exists checkbox if you want to upload this file version even though it already exists for the contract. Collibra checks the version number given in the dialog box. It doesn't look at the file name, nor the version number mentioned inside the file. |
Generate a new data contract version
Collibra can generate a new draft manifest file based on the latest synchronized metadata in Collibra.
- Navigate to a Data Contract asset page and click Contract Manifest.
The active data contract manifest is shown. - If needed, select another version from the drop-down list.
The selected data contract manifest acts as the base for the new draft. - From the More icon, select Generate a new version.
The Generate and upload a new version dialog box appears. - In New version, enter the version number that you want to give to the new draft.
- Click Generate.
Collibra retrieves the latest synchronized metadata and updates the relevant content in the base manifest file. Any existing content that is not synchronized by Collibra remains unchanged, ensuring that unsynchronized content is preserved.
The Review manifest step shows the new draft as code. - Review the draft content and make changes as needed.
You can also download the new draft at this point. - Once all is good, click Continue.
The Upload step is shown. - Complete the fields as needed:
Field Description New version to upload The version of the manifest.
For a new manifest, by default, the version is set to 0.0.1.
During the upload process, Collibra checks whether the provided version number already exists for the data contract. It doesn't look at the file name, nor the version number mentioned inside the file.
If you don't force an overwrite through the Force overwrite if version already exists checkbox, Collibra shows an error message if the version already exists.
Set as active manifest version Select the Set as active manifest version checkbox if you want the new file to become the active version. This means that the previous active version will be deactivated. Force overwrite if version already exists Select the Force overwrite if version already exists checkbox if you want to upload this file version even though it already exists for the contract.
Collibra checks the version number given in the dialog box. It doesn't look at the file name, nor the version number mentioned inside the file.
- Click Upload.
The new version is uploaded. Depending on your selections, the file is now the active version.
Make a data contract version active
- Navigate to a Data Contract asset page and click Contract Manifest.
The current, active data contract manifest is shown.
- Select the version you want to make active from the drop-down list.
- From the More icon, select Set as active.
The Set new active version dialog box appears informing you that this action will mean the existing active version becomes inactive. - Click Set as active.
The selected version becomes active. The active version is tagged with (Active) after the version.
Delete a data contract version
To delete a manifest file, select the version from the drop-down list and from the More icon select Delete.
You can't delete manifest files that are active.