Working with the Template Library page

Rule templates simplify rule authoring by providing a centralized library of reusable logic. You can create reusable rules to minimize the manual effort of applying similar data validation logic across multiple data sets. You can also export rule templates and import them into another environment to promote them from a lower environment, such as non-production, to a higher environment, such as production.

Prerequisites

You have one of the following global permissions based on the actions you want to perform:

Action Permission requirement
View a rule template

Data Quality > Deploy Templates > Viewer
Create and edit a rule template Data Quality > Manage Templates > Creator
Export a rule template Data Quality > Manage Templates > Creator
Import a rule template Data Quality > Manage Templates > Creator

Adding a rule template

  1. On the main toolbar, click Products icon → Data Quality.
    The Monitoring Overview page opens to the available data sources in your environment, showing their high-level metadata.
  2. In the sidebar navigation on the left, click Template library.
    The template library opens.
  3. If necessary, filter the rule templates in the table using the filter options.
    ToleranceDescription
    Dimensions

    The type of issue to filter by, including Accuracy, Completeness, Consistency, Integrity, Validity, and Duplication.

    NameThe name of the rule template.
    DescriptionThe description of the rule template.
    Created byThe creator of the rule template. For out-of-the-box rule templates, this value is "Collibra".
    Search iconSearchSearch for a specific rule template.
    Filter iconShow/hide filtersShow or hide the filter options.
    Clear allClear the filter selections to show all results.
  4. Click Plus iconTemplate.
  5.  Enter the Name of the rule template.
  6.  Select the Primary dialect for the rule template you are creating. Default value is "Spark".
  7. From the SQL editor in the SQL tab, enter your SQL query.
    Tip 
    • By default, you cannot edit the SELECT * FROM {{dq-job-name}} WHERE clause.
    • The column variable must be represented as {{column}}.
    • When you deploy the template, the column variable parameter is automatically updated based on the primary column.
    • Templates currently support only one column variable parameter per template.
  8. Optionally, click Align left icon Format to automatically format the SQL for improved readability.
  9. Click Validate SQL to validate the syntax of your query.
  10. Optionally, select any additional supported dialects.
    1. Click Beak down icon next to each dialect to view or modify the SQL.
    2. To add an unsupported dialect, click Add dialect manually and enter the dialog and SQL in the modal. Click Save.
    3. Select all the appropriate dialects and click Approve selected.
  11. On the Settings tab, enter the rule template details.
    SettingDescription
    Data quality dimensionAssociate a rule with an out-of-the-box (OOTB) or custom dimension.
    Tolerance

    Define the minimum percentage of rule breaks that must be found to affect rule scores and statuses, and for any notifications tied to the rule to send. When a rule filter is applied, the percentage of rule breaks is calculated based on the filtered rows.

    Enter a value between 0 and 100.

    Note The value 0 is equivalent to not setting a rule tolerance threshold. This means that any percentage of rule violations will be considered a breach.

    DescriptionThe description of the rule template.
    Related assetsThe business asset(s) related to the rule template. You can select one or multiple assets.
  12. Click Save icon Save.
    The rule template is created.

Editing a rule template

  1. Select an existing rule template in the template library.
    The rule template SQL and Settings display in read-only mode. The Used In tab shows the rules that use the template. The History tab shows the change history of the template.
  2. If necessary, select the Used In tab and click → Detach from template to remove the connection between the template and the rule.
  3. Click Edit icon Edit.
    The rule template details display in edit mode.
  4. Edit the details as necessary.
  5. Click Save icon Save.
    The changes to the rule template are saved. The changes automatically apply to rules created from the template.
  6. If necessary, in the template library, click → Delete to delete a rule template. You can't delete an OOTB rule template.

Viewing the history of a rule template

The History tab shows a read-only record of every configuration change to the rule template. Use it to find out who changed the rule template, when, and what changed.

Each entry shows the following information.

Item Description
Actor The user, API client, or system that made the change.
Timestamp The date and time of the change.
Change type The kind of change: Create, Updated, or Deleted.
Changes The fields that changed, with their previous and updated values.

To narrow the list, filter the entries by date range, user, or change type. To save the history, export it as a JSON file.

For a rule that existed before the history feature, the oldest entry is a system entry. Any changes made before the history feature was released aren’t tracked.

The history records changes to the following fields: name, description, SQL, dialect, dimensions, tolerance, related assets, and weight. A change that only affects formatting is recorded as a change. Dialect added, removed, or approved is recorded as a create or delete of that dialect.

Note You cannot edit OOTB rule templates. The History tab of an OOTB rule template shows a read-only message instead of history.

Exporting a rule template

Export one or more rule templates as a JSON file to reuse them in another Data Quality & Observability environment. You can then import the file into the other environment.

The exported JSON file contains the name, description, SQL, dimensions, dialect translations (the SQL and whether each dialect is supported and approved), and weight of each rule template. The file doesn't contain deployments, the Used In information, linked Business Rule assets, or the rules that were created from the template.

Note You can't export out-of-the-box rule templates.

  1. Select an existing rule template in the template library.
  2. To export one rule template, click → Export in the specific row. To export multiple rule templates, select the checkboxes of the rule templates, and then click the Export option.
  3. In the dialog box, click Export.
    The rule template(s) download as a JSON file.
  4. Import the exported JSON file to another environment.

Importing a rule template

Import rule templates from a JSON file that you exported from a Data Quality & Observability environment. Use this to promote rule templates from a lower environment, such as non-production, to a higher environment, such as production.

Note Collibra matches rule templates by name, not by ID. Imported rule templates get a new ID in your environment.

  1. In the template library, click Import.
    The import wizard opens.
  2. On the Upload step, drag and drop the JSON file or use the file picker to select it.
  3. Click Next.
  4. On the Map Dimensions step, review how the dimensions in the file map to dimensions in your environment. Dimensions that match a dimension in your environment by name, ignoring case, are mapped automatically. Dimensions without a match are set to Skip.
  5. For each dimension that you want to change, in the list, select a target dimension or Skip. Enter text in the list to filter the dimensions.

    Note If you set a dimension to Skip, Collibra removes that dimension from the imported rule templates, but still imports the rule templates.

  6. Click Next.
  7. On the Resolve Conflicts step, select how to handle rule templates that have the same name as an existing rule template in your environment:
    • Rename: Imports the rule template with the suffix that you enter in the Suffix field added to its name.
    • Overwrite: Updates the existing rule template with the imported one. When you overwrite a rule template, the change applies to every rule that is currently deployed from that template in your environment.
  8. Click Next.
  9. On the Review & Submit step, review the table. It shows the Source Name, Action, and Target Name of each rule template.
  10. Click Submit.
    A progress bar shows while the import runs in the background. When the import completes, the wizard closes and a notification appears. If the import is not completed, a dialog box explains what went wrong.
  11. Set up the imported rule templates in your environment, including the deployments, the Used In information, and the links to business rule assets.

What's next