Filtering in derived relations

When Collibra traverses a relation path in your derived relation type, it accepts every asset that it finds at each node (asset type). You can use filters to narrow the selection. Filters allow you to restrict which assets are accepted at a node. If a node has no filter, all compatible assets are accepted.

How filters work

A filter is associated with a relation path node, such as the head node, an intermediate node, or the tail node.

If a condition is added to any asset type (node) in the path, Collibra evaluates every asset that it encounters at that node.

• Asset matches: If the asset meets the filter conditions, the traversal continues to the next node. If the node is a tail node, the asset is added to the result set.
• Asset doesn't match: If the asset doesn't meet the filter conditions, it is ignored. It is neither returned as a derived relation result nor used to continue the path.
• Head node exclusion: If a filter excludes the head node, the derived relation returns no result. This prevents inconsistent results when navigating in the opposite direction.

Filters are defined in the derived relation type editor. After you save a derived relation type with filters, the filter conditions apply automatically each time Collibra resolves the derived relation. This ensures that your asset pages remain accurate without further manual updates.

Why use filters

Without filters, a relation path in a derived relation type accepts every asset of the compatible type at each node. This is usually what you want, but sometimes, your knowledge graph contains assets that are technically on the path but logically irrelevant to your goal.

You can use filters in the following scenarios:

• Exclude retired assets. A path passes through Column assets, but you want columns that are marked as "Missing from source" to be ignored because such columns are historical artifacts, no longer reflecting your actual data. You can then use the status filter.
• Focus on a specific asset type. A path uses the generic Business Asset relation, but you want only business terms to be included, not policies or data standards. You can then use the asset type filter.
• Filter by relation and attribute values. You want derived relations to surface relations to the PII data category or columns where a PII Boolean attribute set to True. You can then use the relation and attribute filters and combine the two using the OR operator.
• Restrict by location. You want to scope a path to assets that belong to a specific community or domain to keep results relevant to a particular business area. You can then use the community or domain filter.

Combining filter conditions

You can add multiple conditions to a single node filter and combine them using the AND and OR operators, in both the relation type builder and the JSON editor.

You can also use parentheses to group conditions together to create flexible, complex logic, such as ((A OR B) AND (C OR D)) OR E. Up to five levels of grouping (nested parentheses) is supported, for example, (((((A OR B) AND C) OR D) AND E) OR F).

Adding a filter using the relation type builder

The Relation type builder tab in the derived relation type editor provides a visual way to add filters to any node in your derived relation type path. You can't add or modify the filters on any out-of-the-box path nodes because they are system-protected.

1. On the main toolbar, click → Settings.
   The Settings page opens.
2. In the Operating model section, click Relation types.
3. On the Relation types page:
   • To create a new derived relation type, click Add derived relation type.
   • To edit a derived relation type, in the Actions column of the derived relation type, click → Edit.
4. In the derived relation type editor, on the Relation type builder tab, define your head, relation paths, and tail.
5. Click the filter icon next to the node that you want to filter. If it is an intermediate node, click → Add filter instead.
   
6. In the Add filter dialog box, in the Select a criterion field, select the asset property, attribute type, or relation type that should be evaluated (for example, Type, Status, or Category).
7. Select an operator. The available operators depend on the selected criterion.
8. If applicable, in the Select a Value field, specify one or more values.
   Tip 

   • To add more conditions to the same node, click Add filter, and to delete a filter condition, click .
   • To define how the conditions are combined, use the AND and OR operators in the Filter formula field.
   • To set a filter at the end of a relation path, add the filter to the tail node because the tail node is shared by all relation paths.
9. Click Apply filter.
10. In the derived relation type editor, click Save relation type.

Adding a filter using the JSON editor

You can also use the JSON tab in the derived relation type editor to add filters to any node in your derived relation type path. Each node object (that is, headNode, any entry in intermediateNodes, and tailNode) accepts an optional filter property. A filter object within the filter property supports the following properties.

{
  "name": "Column node",
  "filter": {
    "and": [
      {
        "statusId": { 
          "notIn": ["<Missing-from-source-ResourceID>"] 
        }
      },
      {
          "incomingRelations": {
          "typePublicId": "<RelationType-PublicID>",
          "any": {
            "in": ["<TargetAsset-UUID>"]
          }
          }
      }
    ]
  },
  "outgoingEdges": [ ... ]
}

Different filters support different operators. Building a filter using the relation type builder and switching to the JSON editor can help you learn the syntax. The JSON editor also highlights any unsupported operators.

Example: Filtering out retired columns

The following example shows how to use a status filter on an intermediate node. The goal is to create a derived relation type that links Database assets to Data Category assets through the standard catalog hierarchy, but ignoring Column assets whose status is Missing from source.

1. In the derived relation type editor, define the following path: Database → (has) Schema → (contains) Table → (contains) Column → (is categorized by) Data Category.
   
   
2. On the Column intermediate node, click → Add filter.
3. In the Add filter dialog box, enter the following information.

   Field	Description
   Select a criterion	Select Status.
   Operator	Select Does not equal.
   Select a value	Select Missing from source.

4. Click Apply filter.
5. In the derived relation type editor, click Save relation type.

When the derived relation is resolved, any columns whose status is Missing from source are ignored.

Available filters

The following tables contain all available filters, organized by the kind of property that they target. These filters are available on both the Relation type builder tab and the JSON tab.

Asset type and status filters

Asset type and status filters allow you to include or exclude assets based on their asset type or status.

The Asset Type filter resolves against the full type hierarchy. For example, if you filter on Data Asset, any asset whose type is a subtype of Data Asset (such as Table or Column) also matches.

Tag filters

Tag filters allow you to filter assets based on their tags.

Community and domain filters

Community and domain filters restrict the accepted assets to those that belong to specific communities or domains.

Date filters

Date filters allow you to filter intermediate or tail nodes based on the asset creation date or last modification date.

Attribute filters

Attribute filters evaluate one or more values of a specific attribute on the asset. They are available for all attribute kinds. Attributes of the Multiple Selection kind support only the Exists and Does not exist operators.

Relation filters

Relation filters check whether a specific explicit relation type exists on the asset at a given node. This allows you, for example, to accept only assets that have a specific relation established or exclude those that don't have that relation.

Integrity and maintenance

Because a filter references operating model elements such as asset types, statuses, attribute types, and relation types, deleting those elements can affect your derived relation types.

• Deleting operating model types: If you try to delete an asset type, status, attribute type, or relation type that is referenced in a filter, Collibra blocks the deletion with a message. The message includes a link to the affected derived relation types so you can update the filters before retrying the deletion.
• Deleting other referenced resources: If you try to delete a domain, community, or other resource that is referenced in a filter, Collibra doesn't block the deletion, but the affected filter values are removed. When you open an affected derived relation type, the derived relation type editor notifies you so you can review and fix it.