dbt Core source configuration

You can use the Source configuration field in the technical lineage for dbt Core capability to limit the data objects downloaded for processing and improve Collibra Data Lineage performance.

The value of the Source configuration field must be a valid block of JSON code, for example:

Copy

{
    "collibraSystemNames":{
        "projects":[
        {"collibraSystemName":"SystemName"}
        ]
    },
    "materializedMapping":{
        "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW"
    },
    "skipNonMaterializedNodes":false
}

The following table describes the various properties you can use in your JSON code block.

Property	Description	Required?
Collibra System Name	The system or server name of the data source. This field is also the full name of your System asset in Data Catalog. The value of this field must be the same as the full name of the System asset that you created when you registered the data source.	No
projects	This section contains the Collibra system names.	No
collibraSystemName	The system or server name of the data source. This is also the name of your System asset in Data Catalog: Specify this property with the same name as the name of the System asset that you created when you registered the data source. In the following code example, the project is stitched to the `systemname1` System asset in Data Catalog. { "collibraSystemNames":{ "projects":[ {"collibraSystemName":"systemname1"} ] }, }	No
materializedMapping	Indicates how materializations in dbt are mapped. If you do not specify this property, CollibraData Lineage maps materializations to tables by default. You can change the mapping of a materialization to view. In the following example, the ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES materialization is mapped to a view. "materializedMapping":{ "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW" }	No
skipNonMaterializedNodes	Specifies whether to skip dbt nodes that don't build objects in your database. `false` (default): Doesn't skip dbt nodes based on whether they build objects in your database. `true`: Skips dbt nodes that don't build objects in your database, such as data tets and ephemeral models, so they aren't included in technical lineage. A data test isn't skipped if dbt builds a relation for it. This happens when `store_failures_as` is set to a value other than `ephemeral`, or when `store_failures_as` isn't set and `store_failures` is true. If `store_failures_as` is set to `ephemeral`, the data test is skipped regardless of the `store_failures` value. Snapshots, seeds, models with custom materializations, and analyses are never skipped. Note Changing this property from `false` to `true` removes the affected assets from Collibra during the next synchronization.	No