Matillion source configuration

Updated:

The Source configuration field in the Technical Lineage for Matillion capability allows you to map the names of databases in Matillion to the names of their corresponding System assets in Data Catalog.

Tip If you previously integrated Matillion via the lineage harvester (deprecated), you can copy and paste the JSON code from your <source ID> configuration file into the Source configuration field.

The value of the Source configuration field must be a valid block of JSON code, for example:

Copy 
{
    "found_dbname=dbtest;found_hostname=test": {
        "collibraSystemName": "mssql-system-name"
    },
    "found_dbname=testsid;found_hostname=*": {
        "dbname": "oracle-database-name",
        "schema": "oracle-schema-name",
        "collibraSystemName": "oracle-system-name"
    }    
}

The following table describes the various properties you can use in your JSON code block.

Property

 Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>

The information of the supported data sources in Matillion to be collected by Collibra Data Lineage.

<database name>
The database name in Matillion.
<server name>
The name of the server that the database is running on. You can specify found_hostname=* to include all servers.
Tip 

You can use wildcards to capture multiple connection string combinations:

Pattern Description
* Matches everything.
? Matches any single character.
[seq] Matches any character in "seq".
[!seq] Matches any character not in "seq".

Yes
dbname

The name of the database asset in Data Catalog.

If you leave this property blank, the database is stitched to the database of DEFAULT in Data Catalog.

No
schema

The name of the schema asset in Data Catalog. Specify this property with the schema name that you created when you registered the data source.

If you leave this property blank, the schema is stitched to the schema of DEFAULT in Data Catalog.

No
dialect

Select one of the following dialects for your data source

  • azure, for an Azure SQL Server data source.
  • bigquery, for a Google BigQuery data source.
  • db2, for a IBM Db2 data source.
  • generic, for any data source.
  • greenplum, for a Greenplum data source.
  • hana, for a SAP HANA data source.
  • hive, for a Hive data source.
  • impala, for a Impala data source.
  • mssql, for a Microsoft SQL Server data source.
  • mysql, for a MySQL data source.
  • netezza, for a Netezza data source.
  • oracle, for an Oracle data source.
  • postgres, for an PostgreSQL data source.
  • redshift, for an Amazon Redshift data source.
  • snowflake, for a Snowflake data source.
  • spark, for a Spark SQL data source.
  • sybase, for a Sybase data source.
  • teradata, for a Teradata data source.
  • vertica, for a Vertica data source.

No
collibraSystemName

The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source.

If you leave this property blank, the system is stitched to the system of DEFAULT in Data Catalog. If you are missing lineage or your lineage objects aren’t stitching to Catalog assets in Data Catalog as you expect, ensure this property is specified properly.

For example, if you have two databases named Customers. When you register the data source, the Customers-Europe and Customers-USA system assets are created. You can then configure this property as follows.

{
    "found_dbname=dbtest;found_hostname=test": {
        "dbname": "Customers",
        "dialect": "mssql",
        "collibraSystemName": "Customers-Europe"
    },
    "found_dbname=testsid;found_hostname=*": {
        "dbname": "Customers",
        "schema": "oracle-schema-name",
        "dialect": "oracle",
        "collibraSystemName": "Customers-USA"
    }	
}

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

No