Lineage harvester integrations available in beta

Collibra Data Intelligence Cloud supports many data sources and metadata sources, such as ETL tools or BI sources, for which you can create a technical lineage or which you can ingest.

Before Collibra releases a new lineage harvester, we test the new lineage harvester integrations extensively. However, we cannot foresee all possible use cases and scenarios. To further improve the lineage harvester, you can now test new lineage harvester integrations in beta. After a testing period, the new lineage harvester integrations become available for all Collibra Data Lineage users

The following table shows which integrations the lineage harvester currently supports in beta.

Lineage harvester integrations	Available in lineage harvester version	Limitations	How to join
Azure Data Factory	2022.05.0-6 and newer	The result is a complete technical lineage that includes many Azure Data Factory transformations and some datasets. Flowlets are not yet supported. Transformations containing column patterns or rule-based mappings can only be partially analyzed, as they generate column names on the fly during the actual dataflow run. If technical lineage is detected from a dynamically generated column, it is given the placeholder Dynamic Column in the technical lineage viewer. Some reserved variables names, for example {@context}, are not yet supported. For more information, go to Supported Azure Data Factory transformation details.	Apply to the Public ADF Beta program for technical lineage.
SQL-API mode Snowflake ingestion	2022.11 and newer	Snowpipes and Data Definition Language (DDL) queries are not supported. For more information, go to Technical lineage for Snowflake ingestion methods.	Apply to the Public Snowflake Beta program for technical lineage.

Lineage harvester integrations

Available in lineage harvester version

Limitations

How to join

Azure Data Factory

2022.05.0-6 and newer

The result is a complete technical lineage that includes many Azure Data Factory transformations and some datasets. Flowlets are not yet supported.

Transformations containing column patterns or rule-based mappings can only be partially analyzed, as they generate column names on the fly during the actual dataflow run. If technical lineage is detected from a dynamically generated column, it is given the placeholder Dynamic Column in the technical lineage viewer.

Some reserved variables names, for example {@context}, are not yet supported.

For more information, go to Supported Azure Data Factory transformation details.

Apply to the Public ADF Beta program for technical lineage.

SQL-API mode Snowflake ingestion

2022.11 and newer

Snowpipes and Data Definition Language (DDL) queries are not supported.

For more information, go to Technical lineage for Snowflake ingestion methods.

Apply to the Public Snowflake Beta program for technical lineage.

Warning The lineage harvester beta integrations offer early access to new integrations. However, we can only allow a limited number of customers to test the integrations and give feedback. We will make the integrations available for all customers after processing the feedback and improving the lineage harvester.