Lineage harvester integrations available in beta

Collibra Data Intelligence Cloud supports many data sources and metadata sources, such as ETL tools or BI sources, for which you can create a technical lineage or which you can ingest.

Before Collibra releases a new lineage harvester, we test the new lineage harvester and technical lineage via Edge integrations extensively. However, we cannot foresee all possible use cases and scenarios. To further improve the lineage harvester and technical lineage via Edge, you can now test new integrations via the lineage harvester and via Edge in beta. After a testing period, the new integrations will be made available for all Collibra Data Lineage users.

The following table shows the technical lineage via Edge integrations that are currently in beta.

technical lineage via Edge integrations	Description	How to use
Technical lineage via JDBC connection	Create technical lineage for the supported JDBC data sources by using JDBC connections on Edge. For a list of supported JDBC data sources, go to Supported data sources for technical lineage.	In the Services Configuration > Beta features section of the Collibra settings or in Collibra Console, set the Technical lineage via JDBC connection enabled setting to True.
Technical lineage for SQL via folder, ETL tools, and custom lineage	Create custom technical lineage and technical lineage for the following data sources: Supported ETL tools. Supported JDBC data sources by using Shared Storage connections. For a list of supported data sources, go to Supported data sources for technical lineage.	In the Services Configuration > Beta features section of the Collibra settings or in Collibra Console, set the Technical lineage for SQL via folder, ETL tools, and custom lineage on Edge enabled setting to True.
Technical lineage for Tableau	Create a technical lineage for Tableau and Power BI via Edge. Benefits include: Seamless integration with Data Catalog. The Edge User Interface (UI), instead of Command Line Interface (CLI). Tableau connection from Edge, instead of lineage harvester drivers. Refresh metadata via the Tableau Server and Power BI Server assets.	In the Services Configuration > Beta features section of the Collibra settings or in Collibra Console, set the Power BI and Tableau synchronization enabled setting to True.
Technical lineage for Power BI

The following table shows the technical lineage via lineage harvester integrations that are currently in beta.

Lineage harvester integrations	Available in lineage harvester version	Limitations	How to join
Azure Data Factory	2022.05.0-6 and newer	The result is a complete technical lineage that includes many Azure Data Factory transformations and some datasets. Flowlets are not yet supported. Transformations containing column patterns or rule-based mappings can only be partially analyzed, as they generate column names on the fly during the actual dataflow run. If technical lineage is detected from a dynamically generated column, it is given the placeholder Dynamic Column in the technical lineage viewer. Some reserved variables names, for example {@context}, are not yet supported. For more information, go to Supported Azure Data Factory transformation details.	Apply to the Public ADF Beta program for technical lineage.
MicroStrategy integration 2.0		You can access: Microsoft SQL Server repository. Any local or remote PostgreSQL database. The MicroStrategy Intelligence Server has an embedded PostgreSQL repository, as its default repository.	Please contact your Customer Success Manager.

Lineage harvester integrations

Available in lineage harvester version

Limitations

How to join

Azure Data Factory

2022.05.0-6 and newer

The result is a complete technical lineage that includes many Azure Data Factory transformations and some datasets. Flowlets are not yet supported.

Transformations containing column patterns or rule-based mappings can only be partially analyzed, as they generate column names on the fly during the actual dataflow run. If technical lineage is detected from a dynamically generated column, it is given the placeholder Dynamic Column in the technical lineage viewer.

Some reserved variables names, for example {@context}, are not yet supported.

For more information, go to Supported Azure Data Factory transformation details.

Apply to the Public ADF Beta program for technical lineage.

MicroStrategy integration 2.0

You can access:

Microsoft SQL Server repository.
Any local or remote PostgreSQL database.
The MicroStrategy Intelligence Server has an embedded PostgreSQL repository, as its default repository.

Please contact your Customer Success Manager.

Warning The lineage harvester and technical lineage via Edge beta integrations offer early access to new integrations. However, we can only allow a limited number of customers to test the integrations and give feedback. We will make the integrations available for all customers after processing the feedback and improving Collibra Data Lineage.