Supported data sources for technical lineage
Collibra Data Intelligence Platform supports many data sources and metadata sources, including JDBC data sources, ETL tools and BI tools, for which you can create a technical lineage.
For a complete list of required permissions per supported data source type, see the Requirements and permissions section in Prepare the lineage harvester configuration file.
JDBC data sources
The following tables show the supported JDBC data sources.
- Technical lineage via Edge
- Lineage harvester
The following table lists the supported JDBC data sources and connection types you can use when you add capabilities for different data sources. The Shared Storage connection is equivalent to the folder connection type when you use the lineage harvester.
JDBC data source type |
Supported versions |
Connection type |
Scope |
Steps to create technical lineage |
---|---|---|---|---|
Amazon Redshift |
1.2.34.1058 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for Amazon Redshift on Edge. |
Azure SQL Data Warehouse |
Newest version |
JDBC connection, Shared Storage connection |
SQL based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure SQL Data Warehouse on Edge. |
Azure SQL server |
Newest version |
JDBC connection, Shared Storage connection |
SQL based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure SQL server on Edge. |
Azure Synapse Analytics |
Newest version |
JDBC connection, Shared Storage connection |
SQL based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure Synapse Analytics on Edge. |
Databricks Unity Catalog (Beta) | Newest version | Databricks connection | Lineage information from the lineage system tables | Create technical lineage for Databricks Unity Catalog on Edge |
Google BigQuery |
Newest version |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for Google BigQuery on Edge. |
Greenplum |
6.10 and newer |
JDBC connection, Shared Storage connection |
SQL based input. |
Create technical lineage for Greenplum on Edge. |
HiveQL (SQL-like statements) |
2.3.5 and newer |
JDBC connection, Shared Storage connection |
SQL based input and connection via an AWS host. |
Create technical lineage for HiveQL on Edge. |
IBM Db2 |
11.5 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for IBM Db2 on Edge. |
Oracle |
11g, 12c and newer |
JDBC connection, Shared Storage connection |
SQL based input and stored procedures. |
Create technical lineage for Oracle on Edge. |
PostgreSQL |
9.4, 9.5 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for PostgreSQL on Edge. |
Microsoft SQL Server |
2014, 2016 and newer |
JDBC connection, Shared Storage connection |
SQL based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Microsoft SQL Server on Edge. |
MySQL |
5.7, 8 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for MySQL on Edge. |
Netezza |
7.2.1.0 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for Netezza on Edge. |
SAP HANA, including SAP HANA Classic on-premises and SAP HANA Cloud/Advanced |
|
JDBC connection, Shared Storage connection |
|
Create technical lineage for SAP HANA on Edge. |
Snowflake |
Newest version |
JDBC connection, Shared Storage connection |
For more information, go to Technical lineage for Snowflake ingestion methods. |
Create technical lineage for Snowflake on Edge. |
Spark SQL |
2.4.3 and newer |
JDBC connection, Shared Storage connection |
SQL-based input without stored procedures and connection via an AWS host. For Spark SQL data source, we recommend using the folder connection type to connect to the directory with your SQL queries. |
Create technical lineage for Spark SQL on Edge. |
Sybase Adaptive Server Enterprise |
16.0 SP02 and newer |
JDBC connection, Shared Storage connection |
SQL based input without stored procedures. |
Create technical lineage for Sybase Adaptive Server Enterprise on Edge. |
Teradata |
15.0, 16.20.07.01 and newer |
JDBC connection, Shared Storage connection |
SQL based input, including BTEQ scripts. |
Create technical lineage for Teradata on Edge. |
The following table shows the supported JDBC data sources and driver versions that have been tested. You can connect to them via a JDBC driver or by creating a folder.
JDBC data source type |
Supported versions |
Connection type |
Scope |
Steps to create technical lineage |
---|---|---|---|---|
Amazon Redshift |
1.2.34.1058 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for Amazon Redshift by using the lineage harvester. |
Azure SQL Data Warehouse |
Newest version |
JDBC, Folder |
SQL-based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure SQL Data Warehouse by using the lineage harvester. |
Azure SQL server |
Newest version |
JDBC, Folder |
SQL-based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure SQL server by using the lineage harvester. |
Azure Synapse Analytics |
Newest version |
JDBC, Folder |
SQL-based input and stored procedures. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Azure Synapse Analytics by using the lineage harvester. |
Google BigQuery |
Newest version |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for Google BigQuery by using the lineage harvester. |
Greenplum |
6.10 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for Greenplum by using the lineage harvester. |
HiveQL (SQL-like statements) |
2.3.5 and newer |
JDBC, Folder |
SQL-based input and connection via an AWS host. Stored procedures are not supported. |
Create technical lineage for HiveQL by using the lineage harvester. |
IBM Db2 |
11.5 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for IBM Db2 by using the lineage harvester. |
Oracle |
11g, 12c and newer |
JDBC, Folder |
SQL-based input and stored procedures. |
Create technical lineage for Oracle by using the lineage harvester. |
PostgreSQL |
9.4, 9.5 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for PostgreSQL by using the lineage harvester. |
Microsoft SQL Server |
2014, 2016 and newer |
JDBC, Folder |
SQL-based input and stored procedures. Note Only Basic Authentication is supported. NTLM authentication, for example, is not. Note Technical lineage cannot be created for views and procedures if the SQL definitions are encrypted in the database.
|
Create technical lineage for Microsoft SQL Server by using the lineage harvester. |
MySQL |
5.7, 8 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for MySQL by using the lineage harvester. |
Netezza |
7.2.1.0 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for Netezza by using the lineage harvester. |
SAP HANA, including SAP HANA Classic on-premises and SAP HANA Cloud/Advanced |
|
JDBC, Folder |
|
Create technical lineage for SAP HANA by using the lineage harvester. |
Snowflake |
Newest version |
JDBC, Folder |
For more information, go to Technical lineage for Snowflake ingestion methods. |
Create technical lineage for Snowflake by using the lineage harvester. |
Spark SQL |
2.4.3 and newer |
JDBC, Folder |
SQL-based input and connection via an AWS host. Stored procedures are not supported. For Spark SQL data source, we recommend using the folder connection type to connect to the directory with your SQL queries. |
Create technical lineage for Spark SQL by using the lineage harvester. |
Sybase Adaptive Server Enterprise |
16.0 SP02 and newer |
JDBC, Folder |
SQL-based input without stored procedures. |
Create technical lineage for Sybase Adaptive Server Enterprise by using the lineage harvester. |
Teradata |
15.0, 16.20.07.01 and newer |
JDBC, Folder |
SQL-based input and stored procedures, including BTEQ scripts. |
Create technical lineage for Teradata by using the lineage harvester. |
ETL tools
The following table shows the supported ETL tools.
- Technical lineage via Edge
- Lineage harvester
The following table lists the supported ETL data sources and connection types you can use when you add capabilities for different data sources. The Shared Storage connection is equivalent to the folder connection type when you use the lineage harvester.
ETL tool |
Supported versions |
Connection type |
Scope |
Steps to create technical lineage |
---|---|---|---|---|
Azure Data Factory | 2 and newer | API | Commonly supported transformations and activities in Azure Data Factory. For details, go to Supported transformation details. | Create technical lineage for Azure Data Factory on Edge. |
dbt |
1.4 or newer | API for dbt Cloud Shared Storage connection for dbt Core |
Commonly supported model types in dbt. For details, go to Supported transformation details. | |
IBM InfoSphere DataStage |
11.5 and newer |
Shared Storage connection |
Commonly used DataStage ETL components including SQL overrides and transformation details. Collibra Data Lineage supports IBM InfoSphere DataStage transformation logic. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for DataStage on Edge. |
Informatica Intelligent Cloud Services, specifically Cloud Data Integration Tip Data Integration is one of the Informatica Intelligent Cloud services. |
Cloud, newest only |
Informatica Intelligent Cloud Services (IICS) connection Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Informatica Intelligent Cloud Services (IICS) connection. |
Commonly used transformations in Informatica Intelligent Cloud Services: Data Integration, including SQL overrides. Supported data sources are locally stored flat files and databases. |
Create technical lineage for IICS on Edge. |
Informatica PowerCenter |
9.6 and newer |
Shared Storage connection |
Commonly used transformations in Informatica PowerCenter, including SQL overrides. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for Informatica PowerCenter on Edge. |
Matillion |
Newest version |
Matillion connection Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Matillion connection. |
SQL based input without stored procedures. Technical lineage via Edge can only access Redshift and Snowflake projects. |
Create technical lineage for Matillion on Edge. |
SQL Server Integration Services (SSIS) |
2012 and newer Package format version 6 or newer. |
Shared Storage connection |
All commonly used transformations in SSIS, data flows and mappings, including SQL overrides. Important SQL statements from Excel are not supported. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for SQL Server Integration Services on Edge. |
The following table shows the supported ETL tools and driver versions that have been tested. You can connect to them via an API or by creating a folder.
ETL tool |
Supported versions |
Connection type |
Scope |
Steps to create technical lineage |
---|---|---|---|---|
Azure Data Factory | 2 and newer | API | Commonly supported transformations and activities in Azure Data Factory. For details, go to Supported transformation details. | Create technical lineage for Azure Data Factory by using the lineage harvester. |
dbt |
1.4 or newer | API for dbt Cloud Folder for dbt Core |
Commonly supported model types in dbt. For details, go to Supported transformation details. |
Create technical lineage for dbt Cloud by using the lineage harvester. Create technical lineage for dbt Core by using the lineage harvester. |
IBM InfoSphere DataStage |
11.5 and newer |
Folder |
Commonly used DataStage ETL components including SQL overrides and transformation details. Collibra Data Lineagesupports IBM InfoSphere DataStage transformation logic. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for DataStage by using the lineage harvester. |
Informatica Intelligent Cloud Services, specifically Cloud Data Integration Tip Data Integration is one of the Informatica Intelligent Cloud services. |
Cloud, newest only |
API |
Commonly used transformations in Informatica Intelligent Cloud Services: Data Integration, including SQL overrides. Supported data sources are locally stored flat files and databases. |
Create technical lineage for Informatica Intelligent Cloud Services by using the lineage harvester. |
Informatica PowerCenter |
9.6 and newer |
Folder |
Commonly used transformations in Informatica PowerCenter, including SQL overrides. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for Informatica PowerCenter by using the lineage harvester. |
Matillion |
Newest version |
API |
SQL based input without stored procedures. The lineage harvester can only access Redshift and Snowflake projects. |
Create technical lineage for Matillion by using the lineage harvester. |
SQL Server Integration Services (SSIS) |
2012 and newer Package format version 6 or newer. |
Folder |
All commonly used transformations in SSIS, data flows and mappings, including SQL overrides. Important SQL statements from Excel are not supported. You have to prepare a folder with all data objects that you want to process. |
Create technical lineage for SQL Server Integration Services by using the lineage harvester. |
BI tools
The following table shows the supported BI tools.
- Technical lineage via Edge
- Lineage harvester
The following table lists the supported BI data sources and connection types you can use when you add capabilities for different data sources.
BI tool |
Tested versions |
Connection type |
Capability |
Steps to create technical lineage |
---|---|---|---|---|
Tableau |
Newest |
API |
Technical Lineage for Tableau | Create technical lineage for Tableau on Edge. |
Power BI |
Newest |
API |
Technical Lineage for Power BI | Create technical lineage for Power BI on Edge. |
MicroStrategy |
Newest Note Freeform SQL is supported for reports (not cubes or dossiers) if you have MicroStrategy update10 or newer, or MicroStrategy ONE. |
API |
Technical Lineage for MicroStrategy | Create technical lineage for MicroStrategy on Edge. |
The following table shows the supported BI tools.
BI tool |
Tested versions |
Connection type |
Steps to create technical lineage |
---|---|---|---|
Tableau |
Newest |
API. You have to prepare:
|
Create technical lineage for Tableau by using the lineage harvester. |
Power BI | Newest |
API.
Collibra Data Lineage supports:
|
Create technical lineage for Power BI by using the lineage harvester. |
Looker |
Newest |
API. Collibra Data Lineage automatically creates a technical lineage, but stitching is not available. You have to prepare a lineage harvester configuration file for Looker ingestion. |
Create technical lineage for Looker by using the lineage harvester. |
SQL Server Reporting Services (SSRS) or Power BI Report Server (PBRS) |
|
API. You have to prepare:
Important There are known limitations to the metadata returned by the API when integrating PBRS. For example, Power BI reports in PBRS are ingested as Power BI Report assets in Data Catalog, but there is no technical lineage for the reports. |
Create technical lineage for SSRS or PBRS by using the lineage harvester. |
MicroStrategy |
Newest |
Direct connection to the repository. Stitching is not available and there is no true technical lineage. There is only a diagram view that you can access via a Column or Table asset, but not via MicroStrategy assets. You have to prepare a lineage harvester configuration file for MicroStrategy ingestion.
You can access:
|
|
MicroStrategy (NEW) |
Newest Note Freeform SQL is supported for reports (not cubes or dossiers) if you have MicroStrategy update10 or newer, or MicroStrategy ONE. |
You have to prepare a lineage harvester configuration file for MicroStrategy ingestion. Benefits of the new integration method include:
|
Create technical lineage for MicroStrategy by using the lineage harvester. |
Custom technical lineage
You can create a custom technical lineage to include data objects from data sources that are not listed above.
For information on creating a custom technical lineage via Edge, go to Create technical lineage via Edge for custom technical lineage.
For information on creating technical lineage by using the lineage harvester, go Custom technical lineage via the lineage harvester.