Supported transformation details

Collibra Data Lineage supports the most commonly used transformations in the following sources:

For the supported transformations in Azure Data Factory (beta) , go to Supported Azure Data Factory transformation details.

For the supported transformations in Snowflake, go to Technical lineage for Snowflake ingestion methods.

IBM DataStage

Instead of transformations, IBM DataStage uses jobs with stages. IBM Datastage has three job types: parallel jobs, sequence jobs and server jobs. Collibra Data Lineage only supports the IBM DataStage stages of parallel jobs.

For a list of all job stages per job type in IBM DataStage, read the IBM documentation.

Informatica PowerCenter transformations

The following table shows a non-exhaustive list of supported and unsupported transformations in Informatica PowerCenter.

Supported transformations

Unsupported transformations
  • Aggregator
  • Expression1
  • Filter
  • Joiner
  • Lookup
  • Mapplet
  • Normalizer
  • Rank
  • Sorter
  • Source
  • SQL
  • Target
  • Transaction Control
  • Java
  • Python
  • Stored Procedure2
  • XML
    Note 
  1. The transformation is shown if the column (expression) is using at least one column from another connected transformation.
  2. The stored procedures are stored and run in the databases that Informatica PowerCenter connects to. Collibra Data Lineage does not access the Informatica PowerCenter data sources, so Collibra Data Lineage collects the stored procedure names but does not support the Stored Procedure transformation.

Informatica Intelligent Cloud Services

The following table shows a non-exhaustive list of supported and unsupported transformations and constructions in Informatica Intelligent Cloud Services. Specifically, transformations and constructions in the Cloud Data Integration service.

Supported transformations

Unsupported transformations and constructions
  • Expression, including custom expressions in the supported transformations
  • Filter
  • Joiner, including join conditions
  • Lookup
  • Mapplet
  • Router
  • Sequence Generator
  • Source
  • Target
  • Union
  • NEXTVAL

Snowflake

You can create technical lineage for Snowflake by using SQL mode Snowflake ingestion or SQL-API mode Snowflake ingestion. Collibra Data Lineage supports different queries and transformations for each ingestion method. For more information about the ingestion methods, go to Technical lineage for Snowflake ingestion methods.

SQL mode Snowflake ingestion

With the SQL mode Snowflake ingestion, Collibra Data Lineage does not support the following non-exhaustive list of transformations:

  • Snowflake Scripting
  • Snowpark

SQL-API mode Snowflake ingestion

With the SQL-API mode Snowflake ingestion, Collibra Data Lineage supports the Data Manipulation Language (DML) statements from the following sources. The table also shows a non-exhaustive list of unsupported queries and transformations.

Supported transformations

Unsupported queries and transformations
  • Using a driver
  • Direct login
  • Stored procedures
  • Streams
  • The COPY INTO DML command
  • Data Definition Language (DDL) queries
  • Snowflake Scripting
  • Snowpark
  • Snowpipes

SQL Server Integration Services (SSIS)

Collibra Data Lineage supports the following non-exhaustive list of transformations in SQL Server Integration Services:

  • Aggregate
  • Cache Transform
  • Conditional Split
  • Data Conversion
  • Derived Column
  • Fuzzy Grouping
  • Lookup
  • Merge Join
  • Multicast
  • OLE DB Command
  • Row Count
  • Script Component
  • Slowly Changing Dimension
  • Sort
  • Union All
Important 
  • Collibra Data Lineage supports SQL, but cannot parse other languages or scripts, for example SHELL and BAT scripts.
  • SQL statements from Excel are not supported.
  • All SQL queries must be preceded by the keyword SELECT, or else they will be skipped. Furthermore, if a comment precedes the keyword SELECT, the query will be skipped.