Supported transformation details

Collibra Data Lineage supports the most commonly used transformations in the following sources:

For the supported transformations in Azure Data Factory (beta) , go to Supported Azure Data Factory transformation details.

IBM DataStage

IBM DataStage uses jobs with stages instead of transformations. IBM Datastage has three job types: parallel jobs, sequence jobs and server jobs. For a list of all job stages per job type in IBM DataStage, read the IBM documentation.

Informatica PowerCenter transformations

The following table shows a non-exhaustive list of supported and unsupported transformations in Informatica PowerCenter.

Supported transformations

Unsupported transformations
  • Aggregator
  • Expression1
  • Filter
  • Input
  • Joiner
  • Lookup
  • Mapplet
  • Normalizer
  • Output
  • Rank
  • Router
  • Sorter
  • Source
  • SQL in the translate_db_type function
  • Target
  • Transaction Control
  • Union
  • Update Strategy
  • Data Masking
  • Java
  • Sequence Generator
  • Stored Procedure2
  • Web Services
  • XML
    Note 
  1. The transformation is shown if the column (expression) is using at least one column from another connected transformation.
  2. The stored procedures are stored and run in the databases that Informatica PowerCenter connects to. Collibra Data Lineage does not access the Informatica PowerCenter data sources, so Collibra Data Lineage collects the stored procedure names but does not support the Stored Procedure transformation.

Informatica Intelligent Cloud Services

The following table shows a non-exhaustive list of supported and unsupported transformations and constructions in Informatica Intelligent Cloud Services. Specifically, transformations and constructions in the Cloud Data Integration service.

Supported transformations

Unsupported transformations, functions and constructions
  • Data-driven conditions
  • Expression, including custom expressions in the supported transformations
  • Filter
  • Joiner, including join conditions
  • Lookup
  • Mapplet
  • Router
  • Sequence Generator
  • Source
  • Stored Procedure
  • Target
  • Union
  • Aggregator
  • Cleanse
  • Data Masking
  • Deduplicate
  • Hierarchy Builder
  • Hierarchy Parser
  • Hierarchy Processor
  • Input
  • Java
  • Labeler
  • Machine Learning
  • Normalizer
  • NEXTVAL
  • Parse
  • Python
  • Rank
  • Rule Specification
  • Structure Parser
  • Transaction Control
  • Velocity
  • Verifier
  • Web Services

Snowflake

You can create technical lineage for Snowflake by using SQL Snowflake ingestion mode or SQL-API Snowflake ingestion mode. Collibra Data Lineage supports different queries and transformations for each ingestion method. For more information about the ingestion methods, go to Technical lineage for Snowflake ingestion methods.

SQL Snowflake ingestion mode

With the SQL Snowflake ingestion mode, Collibra Data Lineage does not support the following non-exhaustive list of transformations:

  • Snowflake Scripting
  • Snowpark

SQL-API Snowflake ingestion mode

With the SQL-API Snowflake ingestion mode, Collibra Data Lineage supports the Data Manipulation Language (DML) statements from the following sources. The table also shows a non-exhaustive list of unsupported queries and transformations.

Supported transformations

Unsupported queries and transformations
  • Using a driver
  • Direct login
  • Stored procedures
  • Streams
  • The COPY INTO DML command
  • Data Definition Language (DDL) queries
  • Queries or query paths that are not executed 1
  • Snowflake Scripting
  • Snowpark
  • Snowpipes
Note 
  1. If you create technical lineage for Snowflake by using the JDBC connection type, only queries or query paths that are executed are supported. For example, if a SQL query contains a CASE statement, the technical lineage will only show lineage from the WHEN path that was executed. However, if you use the folder connection type to ingest Snowflake, SQL queries that include all paths of a CASE statement will be parsed and reflected in the technical lineage.

SQL Server Integration Services (SSIS)

Collibra Data Lineage supports the following non-exhaustive list of transformations in SQL Server Integration Services:

  • Aggregate
  • Cache Transform
  • Conditional Split
  • Data Conversion
  • Derived Column
  • Fuzzy Grouping
  • Lookup
  • Merge Join
  • Multicast
  • OLE DB Command
  • Row Count
  • Script Component
  • Slowly Changing Dimension
  • Sort
  • Union All
Important 
  • Collibra Data Lineage supports SQL, but cannot parse other languages or scripts, for example SHELL and BAT scripts.
  • SQL statements from Excel are not supported.
  • All SQL queries must be preceded by the keyword SELECT, or else they will be skipped. Furthermore, if a comment precedes the keyword SELECT, the query will be skipped.