Technical lineage

Technical lineage is a detailed lineage graph that shows how data transforms and flows from source to destination across its entire lifecycle. It enables you to easily discover where tables and columns are used and how they relate to each other. You can view a technical lineage for the following asset types:

• Table
• Column
• Looker Look
• MicroStrategy Report
• MicroStrategy Table
• MicroStrategy Column
• Power BI Report
• Power BI Table
• Power BI Column
• SSRS Report
• SSRS Table
• SSRS Column
• Tableau Worksheet
• Tableau Data Attribute

During the technical lineage process, relations of the type "Data Element targets / sources Data Element" are automatically created:

• Between data objects in your data source and assets from registered data sources.
• Between ingested assets from BI sources and Data Catalog assets from registered data sources.

Data objects

You can see two types of data objects in your technical lineage:

• Data objects from your data source that are stitched to assets in Data Catalog and for which you created the technical lineage. These assets have a yellow background.
  Example 

  

• Other objects, for example temporary tables and columns, that are collected from your data sources, but are not stitched to assets in Data Catalog. These objects have a gray background.
  Example 

  

Naming convention

When you create a technical lineage, Data Catalog follows a strict naming convention for the full names of assets. Each asset has a display name and full name. You can freely edit the display name. However, do not edit the full name, because Data Catalog needs it to refresh data sources for which you created the technical lineage and to refresh the technical lineage itself.

When you prepare the Data Catalog physical data layer and the configuration file, you should always use the full name as the name of the corresponding data object in your data source for the following assets:

• System
• Database
• Schema

Note If you want to create a technical lineage for a Google BigQuery database, the project name in the configuration file must be the same as the full name of the Database asset.

Warning Editing the full name of the Schema, Database and System assets may lead to errors during the technical lineage creation process.

The lineage harvester and technical lineage via Edge

The lineage harvester is a command-line tool that you can install, configure and run to create a technical lineage. You also can create a technical lineage on Edge. With technical lineage via Edge, you can use the Edge user interface, which provides seamless integration with Data Catalog.

The typical workflow for creating a technical lineage is the same whether you use the lineage harvester or Edge. For details, go to The lineage harvester and Technical lineage via Edge .

If you want to use technical lineage via Edge and the lineage harvester together, you must use the lineage harvester version 2023.04 or newer. If you want to maintain on Edge the technical lineage that you created by using the lineage harvester, you can add technical lineage capabilities for the data sources with the same source IDs. For details, go to Migrate the technical lineage of a data source.

Note Use the lineage harvester pre-release versions only in non-production environments.

For the steps to create technical lineage for the JDBC data sources and ETL tools, go to Creating technical lineage by using the lineage harvester and Creating technical lineage via Edge. For the list of supported data sources, go to Supported data sources for technical lineage.