About Technical lineage via Edge

You can use Edge to collect metadata from your data sources and create new relations between data elements from your data source and existing assets into Data Catalog. Edge collects transformation logic like SQL scripts and ETL scripts from a specified location, for example a database table or a folder on a file system.

Just like the lineage harvester, Edge connects to different Collibra Data Lineage service instances based on your geographical location and cloud provider.

For details about the typical workflow of creating technical lineage by using the lineage harvester or technical lineage via Edge, go to Technical lineage typical workflow.

For a list of the supported data sources and the technical lineage capabilities and connection types for each data source, go to Supported data sources for technical lineage. For specific steps to create a technical lineage on Edge, go to Creating technical lineage via Edge.

You can also use Edge to create a custom technical lineage. For complete information, go to Create technical lineage via Edge and select Custom technical lineage.

Important If you want to use technical lineage via Edge together with the lineage harvester, ensure that you use the lineage harvester version 2023.04 or newer. For more information, go to Migrate the technical lineage of a data source.

Tip Check out our free Why you should use Lineage on Edge today training course in Collibra University.

Enabling and configuring technical lineage via Edge

To create a technical lineage via Edge, you must enable the feature in the Collibra settings or in Collibra Console.

You can define how Edge accesses the data sources by creating different connections. The following connection types are supported on Edge:

  • JDBC connection, for JDBC data sources and ETL tools.
  • Shared Storage connection, for JDBC data sources and some ETL tools.
  • APIs, for MicroStrategy, Power BI, Tableau, Informatica Intelligent Cloud Services, and Matillion.

Configurations for technical lineage via Edge include the following:

  • General configuration settings in the Collibra settings or Collibra Console, which apply to all data sources for which you create the technical lineage. For example, you can enter your Collibra Data Intelligence Platform username and user password in the general configuration settings.
  • Specific configuration settings for each data source. You can add a technical lineage capability for each data source to provide specific configurations.

Specific permissions might be required to access different data sources. For complete information, go to Data source-specifc permissions.

BI tool ingestion via Edge

Note BI ingestion via Edge is currently available for MicroStrategy, Power BI, and Tableau. To integrate other supported BI tools, you need to use the lineage harvester. You can also use the lineage harvester to integrate MicroStrategy, Power BI, and Tableau.

During the technical lineage process, relations of the type "Data Element targets / sources Data Element" are automatically created:

  • Between data objects in your data source and their corresponding assets in Data Catalog, including the asset that you create when preparing the Data Catalog physical data layer.
  • If you are integrating a BI tool, between ingested assets from BI sources and Data Catalog assets from registered data sources.
Note 
  • You can't work with Edge via the REST API.
  • You can't migrate from Jobserver to Edge to preserve the metadata that you manually added to the assets that you ingested via Jobserver.