Technical lineage admin options

Until recently, the options described in this topic were available only via the CLI lineage harvester (deprecated). For reference, go to Lineage harvesting app command options and arguments.

Admin options and descriptions

Configure and run an admin job

Prerequisites

• You have created an Edge connection and added an Edge capability for each of the data sources you want to include in the technical lineage.

Steps

1. Create a Technical Lineage Admin connection.

   For CollibraData Lineage to connect to and retrieve metadata from the Collibra Data Lineage service instance, you have to create an Technical Lineage Admin connection.

   Prerequisites

   • You have a global role with the Product Rights > System administration global permission.
   • You have a global role that has the Manage Edge sites global permission.
   • You have a global role that has the Manage connections and capabilities global permission.

   Steps

   1. Open a site.
      1. On the main toolbar, click → Settings.
         The Settings page opens.
      2. In the tab pane, click Edge.
         The Sites tab opens and shows a table with an overview of your sites.
      3. In the site overview, click the name of a site.
         The site page appears.
   2. In the Connections section, click Create Connection.
      The Create Connection dialog box appears.
   3. Select the Technical Lineage Admin connection.
   4. Enter the connection information.
      

   Field	Description	Required
   Name	A name for the Edge connection.	Yes
   Description	A description of the connection.	No
   Authentication Type	The authentication method you use to connect to Collibra Data Lineage: Basic Authentication If you choose this method, ignore the rest of the fields. OAuth If you choose this method, you must use the following fields to provide a client ID and client secret. This authentication method is recommended for enhanced security. Important OAuth authentication is not yet available for Collibra Platform for Government customers.	Yes
   Client ID	Your client ID for OAuth authentication. How to obtain a client ID and client secret In Collibra Settings, click OAuth Applications. Click Register New Application. The Register New Application dialog box appears. Enter the following information: For the Application Type, select Platform. Provide a name for the application. In the Integration Type drop-down list, select Technical Lineage. Click Register. Copy and safely store the Client ID and Client Secret. Important This is the only time you are able to see the client secret. For complete information, go to OAuth Applications.	Yes
   Client Secret	Your client secret for OAuth authentication. How to obtain a client ID and client secret In Collibra Settings, click OAuth Applications. Click Register New Application. The Register New Application dialog box appears. Enter the following information: For the Application Type, select Platform. Provide a name for the application. In the Integration Type drop-down list, select Technical Lineage. Click Register. Copy and safely store the Client ID and Client Secret. Important This is the only time you are able to see the client secret. For complete information, go to OAuth Applications.	Yes .

   5. Click Create.
2. Add a Technical Lineage Admin capability to your Edge or Collibra Cloud site.

   Complete the following steps to add the technical lineage capability to your Edge or Collibra Cloud site.

   Prerequisites

   • You have a global role with the Product Rights > System administration global permission.
   • You have a global role that has the Manage connections and capabilities global permission, for example, Edge integration engineer.

   Steps

   1. Open a site.
      1. On the main toolbar, click → Settings.
         The Settings page opens.
      2. In the tab pane, click Edge.
         The Sites tab opens and shows a table with an overview of your sites.
      3. In the table, click the name of the site whose status is Healthy.
         The site page opens.
   2. Select the relevant capability template: Technical Lineage Admin.
   3. Enter the required information.
      

      Field	Description	Required?
      Name	The name of the capability.	Yes
      Description	The description of the capability.	Yes
      Admin connection	The name of the Edge connection you created in the previous step.	Yes
      Property	This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property. You can use this field to set the HTTP timeout duration by adding the httpTimeout property:  httpTimeout Type Value Type Name Example value Text Plaintext httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15. 15 Warning If you are a Collibra Platform for Government customer, this field is required to connect to a Collibra Data Lineage service instance: Properties for Collibra Platform for Government customers Type Value Type Name Value Text Plaintext techlinHost This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-europe-west1.collibra.com. Example: techlin-europe-west1.collibra.com Text Secret techlinKey This is the unique API key to connect to a Collibra Data Lineage service instance. Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Collibra Account Team. <your-techlin-key> See an example	Yes for US government customers.
      Debug	An option to enable logging of a JDBC job. If you enable logging, you can download the output file of the JDBC job in the Edge Jobs dashboard (in preview). The output file contains the logs of the JDBC driver. For more information about downloading the output file, go to Download job output files. Select one of the following values: True Enables logging of the JDBC job. False Disables logging of the JDBC job. This is the default value.	No
      Log level	An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

   4. Click Create.
      The capability is added to the Edge or Collibra Cloud site.
      The fields become read-only.
3. Configure one of the admin options and run the job.

   In this step, you configure one of the admin options and then manually trigger the job.

   You can only choose one option at a time. After you click Synchronize to run the job, you can configure another option. You don't have to create a new Edge connection or add another capability.

   Important 

   To use the List sources option or the Ignore sources option:

   1. Your metadata must be refreshed. You can either wait for the next scheduled synchronization to run, or you can edit the Integration configuration data refresh schedule setting in Collibra Console so that the refresh is done sooner.
   2. If you edit the Integration configuration data refresh schedule setting, you must restart Collibra.

   

   If you don't refresh your metadata (and restart Collibra, if necessary), an error is shown on the Integration Configuration tab.

   Prerequisites

   • A global role that has the following global permission:
     • Catalog, for example Catalog Author
     • View Edge connections and capabilities
   • A resource role with Configure external system resource permission, for example Owner.

   Steps

   1. On the main toolbar, click → Catalog.
      The Catalog homepage opens.
   2. In the tab bar, click Integrations.
      The Integrations page opens.
   3. Click the Integration Configuration tab.
      
   4. Find the connection that you used when you added the technical lineage capability, and click the link in the Capabilities column. If multiple capabilities exist for the connection, expand them to find your capability.
      The capability configuration page opens.
   5. On the Synchronize Configuration section, click Edit Configuration.
   6. In the Admin command drop-down list, select the option you want to run.
      
   7. Enter the required information.
      

      If you selected...	Then...
      List sources	Important  To use the List sources option: Your metadata must be refreshed. You can either wait for the next scheduled synchronization to run, or you can edit the Integration configuration data refresh schedule setting in Collibra Console so that the refresh is done sooner. If you edit the Integration configuration data refresh schedule setting, you must restart Collibra. If you don't refresh your metadata (and restart Collibra, if necessary), an error is shown on the Integration Configuration tab. After the data is refreshed, the green Refresh button that you need click is shown on the Integration Configuration tab. Click the green Refresh button, as shown in the following image. The results are shown in the Sources field. Note  You don't need to click Save; it serves no purpose in this context.
      Ignore sources	Important  To use the Ignore sources option: Your metadata must be refreshed. You can either wait for the next scheduled synchronization to run, or you can edit the Integration configuration data refresh schedule setting in Collibra Console so that the refresh is done sooner. If you edit the Integration configuration data refresh schedule setting, you must restart Collibra. If you don't refresh your metadata (and restart Collibra, if necessary), an error is shown on the Integration Configuration tab. For each source that you want to ignore, ensure that the Active checkbox in the respective technical lineage Edge capability is cleared. For each source that you want to ignore, ensure that the Active checkbox in the respective technical lineage Edge capability is cleared. For more information, go to Delete the of a data source on Edge. In the Sources drop-down list, select the source or sources you want excluded from the technical lineage. Click Save. In the Synchronize Configuration section, click Synchronize. To view confirmation of the ignored source or sources: Open the Activities list. In the row containing the job, click Result. The Synchronization Results dialog box appears.
      Analyze files	In the File ID field, specify the file ID of the ZIP file on the Collibra Data Lineage service instance that you want analyzed. Enter only one file ID per field. To specify more than one file, click Add File ID. Tip When using the Load option in the Processing Level setting in your Edge capacity, the job result includes a link to download a ZIP file with the metadata. The link resembles the following: https://integrations.collibra-abc.com/rest/2.0/files/01944f12-7665-7d9c-8bc5-aa426b6a63cc. In this example, the file ID that you have to specify is: 01944f12-7665-7d9c-8bc5-aa426b6a63cc. Click Save. In the Synchronize Configuration section, click Synchronize. To view confirmation of the analyzed files: Open the Activities list. In the row containing the job, click Result. The Synchronization Results dialog box appears.
      Sync	Click Save. Click Synchronize. A single synchronization job is triggered for all configured data sources. To view the ingestion report: Open the Activities list. In the row containing the synchronization job, click Result. The synchronization report summary appears.
      Export lineage (in preview)	In the Export type drop-down list, select Custom lineage, Custom lineage with transformations, or Transformations. In the Source drop-down list, select a single data source or clear the field to include all data sources. Click Save. Click Synchronize. To access the ZIP file: Open the Activities list. In the row containing the job, click Result. The Synchronization Results dialog box appears. Click Download.