Delete the technical lineage of a data source

You can delete the technical lineage of a data source if you no longer want to see it in the technical lineage graph.

Note  You always need at least one source in your lineage harvester configuration file.

Before you begin

Install the lineage harvester 2023.03 or newer.

Steps

  1. In the lineage harvester folder, open your lineage harvester configuration file.
  2. Delete the section with connection properties of the data source you no longer want to see a technical lineage for.
  3. Save the configuration file.
  4. Start the lineage harvester in the console and run the following command to ignore the data source:
    • For Windows: .\bin\lineage-harvester.bat ignore-source <source_ID>.
    • For other operating systems: ./bin/lineage-harvester ignore-source <source_ID>.
    Note <source_id> is the ID of the data source that you want to ignore. You can specify only one source ID, and the source ID must not contain any spaces. If your source ID includes spaces, you can use the lineage harvester pre-release version 2023.04-0-4 or newer as a workaround. With this version, you can enclose the source ID with spaces in double or single quotation marks, for example ignore-source "Source A".
    The data source is excluded from the list of data sources that are used to create the technical lineage.
  5. Synchronize the technical lineage by running any of the following commands:
    • The sync command:
      • For Windows: .\bin\lineage-harvester.bat sync
      • For other operating systems: ./bin/lineage-harvester sync
    • The full-sync command:
      • For Windows: .\bin\lineage-harvester.bat full-sync
      • For other operating systems: ./bin/lineage-harvester full-sync

    For more information, go to Lineage harvesting app command options and arguments.

  6. When prompted, enter the password to connect to your Collibra Data Intelligence Cloud and data sources in the configuration file.
    7. The lineage harvester uploads the metadata of the remaining data sources in the configuration file to the Collibra Data Lineage service.
    The Collibra Data Lineage service synchronizes the technical lineage and removes the deleted data source from the technical lineage graph.

What's next

You can view the technical lineage. For more information, go to Technical lineage viewer.

You can check the progress of the technical lineage creation in Activities in your Collibra Data Intelligence Cloud environment. The Results field indicates how many relations were imported into Data Catalog. Go to the status page to see the log files of the SQL analysis.

If the lineage harvester log shows an error message or the harvesting process fails, you can use the technical lineage troubleshooting guide or Collibra Support Portal to fix the error.