This topic explains how source IDs and data sources work in technical lineage. It also clarifies common misconceptions that can lead to unexpected lineage results.
What a source ID is
A source ID is the unique identifier for a data source in technical lineage. You specify the source ID when you configure a lineage capability. Every piece of lineage data is associated with the source ID, not the lineage capability. This distinction affects what happens when you change, delete, or reuse a source ID.
What happens when you change a source ID
If you change the source ID in an existing lineage capability and synchronize, Collibra Data Lineage treats the new value as a new data source. It does not rename or replace the existing data source. Instead, the original source ID and its lineage remain until you delete them.
As a result, your lineage contains two data sources: the original one and the new one. If your goal is to rename a source, follow these steps:
- Update the source ID in the lineage capability.
- Synchronize the lineage capability.
- Delete the lineage for the old data source if you no longer need it.
If you change this value, the next synchronization adds another data source to the lineage instead of renaming the existing one. To change the source ID, delete the lineage for the existing data source first, then configure the new source ID.
For more information, go to About source IDs and the data source lifecycle.
We recommend that you do not change the source ID after the initial configuration. For BI data sources, changing the source ID can cause synchronization to fail. If you need to change the source ID, first delete the lineage for the existing data source, and then configure the new source ID. For more information, go to About source IDs and the data source lifecycle.Lineage data is replaced, not accumulated
Lineage is not incremental within a single data source. Each successful synchronization replaces all previously harvested lineage for that source. It does not merge or append new lineage to the existing data. If an object is no longer present in the latest analysis, it is removed from the lineage.
This behavior ensures that the lineage always reflects the current state of the data source rather than accumulating outdated lineage over time.
How data sources are merged during synchronization
While lineage within a single data source is replaced on each synchronization, lineage across multiple data sources is merged. During the Sync stage, Collibra Data Lineage combines the most recent successfully harvested and analyzed lineage from each active data source into a single lineage graph.
This behavior has an important consequence: if one active data source fails during the Sync stage, synchronization is blocked for all active data sources. To resume synchronization, resolve the issue with the affected data source or ignore it. For more information, go to technical lineage admin options.
If a data source fails during Load or Analyze, its previously successfully analyzed lineage remains available and is used in the next Sync until a newer successful analysis replaces it.
For details of each stage, go to Tips for successful lineage synchronization.
What happens when you delete a technical lineage capability
Deleting a lineage capability does not delete the associated data source or its lineage. The lineage remains available because it is associated with the data source, not the lineage capability. After a data source has been successfully analyzed, it continues to exist until you explicitly delete or ignore it.
To remove the data source and its lineage, use the Technical Lineage Admin capability.