Steps overview: Create Custom technical lineage via Edge

The integration steps vary slightly depending on how you choose to connect to your JSON files.

Integrate via Shared Storage connection

# Step Description
1 Review the preflight checks. Key considerations to help ensure successful integration, including required Edge, technical lineage, and data source-specific permissions, network requirements and more.
2

Create a Shared Storage connection.

A Shared Storage connection allows you to grant your capabilities access to files from a shared folder.

Important Shared Storage connection is not supported for Collibra Cloud sites.
3 Define the custom technical lineage.

Define the technical lineage in one (single-file definition) or many (batch definition) JSON files, and then store them in the folder you created in the previous step.

For guidance and example file configurations, go to:
4

Add the Technical Lineage for Custom Technical Lineage capability for Shared Storage connections.

 Add the technical lineage capability to your Edge or Collibra Cloud site. The capability allows the lineage harvester to retrieve data from your data source.
5 Synchronize the technical lineage.

You can synchronize your technical lineage manually or automatically by adding a synchronization schedule.

Integrate via Cloud Storage connection

# Step Description
1 Review the preflight checks. Key considerations to help ensure successful integration, including required Edge, technical lineage, and data source-specific permissions, network requirements and more.
2

Create a Cloud Storage connection.

For guidance on how to create a connection between your cloud-based storage system and Edge or Collibra Cloud site, go to the appropriate topic:

  • Create an AWS connection to an Edge or Collibra Cloud site.
  • Create an Azure Data Lake Storage connection to an Edge or Collibra Cloud site.
  • Create a Google Cloud Platform connection to an Edge or Collibra Cloud site.

3 If you use an AWS connection to access your data source files, prepare S3 for the connection.

Depending on your security requirements and where your Edge site is hosted, you can choose EC2 or IAM authentication types. Both authentication types require you to configure S3 permissions before creating the AWS connection.
4 Define the custom technical lineage.

Define the technical lineage in one (single-file definition) or many (batch definition) JSON files.

For guidance and example file configurations, go to:
5

Add the Technical Lineage for Custom Technical Lineage (Cloud) capability for Cloud Storage connections.

 Add the technical lineage capability to your Edge or Collibra Cloud site. The capability allows the lineage harvester to retrieve data from your data source.
6 Synchronize your technical lineage.

You can synchronize your technical lineage manually or automatically by adding a synchronization schedule.

What's next

After you synchronize the technical lineage, you can view the ingestion report. This shows the impact of technical lineage synchronization on the assets in Collibra.

Helpful resources