Example | Implementing technical lineage for Databricks Unity Catalog via Edge

October 30, 2026

This use case uses Global Logistics as an example business scenario to demonstrate the value of technical lineage for Databricks Unity Catalog.

As the Global Logistics footprint expands, analysts and data engineers must be able to understand and trust the data published in Databricks Unity Catalog. Inventory, shipment, and fulfillment data is transformed across multiple pipelines, making it difficult to assess downstream impact when changes occur or to quickly identify the source of data issues.

Technical lineage provides end-to-end visibility into how logistics data flows from upstream sources to curated, business-ready “Gold-tier” datasets. This enables teams to perform impact analysis, accelerate root-cause diagnosis, and provide compliance teams with a traceable view of critical logistics assets.

Scenario

This use case assumes that the Databricks Unity Catalog integration is already complete and your physical assets are visible in the Data Catalog.

Your goal is to generate technical lineage that stitches to these existing assets. You will configure the Edge site to connect to the Collibra Data Lineage service via basic authentication. To maintain a high security posture, rather than granting Collibra direct access to the system.query.history tables, you will point Collibra Data Lineage to a dedicated custom catalog in Databricks where lineage-related views have been curated for governance.

Collibra Data Lineage extracts lineage for Databases, Schemas, Tables, Columns, metric views, Delta Live Tables, external Delta tables, Volumes (in preview), and Notebooks. For details, go to Databricks Unity Catalog: Supported transformation details.

In this guide, you will do the following:

1. Review and confirm that your environment meets the minimum requirements for Collibra Data Lineage.
2. Enable technical lineage via Edge in Collibra settings using basic authentication.
3. Add the technical lineage for Databricks Unity Catalog capability to your Databricks Workspace connection established during your initial metadata integration.
4. Synchronize the capability to generate technical lineage for your Gold-tier inventory data, automatically stitching it to your existing Catalog assets.

Prerequisites

On your local server

• You can confirm, or know who to reach out to in your organization that can confirm, that your server meets all system requirements outlined in this use case guide.
• You installed an Edge site.

Network

• Confirm that outbound traffic is allowed over HTTPS (Port 443).
• Use DNS names for all firewall rules. If static IPs are required, use a command-line utility such as nslookup to resolve current mappings, as IPs are dynamic.
• Allow access to all service instances in your geographic location. For example, for data in Europe, allow techlin-eu-central-1.collibra.com (AWS) and techlin-europe-west1.collibra.com (GCP).
• We highly recommend allowing the techlin-us-east-1.collibra.com instance. Edge uses this instance as a secondary endpoint for license verification if the primary regional instance is unavailable.

Within Collibra

• You created an Edge site.
• You have integrated Databricks Unity Catalog.
• You have the Catalog Authorglobal role with the following global permissions:
  • Catalog > Advanced Data Type > Add
  • Catalog > Advanced Data Type > Remove
  • Catalog > Advanced Data Type > Update
  • Catalog > Technical lineage
• To add the technical lineage for Databricks Unity Catalog capability:
  • A connection to Databricks exists in your Edge site.
  • You have a global role with the Product Rights > System administration global permission.
  • You have a global role that has the Manage connections and capabilities global permission, for example, Edge integration engineer.
  • You have the View Edge connections and capabilities global permission.
• You have a resource role with the Configure external system resource permission, for example, Owner.

Within Databricks

• The CAN USE permission on your SQL warehouse to only integrate source tags. If you also want the integration to stop the compute resource after extraction, the access token or OAuth client needs CAN MANAGE instead.
  You can create a dedicated compute resource for generating technical lineage, or use an existing compute resource with Unity Catalog support. For details, go to Compute permissions in Databricks documentation.
  

  To prevent the compute resource from running for the entire duration of the synchronization, you can also configure automatic stop settings in Databricks:

  • Compute cluster: Terminate after … minutes of inactivity
  • SQL warehouse (including serverless SQL warehouse): Auto Stop

  The setting ensures that compute resources automatically stop after a period of inactivity. For more information, go to the Databricks documentation for compute clusters or SQL warehouses.

• To maintain a high security posture, create a dedicated custom catalog in Databricks for governance purposes, for example, collibra_lineage_metadata. This ensures the Collibra service principal only interacts with the metadata required for lineage.
  

  Create a custom catalog with proper permissions

  If you prefer not to grant direct access to the system tables, create a custom catalog with views of the required system tables:

  1. Create a catalog and grant the following privileges: BROWSE, EXECUTE, READ VOLUME, SELECT, USE CATALOG, and USE SCHEMA.
     Show an example

     This example shows a custom catalog named demo with its associated privileges.

     
  2. Create two schemas in the catalog: access and query.
     Show an example

     
  3. Grant the following privileges on the access and query schemas: EXECUTE, READ VOLUME, SELECT, and USE SCHEMA.
     Some privileges might be inherited from the catalog privileges granted in step 1.
     Show examples

     This example shows the privileges for the access schema.

     

     This example shows the privileges for the query schema.

     
  4. In the access schema in the demo catalog, create the following views:
     • A column_lineage view based on the system.access.column_lineage table.
     • A table_lineage view based on the system.access.table_lineage table.
     Show an example

     
  5. In the query schema in the demo catalog, create a history view based on the system.query.history table.
     Show an example

     
  6. Grant the SELECT privilege on the following views created in steps 4 and 5: column_lineage, table_lineage, and history.
     Show examples

     

Enable technical lineage via Edge

This step activates Collibra Data Lineage on your Edge site and prepares it for stitching.

1. Open the Services Configuration tab:
   1. On the main toolbar, click → Settings.
      The Settings page opens.
   2. Click Services Configuration.
   3. Click Edit configuration.
2. In the Lineage on Edge section, enter the following information:
   • DGC user name: edge_system_user.
   • DGC user password: **************.
   • Collibra system name: False (default value).
3. Click Save all.

Add the technical lineage for Databricks Unity Catalog capability

1. Open a site.
   1. On the main toolbar, click → Settings.
      The Settings page opens.
   2. In the tab pane, click Edge.
      The Sites tab opens and shows a table with an overview of your sites.
   3. In the table, click the name of the site whose status is Healthy.
      The site page opens.
2. In the Capabilities section, click Add capability.
   The Add capability page appears.
3. Select the Technical Lineage for Databricks Unity Catalog capability template.
4. Enter the required information.
   
   • Name: Logistics_Inventory_Tech_Lineage
   • Description: Technical lineage for Gold-tier Logistics data, stitching to assets in the Databricks_Logistics_Production system.
   • Databricks Connection: Databricks_Logistics_Production. We use the same connection created for the Databricks Unity Catalog integration.
   • Compute Resource HTTP Path: /sql/1.0/warehouses/a1b2c3d4e5xxxxxx
   • Source ID: Databricks_Logistics_Production
   • TechLin Admin Connection (beta): Leave blank
   • Time Frame: 30
   • Property: Leave blank
   • Processing Level: Sync
   • Active: Checked
   • Save Input Metadata: Unchecked
   • Ingest lineage from external tables: Checked
   • Also ingest lineage from table_lineage: Checked
   • (Deprecated) Filters: Leave blank
   • Logging configuration: Leave blank
   • Memory (MiB): Leave blank
   • JVM arguments: Leave blank
   • Debug: false
   • Log level: Leave blank
5. Click Add.
   The capability is added to the Edge or Collibra Cloud site.
   The fields become read-only.

Manually synchronize your technical lineage

Sources

• Integrate Databricks Unity Catalog via Edge on k3s
• Steps overview: Create technical lineage for Databricks Unity Catalog via Edge

What's next

View the technical lineage graph.