Synchronize technical lineage for Databricks Unity Catalog

After you add the Technical Lineage for Databricks Unity Catalog capability, synchronize the technical lineage manually or automatically by adding a synchronization schedule.

Prerequisites

A global role that has the following global permission:

Steps

  1. On the main toolbar, click Products icon → Catalog.
    The Catalog homepage opens.
  2. On the main toolbar, click Plus icon.
    The Create dialog box appears.
  3. In the Register with Edge section of the Create dialog box, click Integration Configuration.
    The Integration Configuration tab page opens.
  4. Locate the Databricks connection that you used when you added the technical lineage capability, and click the link in the Capabilities column. If multiple capabilities exist for the Databricks connection, expand them to locate your technical lineage capability.
    The technical lineage capability configuration page opens.
  5. In the Configuration Section section, click Add Configuration.
  6. In the Configuration Section section, click Synchronize now.
    A notification indicates synchronization has started.
    The synchronization job is started. Collibra Data Lineage ingests the metadata from Databricks Unity Catalog and processes the metadata to create technical lineage.
  1. On the main toolbar, click Products icon → Catalog.
    The Catalog homepage opens.
  2. On the main toolbar, click Plus icon.
    The Create dialog box appears.
  3. In the Register with Edge section of the Create dialog box, click Integration Configuration.
    The Integration Configuration tab page opens.
  4. Locate the Databricks connection that you used when you added the technical lineage capability, and click the link in the Capabilities column. If multiple capabilities exist for the Databricks connection, expand them to locate your technical lineage capability.
    The technical lineage capability configuration page opens.
  5. In the Configuration Section section, click Add Configuration.
  6. In the Synchronization schedule section, click Add schedule.
  7. Enter the required information and click Save:
    FieldDescription
    RepeatThe interval when you want to synchronize automatically. The possible values are: Daily, Weekly, Monthly, and Cron expression.
    Cron

    The Quartz Cron expression that determines when the synchronization takes place.

    This field is only visible if you select Cron expression in the Repeat field.

    Every

    The day on which you want to synchronize, for example, Sunday.

    This field is only visible if you select Weekly in the Repeat field.

    Every first

    The day of the month on which you want to synchronize, for example, Tuesday.

    This field is only visible if you select Monthly in the Repeat field.

    At

    The time at which you want to synchronize automatically, for example, 14:00.

    • You can only schedule on the hour. For example, you can add a synchronization schedule at 8:00, but not at 8:45.
    • This field is only visible if you select Daily, Weekly, or Monthly in the Repeat field.
    Time zoneThe time zone for the schedule.

What's next

View the summary of an technical lineage for Databricks Unity Catalog synchronization.