Synchronize technical lineage for Fivetran

After you add the Technical lineage for Fivetran capability, synchronize the technical lineage manually or automatically by adding a synchronization schedule.

Prerequisites

A global role that has the following global permission:

  • The Catalog, for example Catalog Author
  • View Edge connections and capabilities

Steps

  1. On the main toolbar, click Products icon → Catalog.
    The Catalog homepage opens.
  2. In the tab bar, click Integrations.

    The Integrations page opens.
  3. Click the Integration Configuration tab.
  4. In the Integration column, find the technical lineage capability that you created, and click the capability name.
    The Integration configuration page for the capability opens.
  5. In the Synchronization Configuration section, click the Edit icon.
  6. Complete the field as needed.
    FieldAction
    Options updated: <timestamp>

    Options updated: <timestamp> next to Synchronization configuration indicates the last time the data was retrieved.

    A scheduled refresh job is automatically created when you complete the capability configuration. You can also manually refresh the metadata by clicking Options updated: <timestamp>. The scheduled refresh job appears on the Jobs dashboard of your Edge site with the same name as your lineage capability.

    If two jobs with the same name appear on the Jobs dashboard, one is the scheduled refresh job and the other is the lineage synchronization job. By default, the scheduled refresh job runs daily at 23:00 server time. Unlike the lineage synchronization job, the scheduled refresh job appears only on the Jobs dashboard and not in the Activities list.

    This action prepopulates the Destination name and type field.

    Destinations

    Specify one or more Fivetran destination names and types to collect metadata from.

    When you specify a destination, metadata is collected only for the connections associated with that destination.

    If you specify this field and also configure the Source Configuration field of the capability, only data objects from the destinations defined in both places are stitched to assets in Collibra Data Catalog.

    To add a destination, complete the following steps:

    1. Click Add Destination name and type.
    2. In Destination name and type, select the destination you want to collect metadata from. You can add multiple destinations.

      This field is prepopulated with the destination names and types available in your Fivetran environment. Loading the destination names and types into the dropdown list can take some time.

      Technical lineage will include lineage only for the destinations you specify.
  7. Click Save.
  8. In the Configuration Section section, click Synchronize now.
    A notification indicates synchronization has started.
    The synchronization job is started. Collibra Data Lineage ingests the metadata from Fivetran and processes the metadata to create technical lineage.
  1. On the main toolbar, click Products icon → Catalog.
    The Catalog homepage opens.
  2. In the tab bar, click Integrations.

    The Integrations page opens.
  3. Click the Integration Configuration tab.
  4. In the Integration column, find the technical lineage capability that you created, and click the capability name.
    The Integration configuration page for the capability opens.
  5. In the Synchronization Configuration section, click the Edit icon.
  6. Complete the field as needed.
    FieldAction
    Options updated: <timestamp>

    Options updated: <timestamp> next to Synchronization configuration indicates the last time the data was retrieved.

    A scheduled refresh job is automatically created when you complete the capability configuration. You can also manually refresh the metadata by clicking Options updated: <timestamp>. The scheduled refresh job appears on the Jobs dashboard of your Edge site with the same name as your lineage capability.

    If two jobs with the same name appear on the Jobs dashboard, one is the scheduled refresh job and the other is the lineage synchronization job. By default, the scheduled refresh job runs daily at 23:00 server time. Unlike the lineage synchronization job, the scheduled refresh job appears only on the Jobs dashboard and not in the Activities list.

    This action prepopulates the Destination name and type field.

    Destinations

    Specify one or more Fivetran destination names and types to collect metadata from.

    When you specify a destination, metadata is collected only for the connections associated with that destination.

    If you specify this field and also configure the Source Configuration field of the capability, only data objects from the destinations defined in both places are stitched to assets in Collibra Data Catalog.

    To add a destination, complete the following steps:

    1. Click Add Destination name and type.
    2. In Destination name and type, select the destination you want to collect metadata from. You can add multiple destinations.

      This field is prepopulated with the destination names and types available in your Fivetran environment. Loading the destination names and types into the dropdown list can take some time.

      Technical lineage will include lineage only for the destinations you specify.
  7. Click Save.
  8. In the Synchronization schedule section, click Add schedule.
  9. Enter the required information and click Save:
    FieldDescription
    RepeatThe interval when you want to synchronize automatically. The possible values are: Daily, Weekly, Monthly, and Cron expression.
    Cron

    The Quartz Cron expression that determines when the synchronization takes place.

    This field is only visible if you select Cron expression in the Repeat field.

    Every

    The day on which you want to synchronize, for example, Sunday.

    This field is only visible if you select Weekly in the Repeat field.

    Every first

    The day of the month on which you want to synchronize, for example, Tuesday.

    This field is only visible if you select Monthly in the Repeat field.

    At

    The time at which you want to synchronize automatically, for example, 14:00.

    • You can only schedule on the hour. For example, you can add a synchronization schedule at 8:00, but not at 8:45.
    • This field is only visible if you select Daily, Weekly, or Monthly in the Repeat field.
    Time zoneThe time zone for the schedule.

What's next

View the summary of an technical lineage for Fivetran synchronization.