Synchronize technical lineage for Matillion

To create a technical lineage for Matillion, use this information to synchronize your technical lineage manually or automatically by adding a synchronization schedule after you add the technical lineage capability.

When you synchronize a technical lineage capability, a full synchronization is run, which is equivalent to the full-sync command with the -s argument specified when you use the lineage harvester. You can synchronize as many technical lineage capabilities in parallel as you want. If a full synchronization is in progress, the other capabilities are put in a queue until a new full synchronization starts automatically.

Prerequisites

  • A global role that has the following global permission:
    • The Catalog, for example Catalog Author
    • View Edge connections and capabilities
  • A resource role with Configure external system resource permission, for example Owner.
  • The permissions to retrieve the metadata of the following database components through the JDBC Driver Database Metadata methods:
    • Schemas
    • Tables
    • Columns

Steps

  • Manually synchronize technical lineage for Matillion
  • Add a synchronization schedule
  1. On the main menu, click , and then click Catalog.
    The Catalog Home opens.
  2. On the main toolbar, click .
    The Create dialog box appears.
  3. In the Register with Edge section of the Create dialog box, click Register a data source.
    The Register content page opens.
  4. In the Connection name column, locate the Matillion connection that you used when you added the technical lineage capability. Then, click the capability name in the Data sources/Capabilities column of the Matillion connection.
    The technical lineage capability configuration page opens.
  5. On the Configuration Section tab pane, click Synchronize now.
    A notification indicates synchronization has started.
    The synchronization job is started. The tables and columns from the selected schema are ingested based on the configured table rules.
  1. On the main menu, click , and then click Catalog.
    The Catalog Home opens.
  2. On the main toolbar, click .
    The Create dialog box appears.
  3. In the Register with Edge section of the Create dialog box, click Register a data source.
    The Register content page opens.
  4. In the Connection name column, locate the Matillion connection that you used when you added the technical lineage capability. Then, click the capability name in the Data sources/Capabilities column of the Matillion connection.
    The technical lineage capability configuration page opens.
  5. On the Synchronization schedule tab pane, click Add schedule.
  6. Enter the required information and click Save:
    FieldDescription
    RepeatThe interval when you want to synchronize automatically, for example daily, weekly or based on a Cron expression.
    Cron

    The Quartz Cron expression that determines when the synchronization takes place.

    This field is only visible if you select Cron expression in the Repeat field.

    Every

    The day on which you want to synchronize, for example Sunday.

    This field is only visible if you select Weekly in the Repeat field.

    Every first

    The day of the month on which you want to synchronize, for example Tuesday.

    This field is only visible if you select Monthly in the Repeat field.

    At

    The time at which you want to synchronize automatically, for example 14:00.

    This field is only visible if you select Daily, Weekly or Monthly in the Repeat field.

    Time zoneThe time zone for the schedule.

What's next?

You can view a summary of the results from the Activities list to see whether the technical lineage is synchronized successfully.

If the synchronization fails or completes with errors, you can use the technical lineage via Edge troubleshooting guide or Collibra Support Portal to fix the errors.