Configuring a new integration

This section shows you how to use the Integration configuration wizard to set up a new integration of Collibra Data Quality & Observability metadata from profile findings, rules, and alerts into Collibra Data Intelligence Platform.

Important If you have multiple integrations set up, only one can be active at a given time.

Note This is a one-time configuration per environment.

Prerequisites

For the Integration API to run successfully, ensure that you review and meet the following criteria.

  • Your Collibra DQ version is 2023.07 or later and Collibra Data Intelligence Platform version is 2023.05 or later.
  • You have an Admin role in Collibra Data Quality & Observability to use the configuration wizard.
  • You have an Admin or Dataset Manager role in Collibra Data Quality & Observability to enable or disable individual datasets for integration.
  • Your Collibra Data Intelligence Platform profile must have Sysadmin and DataSteward global roles to manage all assets and objects of the integration.
  • You have an aggregation path set for your Data Quality Rule in Collibra Data Intelligence Platform Settings Operating model Data quality rules.

Important Do not modify the cardinality of out-of-the-box (OOTB) operating model relations that impact the integration.

Steps

  1. Hover your cursor over the icon and click Integrations.
    The Integrations page opens.
  2. In the upper right corner, click Add Integration.
    The Integration Setup wizard appears.
  3. On the Credentials step, enter the required information.
    4. Option Description Required
    Service Type

    The application with which to integrate Collibra Data Quality & Observability metadata.

    This value is always Data Intelligence Cloud.

    Yes

    Service URL

    The URL of the Collibra Data Intelligence Platform environment to send Collibra DQ metadata. For example, "https://dg-example.collibra.com".

    Yes

    Authentication Mode

    The method of authenticating the API call to Collibra Data Intelligence Platform.

    You can select either Basic or OAuth.

    Yes

    OAuth Token Endpoint

    The Collibra Data Intelligence Platform URL where Collibra Data Quality & Observability sends requests to obtain the OAuth access token. For example, "https://dg-example.collibra.com/rest/oauth/v2/token"

    Yes

    Tip Required when using OAuth as the authentication mode.
    Service Account The username of your Collibra Data Intelligence Platform environment. For example, "integration.user" or "Admin".

    Yes

    Tip Required when using Basic as the authentication mode.
    Password

    The password of your Collibra Data Intelligence Platform environment.

    Yes

    Tip Required when using Basic as the authentication mode.
    Client ID The public identifier provided by your OAuth service provider.

    Yes

    Tip Required when using OAuth as the authentication mode.
    Client Secret The credential string from your OAuth service provider used to verify the identity of Collibra Data Quality & Observability during the token request process.

    Yes

    Tip Required when using OAuth as the authentication mode.

  4. Click Next.
    5. The Connections step opens.
  5. On the Connections step, map your Collibra DQ connections to the appropriate database, schema, table, and column assets in Collibra Data Intelligence Platform.
    1. From the Select Connection dropdown menu, select a Collibra DQ database to integrate into Collibra Data Intelligence Platform.
    2. In yourCollibra Data Intelligence Platform environment, open the asset page of the database you intend to map, then copy the URL from the UUID of the database asset from the URL.
      3. database asset page uuid example screenshot
    3. From the Connections tab in Collibra DQ, paste the copied UUID from Collibra Data Intelligence Platform, or enter the UUID manually.
    4. Click Run to create an automatic mapping between the schemas within your selected Collibra DQ database and the schemas within Collibra Data Intelligence Platform.

      5. Important 
      Only schemas ingested in Collibra Data Intelligence Platform return when you click Run.

    5. When the connection loads, click Show Full Mapping to expand the entire database mapping, including schemas, tables, and columns.

        Important 
        It is critical that you expand all schemas, tables, and columns you intend to map. If you do not expand each item down to the column level, your table and column relations will not be present in your integration.

      1. In the DGC Name column, optionally click the dropdown menu and select a different schema ( schema icon ) to update the schema mapping.

        2. Note When a name match cannot be made, a "Not Configured" message displays.

        Tip Optionally click More icon, then Auto Map to map tables in your DQ schema to those in DGC automtically. You can track the mapping status or cancel an in-progress mapping on the Mapping Status step.

      2. In the DQ Name column, click the schema name to expand the available tables.
        3. The selected schema expands to show the available tables, and their corresponding tables ( table icon ) appear across from them in the DGC Name column.
      3. In the DGC Name column, click the dropdown menu and select a different table to update the table mapping, if necessary.
      4. In the DQ Name column, click the table name to expand the available columns.
        5. The selected schema expands to show the available columns, and their corresponding columns ( column icon ) appear across from them in the DGC Name column.
      5. In the DGC Name column, click the dropdown menu and select a different column to update the column mapping, if necessary.
        6. schema, table, and column mapping example
    6. Optionally click Add New to map another connection and then repeat this process.
  6. Click Next.
    7. The Mapping Status step opens.
  7. On the Mapping Status step, optionally review mapping details.

    8. Mapping Status allows you to view mapping details, such as the number of available tables in a schema are mapped and when a connection mapping fails.

    Column Description
    Connection Name The name of the data source included in the mapping.
    Schema Name The name of the schema included in the mapping.
    Status The state of completion of the mapping.
    Start Time The time stamp, in yyyy-MM-dd hh:mm:ss format, of when the mapping began.
    End Time The time stamp, in yyyy-MM-dd hh:mm:ss format, of when the mapping stopped.
    Tables Mapped The number of tables that mapped out of the total tables available in the schema.
    Exception

    When an error during the mapping occurs, a message describes the issue. If the error message is truncated, you can hover your cursor over it to view the full message in a tooltip.

    When an exception did not occur, the value in this column is N/A.
    Actions When a mapping job is in progress, a Close icon option appears in this column to allow you to cancel the job if necessary.

    screenshot of mapping status step

  8. Click Next.
    9. The Tenants step opens.
  9. On the Tenants step, enter the required information.
    1. Optionally select whether to align your Collibra DQ tenant with an existing Collibra Data Intelligence Platform community.
      1. Align
        • Select a Collibra Data Intelligence Platform community from the dropdown menu with which to align your tenant.
      2. Don't Align,
        • Enter the name of a new community with which to align your tenant.
      2. tenant mapping screenshot

    Note Beneath each tenant-level community, a sub-community is created for each Business Unit that has datasets assigned to it in Collibra Data Intelligence Platform. If a dataset is not assigned to a Business Unit, it is added to a community called Unassigned DQ Job by default.

  10. Click Next.
    11. The Dimensions step opens.
  11. On the Dimensions step, enter the required information.
    1. Select a dimension from the dropdown menu of the Data Intelligence Platform Dimension column that corresponds with the Data Quality Dimension you want to identify as a match.

      2. Note All Data Quality Dimensions from your Collibra Data Intelligence Platform environment are available to map to Data Quality Dimensions in Collibra DQ, including custom dimensions. However, Data Quality Dimensions from Collibra DQ are not customizable at this time.

      dimension mapping screenshot
  12. Click Next.
    13. The Layers step opens.
  13. On the Layers step, select which layers to synchronize to Collibra Data Intelligence Platform.
    1. Select a layer in the Sync on Integration column to automatically create a new asset in Collibra Data Intelligence Platform when the integration runs.
    2. Select the Sync on Breaks option to create a new asset in Collibra Data Intelligence Platform only when the layer has a finding.
  14. Click Finish.

Known limitations

The integration only processes the primary schema or table and their associated columns. This is because the columns of the Collibra Data Intelligence Platform table used by the integration map to the Collibra Data Quality & Observability rules that contain them. A possible workaround to map columns from other tables referenced in the load query is to use a view to join multiple tables, then use that view as the Collibra Data Quality & Observability data source.

What's next?

Enable a dataset for integration and start ingesting metadata from Collibra DQ into Collibra Data Intelligence Platform.