Edit global data source registration settings

In Collibra Platform, you can create a data profile when you register a data source.You can configure global settings of data source registration.

Prerequisites

  • You have the ADMIN or SUPER role in Collibra Console.

Steps

  1. Open the DGC service settings for editing:
    1. Open Collibra Console.
      Collibra Console opens with the Infrastructure page.
    2. In the tab pane, expand an environment to show its services.
    3. In the tab pane, click the Data Governance Center service of that environment.
    4. Click Configuration.
    5. Click Edit configuration.
  2. In the section Register data source, make the necessary changes.
    SettingDescription
    Table types to ignoreA comma separated list of table types that are not ingested. For example, INDEX and SEQUENCE.

    AWS regions restriction

    A list of AWS regions that Data Catalog is allowed to connect to. For example, eu-west-3 and us-east-2. For a list of all AWS locations, see the AWS documentation.

    • If you want to allow Collibra to make a connection to any AWS region, leave the field empty.
    • If you remove a region from this list and that region was previously used for an S3 integration, you may want to delete the Glue database from the previously used region manually. By default, Collibra does not remove it. The Glue database has the following naming convention: collibra_catalog_<Asset Id>_<Domain Id>
      For example: collibra_catalog_d3174a88-5ffe-4d50-8fbe-7bf0832ec3af_5d198ce9-4e56-4d0e-a885-58204da50741
    • When using Edge, a warning is added to the logs if an invalid region is detected in the restricted regions list.
    AWS API call rate

    Allowed number of AWS API calls per second.

    Use this option to limit the number of API calls per second to prevent throttling errors from the AWS API.

    AWS fetch batch size

    Number of assets in a single batch that are fetched from a Glue database and ingested into Collibra.

    Integration configuration data refresh schedule

    CRON schedule that specifies when to refresh configuration options from the external system for integrations that support this functionality.

    Database registration via Edge

    An option to enable database registration via Edge.
    The available options are:

    • True: You can register a data source via Edge or Jobserver.
      Enabling data source registration via Edge doesn't prevent you from registering a data source via Jobserver as well.
    • False: You can register a data source via Jobserver only.

    Warning Jobserver and all related Jobserver integrations reached their End of Life in commercial environments in October, 2024. In Collibra Platform for Government and Collibra Platform Self-Hosted environments, they will reach their End of Life on May 30, 2027.
    For information on registering a data source via Edge, go to Registering and synchronizing a data source via Edge.

    Maximum number of concurrent Edge jobs

    This setting specifies the maximum number of Edge jobs that can run simultaneously to connect to a data source, update a schema list, or synchronize a schema. If the maximum number is reached, subsequent jobs are queued.
    The default value is 10. You can enter a value from 1 to 20.
    This setting applies only to JDBC data sources.

    Data Usage Time-Range Parameter

    This setting requires the SUPER role.

    The time range, in days, that data usage uses to calculate the score.

    Consumer threads in Catalog (Requires restart)

    This setting requires the SUPER role.

    Determines the number of threads used in Catalog to process results when the new JDBC metadata delta ingestion via Edge is enabled.

    Maximum number of ingestion rules

    The setting specifies the maximum number of synchronization rules that can be defined per schema.
    The default value is 10. You can enter a value from 10 to 200.
    This setting applies only to JDBC data sources.

  3. Click Save all.