Create a Databricks connection to an Edge or Collibra Cloud site

Prerequisites

In your Collibra environment

In your Databricks environment

Steps

1. Open a site.
   1. On the main toolbar, click → Settings.
      The Settings page opens.
   2. In the tab pane, click Edge.
      The Sites tab opens and shows a table with an overview of your sites.
   3. In the table, click the name of the site whose status is Healthy.
      The site page opens.
2. In the Connections section, click Create connection.
   The Create connection page appears.
3. Enter the required information.

   Field	Description	Required
   Connection settings	This section contains the general settings of your connection.
   Name	The name of the Edge connection for Databricks.	Yes
   Description	The description of the connection.	No
   Connection provider	The connection provider, which determines the available connection parameters. Select Databricks to connect to Databricks.	Yes
   Connection parameters	This section contains the settings to connect to your data source.
   Workspace URL	Enter the full URL of any Databricks workspace connected to Unity Catalog that you want to integrate. To retrieve the full URL, log into Databricks and copy the URL, including https://. For example: https://123.cloud.databricks.com.	Yes
   Access Token	The security token that was generated in Databricks for the workspace. The access token must be a personal access token (PAT). It is possible to generate a PAT for service principals. For information on the service principal token, go to the Databricks documentation. Note Ensure that your Databricks access token has been granted the required permissions in your Databricks environment.	Yes
   Encryption options	Select the type of encryption used to store the Secret Access Key. Default: To be encrypted by Edge management server.	Yes

4. Click Create.
   The connection is added to the Edge or Collibra Cloud site.
   

What's next

You can now add the Databricks Unity Catalog capability to an Edge or Collibra Cloud site.

Prerequisites

In your Collibra environment

• You either created and installed an Edge site or were granted a Collibra Cloud site.
You have added a vault to your Edge site.
Note  Vaults are not supported on Collibra Cloud sites.
If your data source connection requires a file from your vault, the file must be encoded into Base64 and stored as a regular secret in your vault.
• You have a global role that has the Manage connections and capabilities global permission, for example, Edge integration engineer.

In your Databricks environment

• Your Databricks access token or OAuth client must have the BROWSE permission on the catalogs in Databricks Unity Catalog from which you want to integrate metadata. For more information on the BROWSE permission, go to the Databricks documentation.

• If you want to integrate source tags, additional permissions are needed.
  • The metadata synchronization for Databricks Unity Catalog uses compute clusters (SQL query compute warehouse) to collect source tags. To allow this, grant the following permissions:
    • CAN ATTACH TO
    • CAN RESTART
    During the synchronization configuration, you can define that the compute clusters must stop after the source tags are extracted.
    

  • To integrate source tags from specific tables in system.information_schema, grant the following permissions:
    • USE CATALOG permission on system catalog
    • USE SCHEMA permission on system.information_schema
  • SELECT permission on the following:
    • system.information_schema.catalog_tags
    • system.information_schema.schema_tags
    • system.information_schema.table_tags
    • system.information_schema.column_tags

• If you want to integrate Databricks AI models, ensure that your Databricks access token or OAuth client also has the following permissions:

  • EXECUTE permission on the registered model.
  • USE CATALOG permission on the parent catalog.
  • USE SCHEMA permission on the parent schema.

Steps

1. Open a site.
   1. On the main toolbar, click → Settings.
      The Settings page opens.
   2. In the tab pane, click Edge.
      The Sites tab opens and shows a table with an overview of your sites.
   3. In the table, click the name of the site whose status is Healthy.
      The site page opens.
2. In the Connections section, click Create connection.
3. Select Databricks to connect to Databricks.
   The Create connection page appears.
4. Enter the required information.

   Field	Description	Required
   Name	The name of the Edge or Collibra Cloud site connection for Databricks.	Yes
   Description	The description of the connection.	No
   Vault	The vault where you store your data source values.	No
   Workspace URL	Enter the URL of any Databricks workspace connected to Unity Catalog that you want to integrate. To retrieve the URL, log into Databricks and copy the URL. For example: https://123.cloud.databricks.com. How to use your vault... To use your vault, do the following: In the Value Type field, select Vault Key. Enter the query value to identify the secret in your vault. Example  Note The Query must be a string containing the properties required to identify the secret. Each property must be separated by a semicolon (;). For example: Safe=<SafeName>;Folder=<FolderName>;Object=<ObjectName> If a property is a folder with sub-folders, use a backslash (\) to define the folder path. For example: Folder=Root\Top Secrets\More Secrets For more information about query formats and supported properties, go to the CyberArk Credential Provider documentation. To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Secret Engine Type Select one of the following: Key Value Database Engine Path The engine path to your vault where the value is stored. Secret Path The secret path to your vault where the value is stored. Field If your Secret Engine Type is Key Value, enter the name of the field to your vault where the value is stored. Role If your Secret Engine Type is Database, enter the role specified in the Database engine. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Vault Name The name of your Azure Key Vault in your Azure Key Vault service where the value is stored. Secret Name The name of the secret in your vault where the value is stored. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Secret Name The name of the secret in your vault where the value is stored. Field If the secret stored in your AWS Secrets Manager is a JSON value, for example {"pass1": "my-password", "pass2": "my-password2"}, then you need to specify the Field to point to the exact JSON value that should be used. For example, Secret Name: edge-db-customer; Field: pass. Note If the secret stored in your AWS Secrets Manager is a plain string value, for example my-password, then you do not need to specify the Field. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the name of the secret in your vault where the value is stored. Example	Yes
   Authentication Type	Select the type of authentication that you want to apply. You can select any of the following values: Personal Access Token OAuth For information on OAuth-based authentication in Databricks Unity Catalog, go to the Databricks documentation. Microsoft Entra ID For information, go to MS Entra service principal authentication in the Azure Databricks documentation.	Yes
   Access Token	The security token that was generated in Databricks for the workspace. The access token must be a personal access token (PAT). It is possible to generate a PAT for service principals. For information on the service principal token, go to the Databricks documentation. Note Ensure that your Databricks access token has been granted the required permissions in your Databricks environment. How to use your vault... To use your vault, do the following: In the Value Type field, select Vault Key. Enter the query value to identify the secret in your vault. Example  Note The Query must be a string containing the properties required to identify the secret. Each property must be separated by a semicolon (;). For example: Safe=<SafeName>;Folder=<FolderName>;Object=<ObjectName> If a property is a folder with sub-folders, use a backslash (\) to define the folder path. For example: Folder=Root\Top Secrets\More Secrets For more information about query formats and supported properties, go to the CyberArk Credential Provider documentation. To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Secret Engine Type Select one of the following: Key Value Database Engine Path The engine path to your vault where the value is stored. Secret Path The secret path to your vault where the value is stored. Field If your Secret Engine Type is Key Value, enter the name of the field to your vault where the value is stored. Role If your Secret Engine Type is Database, enter the role specified in the Database engine. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Vault Name The name of your Azure Key Vault in your Azure Key Vault service where the value is stored. Secret Name The name of the secret in your vault where the value is stored. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the required information: Secret Name The name of the secret in your vault where the value is stored. Field If the secret stored in your AWS Secrets Manager is a JSON value, for example {"pass1": "my-password", "pass2": "my-password2"}, then you need to specify the Field to point to the exact JSON value that should be used. For example, Secret Name: edge-db-customer; Field: pass. Note If the secret stored in your AWS Secrets Manager is a plain string value, for example my-password, then you do not need to specify the Field. Example  To use your vault, do the following: In the Value Type field, select Vault Key. Enter the name of the secret in your vault where the value is stored. Example	Yes, if you select Personal Access Token as the authentication type.
   Client ID	The client ID for OAuth-based authentication in Databricks, or the client ID of the Microsoft Entra ID service principal. For information on OAuth-based authentication in Databricks Unity Catalog, go to the Databricks documentation. For information on the Microsoft Entra ID service principal, go to Microsoft Entra service principal authentication in the Azure Databricks documentation. Note Ensure that your Databricks OAuth client or Microsoft Entra ID service principal has been granted the required permissions in your Databricks environment.	Yes, if you select OAuth or Microsoft Entra ID as the authentication type.
   Client Secret	The client secret generated for the OAuth-based authentication on Databricks, or the client secret of the Microsoft Entra ID service principal.	Yes, if you selectOAuth or Microsoft Entra ID as the authentication type.
   Tenant ID	The Directory (tenant) ID for the related application registered in Microsoft Entra ID. For information, go to MS Entra service principal authentication in the Azure Databricks documentation.	Yes, if you select Microsoft Entra ID as the authentication type.

5. Click Create.
   The connection is added to the Edge or Collibra Cloud site.
   

Note Collibra validates the credentials when synchronizing Databricks Unity Catalog.

What's next

If you want to allow for sampling, profiling, and classification, create a Databricks JDBC connection. If you created a Databricks JDBC connection, you can use that JDBC connection when you configure the synchronization page.

You can then add the Databricks Unity Catalog capability to an Edge or Collibra Cloud site.