About the Databricks Unity Catalog integration via Edge

Databricks Unity Catalog is a technical catalog that provides schema information for all the databases in your connected Databricks instances.

You can integrate the metadata of all or multiple databases from Databricks Unity Catalog into Collibra. The integrated assets are Databricks databases, schemas, tables, and columns. You can integrate Databricks Unity Catalog only via Edge.

Integrating Databricks Unity Catalog is the recommended way to work with Databricks Unity Catalog because it shows the hierarchy of the assets and allows you to set up sampling, profiling, and classification (in preview).

Suppose that you want to add 10 databases and profile the data.

If using only the Databricks JDBC connector

  1. Create 10 JDBC connections.
  2. Add the required capabilities to each connection.
  3. Register and synchronize each database individually.

If using the Databricks Unity Catalog integration with profiling

  1. Create 2 connections: one for the integration and one for JDBC.
  2. Add the required capabilities to the JDBC connection.
  3. Integrate Databricks Unity Catalog.
    The resynchronization for the databases is managed through the Databricks Unity Catalog capability, and profiling is performed via the Database asset.

If you previously used a combination of integrating Databricks Unity Catalog and registering an individual Databricks database via the Databricks JDBC driver, and you want to switch to using only the integration, go to Switching to working only with Databricks Unity Catalog integration (in preview).

Tip  You can also integrate Databricks AI models via Edge. To do so, set up your connection permissions and then define the required configuration.

You can configure Edge connections and capabilities without an active AI Governance license. However, AI Governance must be enabled to harvest AI model metadata, ingest corresponding AI assets in Data Catalog, and access the dashboards and features necessary to visualize and govern your AI landscape.

Supported table types

The Databricks Unity Catalog integration supports the following table types:

  • EXTERNAL
  • MANAGED
  • STREAMING_TABLE
  • VIEW

Helpful resources

To learn more about the Databricks Unity Catalog integration, follow our University course.

For more information about Databricks, go to the Databricks documentation.