About the Databricks Unity Catalog integration via Edge

Important 

Choose an option below to explore the documentation for the latest user interface (UI) or the classic UI.

Databricks Unity Catalog is a technical catalog that provides schema information for all the databases in your connected Databricks instances.

You can integrate the metadata of all or multiple databases from Databricks Unity Catalog into Collibra. The integrated assets are Databricks databases, schemas, tables, and columns. You can integrate Databricks Unity Catalog only via Edge.

Integrating Databricks Unity Catalog is the recommended way to work with Databricks Unity Catalog because it shows the hierarchy of the assets and allows you to set up sampling, profiling, and classification (in preview).

Use the Databricks Unity Catalog integration if:

  • You want to integrate multiple databases simultaneously and efficiently.
  • Databricks Unity Catalog is activated in your organization.

With the JDBC connector, you need to register each database individually.

Suppose that you want to add 10 databases and profile the data.

If using only the Databricks JDBC connector

  1. Create 10 JDBC connections.
  2. Add the required capabilities to each connection.
  3. Register and synchronize each database individually.

If using the Databricks Unity Catalog integration with profiling

  1. Create 2 connections: one for the integration and one for JDBC.
  2. Add the required capabilities to the JDBC connection.
  3. Integrate Databricks Unity Catalog.
    The resynchronization for the databases is managed through the Databricks Unity Catalog capability, and profiling is performed via the Database asset.

If you previously used a combination of integrating Databricks Unity Catalog and registering an individual Databricks database via the Databricks JDBC driver, and you want to switch to using only the integration, go to Switching to working only with Databricks Unity Catalog integration (in preview).

Tip  You can also integrate Databricks AI models via Edge. To do so, set up your connection permissions and then define the required configuration. AI Governance is not a prerequisite for this integration. However, you must enable AI Governance to access the AI Governance product pages and leverage features that help you visualize and govern the ingested AI-related data.

Supported table types

Key considerations

The Databricks Unity Catalog integration supports the following table types:

  • EXTERNAL
  • MANAGED
  • STREAMING_TABLE
  • VIEW

Since only the metadata is integrated, you can neither get sample data for tables and columns nor profile or classify them. If you want to do so, you need to register the Databricks database via the Databricks JDBC driver. For more information, go to combining the integration and JDBC driver.

Helpful resources

To learn more about the Databricks Unity Catalog integration, follow our University course.

For more information about Databricks, go to the Databricks documentation.