Databricks AI assets

The Databricks Unity Catalog integration for AI in Collibra comes out of the box with your software.

The functions of the Databricks Unity Catalog AI integration is limited if you do not enable AI Governance. When you enable AI Governance, you can add custom attributes or relations to the Databricks AI asset.

Databricks object Asset type Description
Registered model in Unity Catalog AI Base Model Represents foundational business concepts that define the model’s purpose and governing principles in your Databricks AI workspace.
Model version (status READY only) Databricks AI Model Version

A subtype of AI Model Version that represents AI model versions in Databricks AI.

Served entity on a serving endpoint AI Model Deployment The operational instance where a model version is assigned computational resources to execute and generate real-time outputs.
Serving endpoint with an agent/* task; Knowledge Assistant; Genie Agent; Supervisor Agent; Agent Bricks tile (Information Extraction and Text Classification) AI Agent

An autonomous system that perceives its environment, makes decisions based on predefined rules or learned patterns, and takes actions to achieve specific goals.

Created for a Knowledge Assistant, Genie Agent (only if at least one of its tables is already in Collibra), Supervisor Agent, or Agent Bricks tile (Information Extraction and Text Classification) AI Agent Version

A specific, immutable implementation of the model logic and parameters that serves as the audited "raw" model version before it is activated.

Knowledge Assistant knowledge source; Genie Agent table reference; Supervisor Agent tool; Agent Bricks tile (Information Extraction and Text Classification) AI Agent Tool

A service accessible to an AI agent that automates tasks, generates content, or delivers intelligent insights.

Serving endpoint (serving an ingested model version); Knowledge Assistant or Supervisor Agent serving endpoint, connected to its agent version through the AI Agent Deployment complex relation AI Endpoint The access point for external systems that provides a consistent interface while allowing for the seamless exchange of underlying deployments.
Data quality monitor on the inference table; Scheduled scorer (LLM judge) AI Monitor The configuration of automated tracking parameters and alerting thresholds used to observe a deployment’s real-world performance and detect deviations from expected behavior.
Volume used by a knowledge source or tile Databricks Volume

Unity Catalog objects for governing non-tabular datasets.

Notebook used to train a model version (run input with source type code); a root folder / is created for each model version

Storage Container Represents the folder that contains the notebook file used to train your AI model versions in your Databricks AI workspace. This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.
Notebook used to train a model version (run input with source type code) File Represents the notebook file used to train your AI model versions in your Databricks AI workspace. This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.
Training dataset from a Delta table or inference table (AI Gateway), if not already present Database A collection of data that is systematically organized or structured to make it easy to create, update, and query the information. This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.
Training dataset from a Delta table or inference table (AI Gateway), if not already present Schema

An asset that contains the location of specific data. It provides all the details that are required for setting up a connection to a database or server.

This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.
Training dataset from a Delta table; inference table (AI Gateway) Table

An implementation of data entities in columns and rows, in a given database system. It is the basic structure of a relational database.

This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.
Training dataset from a Delta table or inference table (AI Gateway), if not already present Column

An atomic unit of data that can be stored in a database table.

This asset is optional, you can ingest this data during the integration by selecting the Ingest input datasets and deployment output for AI models and deployments checkbox.

Databricks Unity Catalog diagram view

The following image provides a comprehensive overview of the relations between Databricks Unity Catalog asset types and cardinality of the relation types in the assets' assignment.

Steps

  1. Open the asset page.
  2. Click the Diagram tab.
    The diagram is shown in the default diagram view.
  3. Click to add a new view.
  4. Select the Text option below the diagram view name.
    The diagram view text editor is shown.
  5. Copy the code from the Show JSON code section below and paste it in the diagram view text editor.
  6. Click Save.
  7. Edit the name and description of the diagram view as needed.