Release 2024.10

Data Catalog

• The AWS Sagemaker AI integration via Edge is now generally available. When you integrate Amazon SageMaker, you integrate the metadata of ML models from Amazon SageMaker AI to Collibra Data Intelligence Platform. The resulting assets represent the AWS SageMaker AI model. (idea #DCC-I-914)
• The Azure ML integration via Edge is now generally available. When you integrate Azure AI, you integrate the metadata of ML models from Microsoft Azure ML to Collibra Data Intelligence Platform. The resulting assets represent the Azure ML model. (idea #CAIG-I-4)

AI Governance

• You can now use the notifications feature in your assessment templates, to ensure that:
  • Assessment Owners are notified when a new assessment is started.
  • Assignees are notified when they are assigned to an assessment.
• To help foster collaboration when conducting assessments, you and your colleagues can now access a single assessment and edit and save responses to individual questions.

• A new "Copy Response to Assessed Asset" option in the assessment templates allows you to map assessment responses to characteristics on AI Use Case asset pages. When an assessment is approved, the responses for mapped fields are copied to their corresponding characteristics on the AI Use Case asset page.

Data Governance

• We're introducing an out-of-the-box asset model for Data Products. This includes the Data Product asset type and layout, as well as supporting asset types, characteristics, and the domain type Data Product Catalog. (idea #DCC-I-2716)

Data Notebook

• A new out-of-the-box relation type, "Data Notebook queries/is queried by Database," is introduced to link Data Notebook assets to the databases that were queried in notebooks. This relation is used when you publish a notebook, and it's shown on the Data Notebook asset page, so you can see which databases were queried.
• If you don't have permission to view the content of a published notebook, you can now request access using the new "Request Responsibility" button in the notebook or on the "Notebook" tab on the Data Notebook asset page.

Assessments

• You can now automatically send emails to individuals when they're set as owners or assignees of assessments using the new "Notifications" option in templates. This simplifies communication and ensures everyone involved is promptly informed. By default, notifications are enabled for Collibra templates and new custom templates, and disabled for custom templates created before the Collibra 2024.10 release. (idea #CDG-I-14)
• You can now configure a template field to copy its response in an assessment to a characteristic of the linked asset when the assessment is completed or approved. To facilitate this, the "Copy Response to Assessed Asset" setting is added to the sidebar on the "Template Designer" tab. (idea #DCC-I-2315)
• You can now enter and format rich text in an assessment using the new "Rich Text Box" field in a template. (idea #DCC-I-2214)
• You can now save or discard your response for each assessment field individually. If someone else saves their response in the same field first, you are notified. You can then choose to replace their response with yours or copy yours to the clipboard for later use. This prevents accidental overwriting and enables collaborative editing while ensuring you have control over your input. As a result of this feature:
  • You need to click within the response area to enter or edit a response, including permissions.
  • The "Edit", "Edit Permissions", and "Save" buttons are removed from assessments.

Edge

• Edge now supports custom namespaces for your Edge sites. You can now install multiple Edge sites under unique namespaces in the same Kubernetes cluster. (idea # DCC-I-2709, DIP-I-76)

Usage Analytics

• In the "At a glance" sidebar, you can now see the total page visits for an asset. Click "Show Details" to dive deeper into the analytics. You can check for trends over time and see who frequently visits the asset. (idea #DCC-I-117)
  See what this looks like

  

Miscellaneous

• The "License Change History" for the new license model in "Settings" → "Users and Subscriptions" → "Seats" → "License Change History" → "Daily" is now generally available and captures more details related to actions that impact the number of assigned seats, such as changes in:
  • Group memberships.
  • Global role membership.
  • Responsibilities.
  • Permissions for resource or global roles.
• You can now change the format of dates, times and numbers by selecting a locale. (idea #DGCPLAT-I-370)

Data Catalog

• You'll soon experience faster synchronization via Edge for JDBC data sources. This is thanks to the improved scalable ingestion flow, which is being rolled out region by region for commercial customers.
  The scalable ingestion flow improves performance by tracking changes on the Edge side and only sending the changed metadata to Collibra. (idea #CDC-I-124)
  For more information about the roll-out, contact your Collibra Account Team. For more information about the flow, go to About synchronizing data.
• The "URL"attribute type is available on the global assignment of the Table and Database view asset types. This update prepares for synchronizing the table location via Edge for AWS Glue data sources.
• The Databricks Unity Catalog integration now adds the value "Databricks Unity Catalog" in the "Data Source Type" attribute for Database and Schema assets.
• If you have migrated to Unified Data Classification and use the old data class ID in the REST API rest/catalog/1.0/dataClassification/classifications/<old-data-class-id>, you will receive the new data class ID. This update simplifies updating APIs after migrating to Unified Data Classification.
• You can now view data samples for Column assets containing dates and times in a Date and Time format.

AI Governance

• The Progress Tracker is now a widget that you can reposition on the AI Use Case asset page, to suit your content layout needs. For more information, go to Tailored asset pages and Edit your layout.
• With the Collibra 2024.08 release, we announced that:
  • You can customize the AI Use Case asset status progression, thereby customizing the lifecycle of AI Use Case assets.
  • Your configuration is reflected in the Progress Tracker.

  With this release, if you customize the asset statuses, the configuration is also reflected on the AI Governance landing page. For more information, go to Configure AI Use Case asset statuses.

• The AI Legal Reviewer global role is no longer an out-of-the-box global role. If any users in your organization are assigned the AI Legal Reviewer global role, they can continue using AI Governance without any adjustments.

Data Governance

• Widgets are now introduced in the asset page layout editor to allow you to show custom content on asset pages.
• The Global Create "Recent" tab now shows only the 5 most recently created asset types via Global Create within the past month. Previously, it showed up to 5 asset types no matter how long ago they were created. This change ensures quicker loading times even with large asset, attribute, and relation volumes and provides a more relevant list of asset types. This tab may be empty after this upgrade. Once you create new assets, the "Recent" tab will get populated again.
• You can now create, update, and find assets, attributes, relations, complex relations, and domains by publicId via REST and Java API. You can also retrieve asset types, attribute types, relation types, complex relation types, domain types, and scopes by publicId.
• In the latest UI, a "System-managed" column can now be seen on the type pages in the Operating Model settings. This option allows to differentiate between out-of-the-box and custom types.
• The "Add a Characteristic" dialog box in the Operating Model settings now has a filter for complex relation types, making it easier to find and select the desired type.
• The "Edit Asset Type" dialog box now shows a preview of the asset type's color, symbol, and acronym against the title bar background color to help you assess color contrast. (idea #DCC-I-3173)
• The "Activated" column in the asset types table has been replaced with a "Product" column, which indicates which product is required and whether it is active. If you previously had the "Activated" column selected, you will now see the "Product" column.
• The layout editor now allows you to directly add characteristics to assignments, so you don't have to leave the page anymore.
• You can now get printouts generated by Collibra that match the new user interface design, including the layout, styling, and newly introduced sections. (idea #DCC-I-2803)
• You can now select 2 key workflow buttons to appear prominently on the asset page. This allows your organization to prioritize and highlight the most important actions for your needs. (idea #DGCPLAT-I-515)
• The "Actions" button on community, domain, and asset pages is now shown as the icon and provides a drop-down menu with options to Edit, Move, or Delete.
• The "At a glance" sidebar now maintains its open or closed state when navigating between asset pages, providing a consistent user experience. (idea #DCC-I-3276)
• Key roles in the "At a glance" sidebar are now consistently preserved across different scenarios, including editing, viewing asset pages, and introducing out-of-the-box asset page layouts.
• There is now a list of accepted and restricted MIME types, preventing users from uploading forbidden file types and eliminating the need for error messages and retries.
• We've added placeholder text to sections with empty values on asset pages, providing more context and improving user experience. The message displayed varies depending on the user's role.
• When searching for specific communities, domains, assets, relations, and attributes, bolded text is shown in the search results, making it easier to locate specific terms within the results for improved navigation. (idea #DCC-I-2849)
• On asset pages, you can now show a relation in a list view in addition to a table or tile view.
• When adding a relation, filtering has been simplified and focuses on the display name. Filtering on full name is also still possible.
• The "History" tab of communities and domains and the /activities endpoint of the REST Core API v2 now reflect changes to view permissions. (idea #DCCPLAT-I-899)
• Rich Text attributes on asset pages now have indentation and alignment options.

Data Notebook

• You can now duplicate a notebook directly from the asset page using the new "Duplicate Notebook" option.
• You can now use Microsoft Entra ID to connect to Azure Databricks. (idea #CDC-I-115)

Data Privacy

• The "CCPA" and "GDPR" tabs on the Business Process asset pages are now replaced by the "Privacy" section on the "Summary" tab.

Protect

• In the "Data Access Rule" dialog box, once you select a data category or classification for a masking option, it no longer appears as a choice, preventing duplicate selections.

Workflow Designer

Note Workflow Designer enhancements become available with the upgrade of production environments.

• The CollibraWorkflow Designer is now FedRAMP authorized and available to U.S. public sector agencies. FedRAMP, or the Federal Risk and Authorization Management Program, is a security assessment and authorization standard for U.S. federal agencies. The Workflow Designer is a powerful visual tool for building and managing workflows, enabling teams to work together to create and use new workflows to automate business processes.

  Learn more: Workflow Designer Documentation.

• We have updated Groovy, which is used to compile and execute workflow scripts, from version 3 to 4.

Edge

• The HashiCorp --namespace parameter has been renamed to --vaultNamespace. This change supports the Edge global namespace parameter enhancement, which allows you to have custom namespaces for Edge sites within a shared cluster. If you add a new HashiCorp vault to your Edge site or want to update an existing HashiCorp vault, use --vaultNamespace for the namespace.
  Note If you have an existing HashiCorp vault, it will continue to work as expected. You only need to use the new --vaultNamespace parameter when you want to edit your existing vault.
• Edge sites installed on a bundled k3s cluster now support Red Hat Enterprise Linux 9, in addition to 8. (idea # DCC-I-1572).
• The Edge CLI is now the default tool for managing Edge sites. With the Edge CLI, you can:

  • Install an Edge site on either a bundled k3s or managed Kubernetes cluster.

  • Uninstall an Edge site.

  • Add and manage your Vault integrations.

  • Create and manage Shared Storage connections.

  • Retrieve detailed information about your Edge sites.

  Note The legacy Edge tool will be End of Life February 1, 2025. If you don’t currently have the Edge CLI installed, you must install it and update any automated processes by February 1, 2025.
• We now support the following versions for our managed Kubernetes clusters:

  • AKS 1.30

  • EKS 1.30

  • GKE 1.30

  • OpenShift 4.16

• We have improved the log and error messages for Edge site jobs that have timed out or been canceled.
• When an Edge site capability completes, Edge now removes the connection secrets. This change improves the security of Edge.
• When you uninstall an Edge site, Edge now deletes the namespace.
  Note This is only applicable to new Edge sites.
• New Edge sites installed on a bundled k3s cluster no longer need 500 GB of dedicated storage. If your Edge site is already installed on a bundled k3s cluster with 500 GB of dedicated storage, you do not need to do anything.
• We have improved the installation process for Edge sites installed on an OpenShift shared Kubernetes cluster:
  • We removed the restrictive group membership requirement for Security Context Constraints.

  • You can now specify a UID within Security Context Constraints (SCC) when you install an Edge site on an OpenShift Kubernetes cluster.

Usage Analytics

• The CSV file downloaded from the "Active Users" section now contains the following additional columns so you can see where users are spending most of their time in your Collibra environment:
  • Asset Visits
  • Diagram Visits
  • Dashboard Visits
  • Domain Visits
  • Community Visits
  • Number of Sessions

Collibra Console

• To reduce the size of the file and provide more relevant information, the dgc.log file no longer retains function calls from external libraries in the stack trace.

Security

• We have made some changes to improve the security of our platform and keep your data safe. We've deployed mandatory Cross-Site Request Forgery (CSRF) tokens. If you encounter any issues after this upgrade, such as problems with third-party integrations, please contact Collibra Support.

API

• The REST Core API v2 users/current/permissions endpoint is deprecated in favor of users/current/globalPermissions.
• We have improved the documentation of the /mappings/externalSystem/mappedResource/bulk endpoint of the REST Core API v2 and the error message in case the "mappedResourceId" does not exist in Collibra.
• We have improved the documentation of the getWorkflowDefinition method of the Java Workflow API v2 WorkflowEvent bean.
• The REST Import API v2 now returns a specific error message when a relation type identified by a fully qualified relation type name does not exist in Collibra.
• As part of the migration process from the Java Core API v1 to v2, a number of v1 methods are now removed.See what is changing

  Interface v1	Method v1	Change Removed: The method is no longer available and there is no replacement. Drop: The method is deprecated with no replacement and will be removed in the future. Convert: The method is deprecated and will be removed in the future, but there is a replacement available.	Interface v2	Method v2
  InputViewComponent	getColumnNames	Removed
  InputViewComponent	importCSV	Removed	ImporterApi	importCsvInJob
  InputViewComponent	simulateImportCSV	Removed	ImporterApi	importCsvInJob
  InputViewComponent	importExcel	Removed	ImporterApi	importExcelInJob
  InputViewComponent	simulateImportExcel	Removed	ImporterApi	importExcelInJob

  For a complete list of changes that impact Java API v1 methods, see the Java API v1 to v2 mapping on the Developer Portal.

Miscellaneous

• For Commercial customers who did not raise significant blockers, the latest UI will be enabled by default in production environments on October 27, 2024. Impacted customers have received notifying emails and can respond to the Customer Success team by October 15.
  Tip 

  • You can still revert to the classic UI after October 27.
  • For new customers, the latest UI is enabled by default.
  • For other customers, the latest UI remains disabled by default.
• Numbers and dates in tables now cut off instead of showing all content. Previously, all cell types, including dates and numbers, would show all content (wrap). To view the full number or date, expand the width of the column by dragging the divider between column headers.
  See what this looks like

  

• You can now see the "Responsibilities" tab of the user "Profile" or group page also in a table layout.
• In the Dashboard text widget, we've highlighted the toolbar component to switch between the Rich text editor and the HTML editor. The Rich Text editor now validates HTML tags, disabling the editor if unsupported tags are present.
• If, in the Browse pane, you click the link for a community, domain, asset view, or dashboard of which you are already on the page, the Browse pane now closes. Previously, when you clicked the link, the Browse pane remained open.
• Charts and tables within the bar chart widget element are now consistently ordered from highest to lowest.

Data Catalog

• You can no longer add attributes that conflict with the operating model to a view on an asset page.
• You can again find renamed data classes when you search in Collibra.
• The "Data Classification" page now refreshes the data classes correctly.
• The API to read classification matches in bulk now returns the correct total number of elements in the "total" field.
• The "Sampling" tab in the Table asset page no longer shows generic error messages if the corresponding Edge site is down.
• If you request sample data via Edge, we now report sooner that the "Request sample data" job is completed.

Data Governance

• You can no longer create domain type, asset type, attribute type, complex relation type, scope or status names containing only whitespace.
• We've fixed an issue where the import button in the migration settings menu required multiple clicks to initiate a new migration import. The button now functions correctly on the first click.
• We've fixed an issue in the operating model migration export UI, ensuring that the view selection tab accurately reflects the community/domain hierarchy as displayed in the organization browser.
• The publicId of the asset type is now shown instead of its UUID in the import summary for tailored asset page layouts, improving readability for users.
• The confirmation dialog box when moving asset types to a different parent asset type now accurately displays the correct asset type name, fixing a previous issue where the wrong name was shown.
• You can no longer add images to columns in a table view, preventing data-fetching errors. The ability to upload and drag-and-drop images has been removed from all table views.
  Tip You can still add an image to an attribute via the resource page.
• Community, domain, and asset names are now shown exactly as you save them, including any spaces.
• You can now enter a year greater than 2099 in date attributes of assets.
• Domain history now accurately reflects all workflow activity, including those initiated by start events, making it consistent across the history tabs of communities, domains, and assets.
• History entries related to imports are now organized under a dedicated "Import" heading.
• The "Restore default" button in the layout editor now correctly restores the layout to its default configuration.
• You now see the correct button style when only 1 workflow action is available. (idea #DIP-I-222)
• We've fixed an issue where keyboard navigation didn't properly save selected dates within attributes. Now, using the "tab" key correctly saves the selected date and moves to the next field.
• We've fixed issues related to Boolean attributes, ensuring don't have pre-selected values and allow you to select, deselect, or delete them without an error.
• We've improved the performance of retrieving relations using the GET /relations API by joining the representations and relations tables, eliminating the need for separate database requests per asset when countLimit is set to -1.
• We've prevented extremely large numbers within number attributes from overflowing by resetting them to a specific limit value and showing them in scientific notation. For example, 9223372036854776000 is shown as 9.2233720369e+18.
• We've removed an unnecessary warning when moving a domain from a sub-community to the parent community.
• Words are no longer split up when shown in asset previews, ensuring that entire words are displayed on a single line or moved to the next line if they don't fit. (idea #DIP-I-184)
• We've fixed an issue where imports would fail if the relation property column wasn't the first column in the import data. The import now successfully completes regardless of the column order.
• We've fixed an issue where creating a domain with an ID that already belongs to a community resulted in a server error (HTTP 500). The fix ensures the API now returns a clear client error (HTTP 400) with a specific message explaining the conflict.
• We've fixed an issue where adding a relation to a new table caused unexpected column changes after refreshing the page.
• We've fixed an issue with the page title to accurately display "Global View" when users access the Global View page, improving user clarity and navigation.
• The "Asset count" column of the "Stewardship" → "Tags" page now reflects the correct number of tags that were added via the REST Import API v2.
• We improved the error message in the hierarchy view, providing clearer information when a view contains a deleted relation type.

Data Marketplace

• The "Organization" tab on the "Data Marketplace Scope" page now shows all the domains, even if the total exceeds 100.

Data Notebook

• The "DB" Snowflake driver parameter is now compatible with Data Notebook.

Data Privacy

• Scopes no longer overlap on Metamodel and Admin communities.

Workflows

• Completing a workflow on a resource page that changes the same resource no longer requires a refresh to display the change.
• You can again use java.time.LocalDate in your Groovy scripts.
• You can again submit forms with date fields where the default value is an expression.
• Following a link to a task from an asset workflow notification email now opens the asset page with the task sidebar showing that task.

Workflow Designer

Note Workflow Designer fixes become available with the upgrade of production environments.

• Deploying a workflow that has a sequence flow with a condition expression created by using the condition builder no longer removes that condition.
• Using the "Add expression" function of the condition builder no longer produces unexpected results.
• The unsupported "Template" option is no longer available for the "Body" filed of the "Email properties" attribute of an email task.

Edge

• When running an Edge test connection, the full Stacktrace logs are now fully shown.
• When uploading a shared storage connection file via the Edge CLI, file names can now include white spaces and special characters.
• We fixed an issue where some Edge sites that were eligible for an upgrade could not be upgraded. With this fix, you can upgrade eligible Edge sites.
• We have fixed an issue where some Edge sites lost 1 CPU due to an outdated component. With this fix, this component has been removed from all Edge sites.

Search

• If an asset is in both your top 3 recently visited and top 3 matching lists, it now appears only once in the search suggestions to avoid duplicates.
• An error no longer occurs when searching for a text containing HTML entities (for example, Job_Reference).
• Using Command+Click on a resource in the search suggestions now opens the resource only in a new tab, without affecting the current tab.

API

• The synchronization endpoints of the REST Import API v2 return again an error when you start a synchronization process with the same "synchronizationId" as an already running synchronization process.
• Queries for the "Output Module" resource of the REST Core API v2 return again the correct number of entries.
• Queries for the "Output Module" resource of the REST Core API v2 no longer return duplicate results when sorting by a related asset display name.
• Successful SCIM PATCH calls no longer return a body if no "attributes" or "excludedAttributes" are specified.

Miscellaneous

• Some users created before 2023 can again change their profile language preference.
• You can now use the _blank attribute for links in text widgets on your dashboards.
• The "Hover" color customization for tabs now applies to both the text and the tab indicator. Previously, it only applied to the text.
• When you enter unsupported special characters in the "Alt Text" field of the "Insert Image" dialog box, an error message now appears and the "Save" button is disabled. Previously, unsupported special characters were not saved, and no error message appeared.

Beta features

• Collibra's dashboards have been redesigned, offering a new left-side menu with bookmarked dashboards at the top. Admins can now customize the dashboard order for everyone using a drag-and-drop functionality. Join the beta testing. (idea #DGCPLAT-I-1408, #DGCPLAT-I-1327, #DIP-I-141, #DCC-I-2075)
• The AWS Bedrock AI integration via Edge is now available in public beta. When you integrate Amazon Bedrock, you integrate the metadata of ML models from Amazon Bedrock AI to Collibra Data Intelligence Platform. The resulting assets represent the AWS Bedrock AI model. (idea #DCC-I-2756)
• You can now register your applications to obtain a client ID and client secret that are required to request an access token on the new "Settings" → "Manage OAuth (Beta)" page. This allows you to securely authenticate your applications to access Collibra public APIs without requiring individual user credentials. (idea #DGCPLAT-I-500)
• A new "HTTP task" component is now available in the Workflow Designer, allowing you to have your workflows securely interact with external REST APIs. (idea #DCC-I-2035)

  Learn more: HTTP task.

  Note Workflow Designer features become available with the upgrade of production environments.

• Data Notebook now supports the Teradata data source. (idea #DIP-I-75)

Collibra 2024.10.1

• Editing an asset view filter no longer removes existing filters.

Collibra 2024.10.2

• When you manually change the status of an asset in the UI, you can once again only select a status from the relevant assignment.

Collibra 2024.10.3

• We have improved the scalable ingestion flow to better handle deleted assets that are still present at the source.
• Importing a large number of assets and restarting your environment shortly afterwards, no longer causes issues with other jobs.

Collibra 2024.10.4

• Data attributes now display the correct date by properly accounting for time zone differences after selection.
• The "&&" operator of the REST Search API v2 works again as expected, excluding results that do not contain all the search terms.

Collibra 2024.10.5

• We have improved the security of our platform.
• An error no longer occurs when you remove a complex relation from an asset page.

Collibra 2024.10.6

• We reduced the loading time of certain asset pages.

Edge and Data Lineage updates

January 6, 2025

(data-lineage-2025.01.1) 

December 23, 2024

(data-lineage-2024.12.4) 

December 16, 2024

(data-lineage-2024.12.3) 

December 15, 2024

(collibra-edge-2024.10.70) 

December 9, 2024

(data-lineage-2024.12.2) 

December 8, 2024

(collibra-edge-2024.10.63) 

December 2, 2024

(data-lineage-2024.12.1) 

December 1, 2024

(collibra-edge-2024.10.56) 

Lineage Harvester (CLI and Edge)

• The Analyze Only option in the technical lineage Edge capabilities is now deprecated. The functionality is replaced by the new Processing Level drop-down list. This new mandatory setting gives you more control in configuring when metadata is harvested, analysed, and synchronised.
• When ingesting Snowflake data sources using SQL-API mode, the SQL queries to harvest the data are updated to be more lightweight.
• When creating a custom technical lineage via the CLI lineage harvester or Edge using the batch definition method, the lineage harvester now validates the syntax in your JSON files.

November 25, 2024

(data-lineage-2024.11.4) 

November 24, 2024

(collibra-edge-2024.10.49) 

Security

• We improved the security of integrations via Edge.

Metadata integrations

• You can now download the input metadata from the Databricks Unity Catalog Lineage "Synchronization Results" dialog box if you selected the "Save Input Metadata" checkbox when you created the Databricks Unity Catalog Lineage capability.
• When you synchronize a Vertex AI integration, you can now download the input metadata file from the Sync results dialog box on the Activities page. In order to achieve this, you must select the "Save Input Metadata" checkbox when you create a Vertex AI capability.
• When registering a Databricks data source, a new domain is now created for each new database and schema if no explicit domain mapping is provided.
  • Database domains follow the naming pattern "System domain name > Database name," while schema domains follow "Database domain name > Schema name."
  • Domain mappings take precedence over the creation of new domains.
  • Existing Database and Schema assets remain in their current domains.
• The Dataplex lineage can now handle GCS file names which contain wildcards.

November 18, 2024

(data-lineage-2024.11.3) 

November 17, 2024

(collibra-edge-2024.10.42) 

November 11, 2024

(data-lineage-2024.11.2) 

November 10, 2024

(collibra-edge-2024.10.35) 

November 4, 2024

(data-lineage-2024.11.1) 

November 3, 2024

(collibra-edge-2024.10.28) 

October 28, 2024

(data-lineage-2024.10.4) 

October 21, 2024

(data-lineage-2024.10.3) 

October 20, 2024

(collibra-edge-2024.10.14) 

October 14, 2024

(data-lineage-2024.10.2) 

October 13, 2024

(collibra-edge-2024.10.07) 

October 7, 2024

(data-lineage-2024.10.1)