Release 2023.11

• Release information
• Highlights
• Metamodel changes
• New features
• Enhancements
• Fixes
• Beta features
• Collibra maintenance updates
• Edge updates

Release information

• Publication dates:
  • Release notes: October 19, 2023
  • Documentation Center: November 2, 2023
• Release date of Collibra 2023.11.0: November 5, 2023
  • Upgrade non-production environments: November 5, 2023
  • Upgrade production environments: November 19, 2023
• Release date of Collibra 2023.11.1: November 19, 2023
• Release date of Collibra 2023.11.2: November 19, 2023
• Release date of Collibra 2023.11.3: December 10, 2023
• Release date of Collibra 2023.11.4: December 10, 2023
• Release date of Collibra 2023.11.5: December 20, 2023 (Collibra Cloud for Government only)
• Release date of Collibra 2023.11.6: On demand only
• Release date of Edge 2023.11.1: November 5, 2023
• Release date of Edge 2023.11.2: November 12, 2023
• Release date of Edge 2023.11.3: November 19, 2023
• Release date of Edge 2023.11.4: November 26, 2023
• Release date of Edge 2023.11.5: December 3, 2023
• Release date of Edge 2023.11.6: December 10, 2023
• Release date of Edge 2023.11.7: December 17, 2023
• Release date of Edge 2023.11.8: January 7, 2024
• Release date of Edge 2023.11.9: January 14, 2024
• Release date of Edge 2023.11.10: January 21, 2024
• Release date of Edge 2023.11.11: January 28, 2024
• Relevant Jobserver version: 2023.11.0-49

Highlights

• Data Catalog
• • The feature to register source tags from Snowflake is now generally available for Cloud customers. With this general availability, we improved the way these tags are displayed.

• Data Marketplace
• • The feature to manage personal and group-specific filters in Data Marketplace is now generally available. With this general availability, we now allow one user group to have multiple assigned group-specific filters. (idea #DCC-I-1080)
  • You can improve the search experience in Data Marketplace by tailoring the available filters in the Filters pane. (idea #DCC-I-1580)
    You can now:
    • Change the display name of filters including the out-of-the-box filters.
    • Add descriptions to filters, which will appear as tooltips in the Data Marketplace Search Results page.
    • Define whether a filter should be expanded or collapsed by default.
  • The shopping basket feature has been enhanced. (idea #DCC-I-219, DCC-I-2329, CAT-I-282)
    • You can now show or hide the shopping basket in your entire environment.
      The existing option to show or hide the shopping basket in Data Marketplace now also impacts the visibility of the shopping basket in other areas of your environment, such as Data Catalog.
      • To make sure that the shopping basket options can be used by customers who are not using Data Marketplace, we moved these options to their own section in the Data Marketplace Settings page.
      • The Data Marketplace Settings now include an overview page of all the settings and useful links to documentation, university courses, and demos.
    • You can now also define which asset types you want to allow in the shopping basket. By default, the shopping basket allows reports and data sets.

      Important When you allow extra asset types, the Request Assets Access workflow must be updated accordingly.

• Data Privacy
• • A new version of Data Privacy is now released with the following changes:
    • The Business User Interface is now deprecated. As a result, the interface is no longer accessible and the related workflow is removed from the CMA files for 2023.11. The Business User Interface was an external web page for Business Users to view and propose Business Process assets in a domain.
    • The button in the email invitation requesting your input on a Process Register domain now opens the Process Register domain from which the email was triggered.

• Edge
• • Edge is now compatible with the latest versions of k3s and EKS.

    • K3s: Edge supports k3s versions 1.22 and higher.

      • Reinstall your Edge site to upgrade to the latest version of k3s.
        Important In the next quarterly release, 2024.02, Edge sites using k3s versions older than 1.27 will be deprecated. While we do not anticipate this change to break existing Edge sites, support will be limited for troubleshooting Kubernetes versions that are no longer officially supported by the Kubernetes foundation. You will need to reinstall your Edge site to upgrade to the latest version of k3s before the 2024.02 release. For more information, go to the Edge Compatibility table.
    • EKS: Edge supports EKS versions 1.23 and higher for existing Edge sites. New Edge site installations must be installed using EKS 1.27.

      • You must follow these instructions on how to disable Datadog's pod policy in order to successfully upgrade your EKS version from EKS 1.24 or lower to the latest supported version, EKS 1.27.

        Important In the next quarterly release, 2024.02, Edge sites using EKS versions older than 1.27 will be deprecated. While we do not anticipate this change to break existing Edge sites, support will be limited for troubleshooting Kubernetes versions that are no longer officially supported by the Kubernetes foundation. Go to our Support site to follow the instructions on how to upgrade your Edge sites using EKS from EKS 1.24 and lower to EKS 1.27 before the 2024.02 release. For more information, go to the Edge Compatibility table.

New features

Assessments

• API access to Collibra Assessments is now available. You can use the new Assessments REST API to retrieve an assessment, start an assessment, and get lists of both assessments and templates. Additionally, you can call the Assessments API within a workflow to retrieve or start an assessment. This allows you to extract data from Collibra Assessments for further data processing, reporting, trend analysis, making informed data governance decisions, or reintegrating the data into Collibra assets. (idea #DCC-I-438)

Security

• You can use your own encryption key to encrypt the virtual disks of your Cloud environment. With this, you have full control over your data and can revoke access to that data at any time.

Enhancements

Data Catalog

• If the name of the out-of-the-box resource role Owner has been changed, the new name is now used in the "Register Data Source" dialog. (idea #DCC-I-2322)
  
  See what this looks like

  
  

• The “Show inherited responsibilities” checkbox is now available in the views on Catalog index pages, such as "Reports" and "Data Sources". For detailed information and the required setup, go to Show responsibilities in an asset table. (ticket #124292, 124682) (idea #DGCPLAT-I-1435)
  
  See what this looks like

  
  

• When you define S3 crawlers on Edge via API, you can now specify the level from which tables have to be created during the S3 integration. By default, tables are created from the top level, level 1. You can specify a number if you want to create tables starting from another level, such as 2 or 3.
• Catalog views no longer take an unusual amount of time to load when expanding the hierarchies. (ticket #116828, 117222).

Data Governance

• Asset pages now support printing in A5 paper format. (ticket #119398)

Browser Extension

• You can now have the Collibra platform URL preconfigured during mass deployment. This feature enables all users to begin using the Browser Extension immediately after opening it for the first time, ensuring a faster onboarding experience. (idea #DCC-I-836)
• When you now click the “Open on Current Page” button in the configuration dialog box, the dialog box closes and the Browser Extension is launched on the current page with the Search box focused, enabling you to use the extension immediately.

Security

• The Enhanced CSRF console configuration option is enabled by default for new environments.
• The password complexity console configuration options are preconfigured for new environments to improve security and meet regulatory requirements.

Miscellaneous

• The gender of the out-of-the-box users, such as the Admin, System, or Workflow users, is now set to Unknown.
• We have updated the license expiration message to account for customers who have already renewed the license or have a multi-year agreement.

Fixes

Data Catalog

• If you defined a Cron expression to schedule the synchronization of a data source via Edge and the defined Cron expression expires, the Configuration page opens showing a notification instead of not loading at all. (ticket #107362)
• Jobserver upgrades will no longer reset the Jobserver Security Configuration to "None". Instead, the configuration is not impacted by the upgrade. (ticket #120119)
• If a user requests sample data via Edge, we no longer validate whether the user has access to the Database asset. We verify only the View permission of the asset for which the user wants to collect the sample data. (ticket #116059)

Data Governance

• The asset workflow form components no longer suggest assets based on full name (only on display name).
• We have clarified in File API documentation that file IDs can be temporary and therefore trying to get a file that used to exist by it's former ID and will return no result.
• We have fixed an issue when deleting a community with sub-communities that had views on at least one sub-community. (ticket #122147)
• When you create an new scoped assignment on the assettype page, the list of characteristics of that new scoped assignment is now automatically opened. This fix applies to beta releases only.

Data Privacy

• The Log Potential Security Breach workflow no longer fails if you assign a group to the Community Manager responsibility. (ticket #114307, 111944)

Workflows

• Defining SetActivityStreamListener at the process level now correctly shows the "View relevant changes" section on the workflow task.
• A user task using the same form reference as a previous task no longer displays the default values if they were cleared on the previous user task.
• You can again delete workflows if you have a role with the Workflow Administration global permission.

Search

• When you filter by a tag that contains a comma, the comma is now treated as a part of the tag, instead of as a separator. For example, when you filter by the tag "Diversity, Equity, and Inclusion," the entire tag is treated as a single tag, instead of as separate tags ("Diversity," "Equity," and "Inclusion").

Browser Extension

• The "Search in Collibra" option works again.
• Auto-matching now works on a dashboard whose name contains a special character.

Security

• An HTTP header with the Accept-Language set to * no longer causes a warning in the Collibra logs and the default language is used.
• Environments where the Limited CSRF console configuration option is enabled but the Enhanced CSRF option is not no longer prevent new users for signing in for the first time. (ticket #120568)
• A check preventing the creation of case insensitive user name duplicates no longer prevents you from changing the case of an existing user name. (ticket #121158)
• Fixed a path traversal security vulnerability.
• Fixed an SQL injection security vulnerability.

Miscellaneous

• When you apply multiple filters using the "does not contain" operator, the filters now operate under a logical OR operation instead of an AND operation. As a result, an entry will be excluded from the results if it matches at least one of the specified filters, instead of requiring the entry to match all of the filters. (ticket #117258)
• Filtering on inherited responsibilities no longer slows the performance. (ticket #108064)

• If you go to the Homepage that was last edited by a user who is now deleted, the Homepage no longer crashes. (ticket #123828, 127095)

• The Business Qualifier drop-down list in a business qualifier diagram no longer shows assets to which you do not have the permission to view. (ticket #115379)

Data Catalog

Unified Data Classification is a new method to classify your data via Edge. (idea #DCC-I-2411, DCC-I-1965)

• This new method allows for the creation of custom data classes based on classification rules that can be used and detected by an automatic data classification process on Edge.
• Its classification rules can be regular expressions or lists of values.
• The method includes a list of out-of-the-box data classes.
• The data classification capability is independent from the profiling capability. This means you can start the classification of a table or column separately and that the profiling activities take less time.

Important Currently, no data classification migration is available. If you use data classification already and you activate the new data classification method, your current data classes and classifications are no longer available.

Data Marketplace

• We are launching Data Similarity in Data Marketplace. Data Similarity allows you to see similar Table assets inside of a Table asset preview. This will help your data analysts to navigate the data landscape and find alternative data, and will therefore reduce their time to value.
  When you enable the Data Similarity Beta feature, extra AI algorithms, which use privacy enhancing technologies, will run during data profiling. These algorithms will detect similar data across all data sources that you have profiled after you enabled this feature.
  

  Note Data Similarity is currently available for Table assets.

  Tip For now, similar data is available in Data Marketplace only. If you are interested in expanding on this feature to help you discover and reduce data redundancies, and reduce storage and risk assessment costs, add your vote in the Ideation platform.

• We have improved on the Collections feature based on your feedback. You can now:
  • Search through your collections.
  • Sort collections by creation date, last modified data, and name.
  • Sort assets in a collection by time added and name.
  • Filter assets based on their asset type.

Search

• You can now use the search function without interruption even when the search index is rebuilding, by enabling the new Uninterrupted Search configuration setting.
  Note 

  • This setting is disabled by default.
  • When the setting is enabled, the search results may not be current until the rebuild is complete.
  • If you change the setting, you need to rebuild the search index for the change to take effect.

Knowledge Graph API

• Using the OR operator for multiple collection filters no longer returns no results.

Collibra maintenance updates

Collibra 2023.11.1

• We fixed an issue where connections did not display on the Connections tab of an Edge site. (ticket #129320, 129324, 129325)
• It takes less time to open the “Register content” page to register a data source via Edge. (ticket #120020)
• We fixed an issue with the relevancy score of certain search results. (ticket #125074)
• We fixed an issue that caused an error in Data Marketplace when the number of organizations exceeded 65k. (ticket #127227)
• Workflows with scripts that use an array of strings where an array of UUIDs is expected continue to exceptionally work. (ticket #129282, 129299, 129389, 129391, 129392, 129413, 129443, 129464, 129499, 129715, 129722)

  Important Important: If you have such workflows, update your scripts so that they follow the API specification. To convert a String to UUID, use string2Uuid(<your_string>).

Collibra 2023.11.2

• When creating a hierarchy view, the co-role of a relation is now shown in the Configure Hierarchy dialog box. (ticket #129396, 129542, 129551, 129582, 129631, 129648, 129671, 129862, 130084)

Data Lineage 2023.11.1

• When integrating MicroStrategy:
  • Intelligent Cubes are now ingested even if some properties are missing in the API call. (ticket # 127046)
  • MicroStrategy Data Model assets have a relation to MicroStrategy Folder assets.

Data Lineage 2023.11.3

• When integrating Power BI, the Owner in Source attribute on Power BI Dataflow asset pages now correctly reflects the real owner of the dataflow in Power BI.
• When integrating Looker and filtering on a folder, metadata processing no longer fails if there are Looks that don’t pass the filter, but are related to a Tile that is ingested. In this case, processing continues and the following warning message is included in the source code in the Sources tab page: “Look <Look ID> was not found in folder <folder ID>.” (ticket # 125911)

Data Lineage 2023.11.4

• When integrating Power BI:
  • If a data set in Power BI has a description, the description is now ingested and shown on the asset page of the corresponding Power BI Data Model asset.
  • If reports are ingested from multiple tenants, the report URLs on the asset pages in Collibra now include the tenant ID. Previously, without this differentiation in the URL, the links accessed the page of the same, default tenant, even if a given report was not located in that tenant. (ticket #129557)

Data Lineage 2023.12.2

• When creating technical lineage via Edge, you can now use a man-in-the-middle (MITM) proxy between and the Collibra Data Lineage service instances. For details on which data sources support the use of proxies, see Create a technical lineage via Edge and select your data source. (ticket #120946, 121788, 124013, 124390, 126170, 128684, 130493, 132075)

Data Lineage 2023.12.3

• When integrating Tableau, the relation “Report uses / used in Data Attribute” is now created between Tableau Dashboard assets and Tableau Data Attribute assets.
• When when using the batch definition method (beta) to create a custom technical lineage, table-level lineage is now correctly shown, even if you don't include the source code files. (ticket #129835)
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support SELECT… EXCLUDE statements. (ticket #131342)

Data Lineage 2023.12.3.1

• When attempting to create a custom technical lineage via the new batch definition method (beta), if your JSON files don't conform to the required JSON schema, an error message is now included in the log file.