Only show release notes of Collibra Platform for Government-certified features

Release 2024.05

Release information

  • Publication dates:
    • April 22, 2024: Release notes
    • May 2, 2024: Documentation Center
  • Release dates of Collibra Platform:
    • May 5, 2024: 2024.05.0 (Upgrade non-production environments )
    • June 2, 2024: 2024.05.1 (Upgrade production environments )
    • June 9, 2024: 2024.05.2
    • On demand: 2024.05.3
    • On demand: 2024.05.4
  • Edgeand Data Lineageupdates
    • May 5, 2024: Edge 2024.05-0 and Data Lineage 2024.05.1
    • May 12, 2024: Data Lineage 2024.05.2
    • May 19, 2024: Edge 2024.05-14 and Data Lineage 2024.05.3
    • May 26, 2024: Edge 2024.05-21 and Data Lineage 2024.05.4
    • June 1, 2024: Edge 2024.05-27 and Data Lineage 2024.06.1
    • June 3, 2024: Data Lineage 2024.06.1.1
    • June 9, 2024: Edge 2024.05-35
    • June 16, 2024: Edge 2024.05-42
    • June 23, 2024: Edge 2024.05-49
    • June 25, 2024: Edge 2024.05-51 and Data Lineage 2024.06.4.1
    • June 30, 2024: Edge 2024.05-56 and Data Lineage 2024.07.1
    • July 7, 2024: Edge 2024.05-63 and Data Lineage 2024.07.2
    • July 14, 2024: Edge 2024.05-70
    • August 4, 2024: Edge 2024.05-91
    • August 11, 2024: Edge 2024.05.98
    • August 24, 2024: Edge 2024.05.111

Highlights

  • New look and feel (latest UI) is now generally available
  • Our new, refreshed Collibra experience is now generally available. It provides a completely redesigned look and feel that is applied across the platform. With improved color selection, layouts, visual separation, and many other design elements, the platform is now more engaging and easier to use than ever. Additionally, many features are built upon this new look and feel, such as our Tailored asset pages, AI Governance, and many more. Check out the changes and features that come with the new Collibra user interface.
    • For commercial customers, the new look and feel is enabled by default in non-production environments.
      In production environments, the new look and feel is disabled by default. To enable it, go to Enable or disable the latest user interface.
    • For Collibra Cloud for Government and Collibra Platform Self-Hosted, the new look and feel is disabled by default in both non-production and production environments. To enable it in any environment, go to Enable or disable the latest user interface.
    Important The documentation identifies the new UI as latest and the old UI as classic. You can find an option to switch the documentation between the classic and the latest UI.
  • Collibra AI Governance
  • Collibra AI Governance is now generally available. AI Governance allows you to catalog, assess, and monitor any AI use case to deliver trusted and valuable AI, and is designed to help you with the following:

    • Improve your organizational access to, and adoption of, AI systems.
    • Promote visibility, productivity, compliance, and accountability around AI use cases.
    • Mitigate the data privacy, intellectual property, and ethical risks associated with AI.
    • Prevent the duplication of efforts in the development and adoption of AI systems.
    • Drive meaningful business evaluations of AI use cases prior to investing in them.
    • Provide cross-functional collaboration, to address the internal use of AI systems for common business functions.

    To learn more about enabling AI governance in your environment, reach out to your Collibra Account Team.

    Note This is available only in the latest UI.

  • Data Notebook
  • We've introduced Data Notebook, a querying tool that's integrated directly into the Collibra Platform to enable you to find and query data in real time via an SQL editor. With Data Notebook, you can run queries against your data sources, reducing the time required to access and explore ingested data. Data Notebook also promotes collaborative efforts by allowing you to create assets from notebooks, giving your teams a centralized knowledge repository within Collibra.
    Note This is available only in the latest UI.
  • Usage Analytics
  • The Usage Analytics dashboard is now revamped to include more metrics and viewership information, such as tracking dashboard visits, and to make the information easier to consume with improved visualizations and filtering throughout the dashboard.

    This new dashboard introduces the following key features to provide insights into resource usage and user engagement within your Collibra environment:

    • Resource visit metrics: Track visit frequency not only for assets but also for communities, domains, diagrams, and dashboards, including the most visited resources.
    • Engagement metrics: Track user engagement with usage rates and user retention statistics, offering insights into user behavior.
    • Advanced filtering: Filter metrics by organization, asset type, user group, role, license type, and usage period, with options to exclude administrators or deactivated users for focused analysis.
    • Interactive charts: Interact with charts by excluding specific categories from view or switching to a table display mode to tailor the visualization to your needs.
    • Downloadable data: Download the metrics in CSV files for further analysis.
    Note This is available only in the latest UI.
    (idea #DCC-I-1383, DCC-I-1374, DCC-I-1316, DCC-I-1419, DCC-I-724, DCC-I-335, DCC-I-381, DCC-I-1643, DCC-I-1839, DCC-I-2534)
  • Search
  • To make it easier to find curated data, we've now integrated Data Marketplace into the Collibra Platform Search. With this unified search experience, you can seamlessly switch between global search for searching in all resources and the Data Marketplace search for searching in a curated subset of assets. To facilitate this, the Data Marketplace option is now added to the search page.
    Note 
    • This is available only in the latest UI.
    • The Data Marketplace option is shown on the search page only if you have the Data Marketplace global permission or the Sysadmin global role.
    • If you have previously configured Data Marketplace and want to use unified search, your existing settings for Data Marketplace scope, filter facets, relation indexes, and asset previews are automatically considered. Additional configuration, however, is needed to define quick filters and search filters.
  • Data Catalog
  • We're pleased to announce that Collibra AI for automated description recommendations is now generally available. This feature helps you accelerate the creation of descriptions for your assets. (idea # DCC-I-1392)
    • Collibra AI can help you to create a description for a Column, Table, Database View, or Data Set asset.
    • In Table assets, Collibra AI can also help you to create descriptions for all columns in the table, in a single action.
  • Identifying columns that contain sensitive information, such as Personally Identifiable Information (PII) or Protected Health Information (PHI) is very important. To allow you to create such an overview, we now automatically create a link between the asset and the Data Category, when you approve a data classification for an asset and the data class is linked to a data category.
    The relation “is categorized by Data Category / categorizes Data Asset” is now assigned to the Column asset type. In the latest UI, this relation can also be included in the asset page layout.
  • The Google Vertex AI integration via Edge is now generally available. When you integrate Vertex AI, you integrate the metadata of ML models from Vertex AI to Collibra Data Intelligence Platform. The resulting assets represent the Vertex AI model.
  • Data Lineage
  • DAX analysis via Collibra AI is now generally available. When integrating Power BI, you can now enable DAX analyzing, to allow for column-level lineage of calculated columns and measures. (idea #DCC-I-265)
  • You can now integrate the following BI tools via Edge:
    • Looker
    • SQL Server Reporting Services (SSRS) / Power BI Reports Server (PBRS)
    These integrations bring improved performance and new scanners that can process more complex data objects and lineage.
  • You must now use the Edge CLI tool to manage files in the Shared Storage connection. Previously, you would upload files to the edge server. Now, you have to upload files to the Edge server and use the Edge CLI tool to copy those files to the cache that Collibra Data Lineage reads. If you have existing Shared Storage connection folders with sources files, you must log in to your Edge server and upload source files to the cache by using the Edge CLI tool.

  • Data Privacy
  • To integrate Data Privacy better with Collibra Platform and its products, the Data Privacy asset types are now included in the out-of-the-box operating model. As a result, the following are available out of the box:
    • A total of nine Data Privacy asset types, including their characteristics and statuses.
    • Data Privacy domain types and resource roles.
    Important 
    • These asset types are added in every Collibra environment but activated for only those who have purchased Data Privacy.
    • Due to this change, the names of some of your existing custom asset types or resource roles may now have the suffix "local." For more information, go to the Support Portal.
    • Any characteristics previously removed from these asset types are now restored, ensuring their availability for future enhancements. All other changes remain as they were. For more information, contact the Collibra Account Team.
  • Edge
  • Edge sites can now be installed on the following managed, shared Kubernetes clusters:
    • Azure Kubernetes Service (AKS)
    • AWS Fargate using EKS
    • Google Kubernetes Engine (GKE)
    • OpenShift

    If you subscribe to any of these Kubernetes services, you can now operate your Edge site with system requirements that are aligned to your preferred platform.

    Note 
    • Edge sites installed on a managed Kubernetes cluster can only be installed via the Edge CLI or helm chart. This includes newly installed Edge sites on an EKS cluster.
  • There are 2 new installation methods for Edge sites installed on a managed Kubernetes cluster:
    Note 
    • You can no longer use the previous method to install Edge sites on a managed Kubernetes cluster, including EKS.
    • You should only use the Helm chart installation method if you are experienced with helm and Kubernetes. Support will be limited for helm chart installations.
    • K3s Edge site installations are not impacted by this change.
  • Documentation update
  • To make sure all known issues and troubleshooting information is gathered in one location, we're moving this type of information to the Support Portal. Any known issues and troubleshooting information in the Documentation Center will gradually be replaced with links to the relevant information in the Support Portal.
  • Metamodel changes
  • Important New out-of-the-box asset model objects might affect your custom-created ones. For example, if you have created an asset type with a specific name, and we release a new out-of-the-box asset type with the same name, your custom object’s name could change automatically, or the out-of-the-box object’s name might be different from what’s in the release notes.

  • With the general availability of AI Governance, we've made the following metamodel changes:
    • Added a new asset type Vendor. It is a child of the Party asset type that is included with Data Privacy.
    • For the AI Use Case asset type:
      • Added a new attribute type Business Risks.
      • The attribute type Use Case Stage will be automatically removed from the out-of-the-box global assignment if there are no instances of it in use.
      • The relation type "AI Use Case transforms / is transformed Asset" has been renamed to "AI Use Case infers from / used to infer Asset". The UUID (00000000-0000-0000-0000-000000007099) is unchanged.
    • For the AI Model asset type:
      • Added the following relation types:
        • AI Model trained by / trains Asset (UUID: 00000000-0000-0000-0000-000000007102)

        • AI Model infers from / used to infer Asset (UUID: 00000000-0000-0000-0000-000000007103)

        • AI Model has output / is output Asset (UUID: 00000000-0000-0000-0000-000000007104)

        • AI Model is provided by / provides Vendor (UUID: 00000000-0000-0000-0000-000000007105)

        • AI Model uses / is used by AI Model (UUID: 00000000-0000-0000-0000-000000007106)

      • Added the new attribute types:
        • Version
        • Repository
      • The following attribute types will be automatically removed from the out-of-the-box global assignment if there are no instances of them in use:
        • Descriptive Example
        • Normalized Discounted Cumulative Gain
      • The attribute type Retrain Cycle has been changed from a selection attribute type to a plaintext attribute type.
      • The attribute type Model Type now has the following possible values: Generative AI, Classification, Regression, Computer Vision, Reinforcement Learning, and Image Classification.
    • Due to the Databricks AI model integration, we have added a new asset type, Databricks AI Model. This asset type is a subtype of AI Model that represents AI models in Databricks Unity Catalog.
  • The following new out-of-the-box global role and global permissions are now added in Assessments
    • Assessments Template Manager global role: Allows users to view, create, edit, and manage assessment templates.
    • Conduct Assessments global permission: Allows users to conduct, edit, copy, delete, and make assessments obsolete, including only public assessments and those that they own or are assigned.
    • Manage Templates global permission: Allows users to view, create, edit, and manage assessment templates.
  • To help you fine tune workflow management in Collibra, you now have 2 new global permissions that users require to:
    • Start workflows
    • Participate in workflows

    These new workflow permissions are designed to give you more control over who in your environments can use workflows. This allows administrators to restrict the ability of users to trigger workflows, which incurs a cost for the organization.

    Learn more about How to manage the new workflow permissions on the Developer Portal.

    Important These permissions are not included by default in any global role. You must assign the permissions to the global roles that you want to be able to perform these actions. Users without the Start workflows permission might not be able to see the Plus icon global create button anymore.

    Note Users that have a global role with the Workflow Administration or System administration global permissions do not require these permissions explicitly.

New features

Data Catalog

  • Google Dataplex Catalog integration now supports the GCS Buckets asset type.

Protect

  • APIs are now available for integrating Protect with your custom data sources. You can use these APIs to build custom integrations for reading the policies from Protect and enforcing them in your own data sources.

Workflows

  • To help you fine tune workflow management in Collibra, you now have two new global permissions that users require to:
    • Start workflows
    • Participate in workflows

    Learn more about How to manage the new workflow permissions on the Developer Portal.

    Important These permissions are not included by default in any global role. You must assign the permissions to the global roles that you want to be able to perform these actions. Users without the Start workflows permissions might not be able to see the Plus icon global create button anymore.

    Note Users that have a global role with the Workflow Administration or System administration global permissions do not require these permissions explicitly.

Edge

  • Edge now supports installing Edge sites on the following managed, shared Kubernetes clusters:
    • Azure Kubernetes Service (AKS)
    • AWS Fargate using EKS
    • Google Kubernetes Engine (GKE)
    • OpenShift

Collibra Console

  • There is a new Upload configuration option that allows you to enable or disable the download of attachments.

Enhancements

New look and feel (latest UI)

  • When mandatory characteristics are empty, they will still appear on the asset page. This ensures that you can clearly see critical information is missing and makes it easy for authors to edit those characteristics. Previously, all empty characteristics would not appear unless the “Show empty” option was on. The option has been renamed to "Show Empty Optional Values". (idea #DCC-I-3077)
  • Editable cells in tables now show a pencil icon. The content in those cells can be edited by double-clicking anywhere in the cell. (idea #/DCC-I-2867)
  • You can now copy the text in a table. (idea #DCC-I-3316)
  • The dialog box for adding a relation to an asset now shows the name of the relation type.
  • The “Delete” confirmation when deleting a domain or community can now be typed in your own language.
  • We have optimized assignment fetching when rendering tailored asset pages, so there is no delay in displaying the page.
  • If you have an author role and are making changes on an asset page via in-line editing, save and cancel options are added for all fields except single, boolean, and date picker. (idea #DCC-I-2889, DCC-I-3169)
  • Column assets are loaded like the other assets in the preview without delay.
  • Sysadmin users can now migrate the Tailored Asset Page layout between environments by selecting the assignment linked to the layout that needs to be migrated. (idea #DCC-I-3014)
  • The dashboards shown on the Homepage, on the Dashboards tab in the browse pane, and on the dashboard tab bar now follow the same order: pinned dashboards in alphabetical order, followed by unpinned dashboards in alphabetical order. (idea #DCC-I-1892)
  • To mitigate potential performance issues, you can now sort only the following columns in the Users table: First Name, Last Name, User Name, and Email.
  • On the Custom Theme page, the analog clock is now replaced with a digital clock.
  • With the general availability of the latest UI, we have moved the console setting "New frontend experience enabled" from the Beta section to the "Frontend features" section.

Data Catalog

  • You now need the Classification > Data Classes > Read global permission to open the Data Classification page.
  • To open the details of a data class in the Data Classification page, you now click the Preview button, instead of clicking the data class name. (idea #DCC-I-2592)
  • We have removed the requirement to enable some metadata integrations in the settings. For example, you no longer need to enable the "Databricks Unity Catalog synchronization via Edge" setting to be able to integrate Databricks Unity Catalog.
  • The S3 integration via Edge now integrates the following additional information:
    • For Columns assets: "Description from source system" and "Column position".
    • For Table assets: "Description from source system" and "Table type".

    Note You can't integrate the descriptions from source directly. First, integrate S3 once, then find description for Table and comment for Column in the Glue DB and add description/comment there, and finally, resynchronize S3.

  • To increase the performance of the Snowflake metadata synchronization we can now read the Snowflake Source Tags from SNOWFLAKE.ACCOUNT_USAGE schema. You can configure this in the Edge Capability with property tags-strategy and value SINGLE_CALL.

    Note This requires the SELECT permission on SNOWFLAKE.ACCOUNT_USAGE.TAG_REFERENCES table.

  • For Databricks Unity Catalog integrations, you can now define the domain and extensible properties mappings in the integration configuration screen, instead of in the capability parameters.
    Note This is available only in the latest UI.

Data Lineage and BI integrations

  • We have removed the requirement to enable lineage, and BI tool and ETL integrations in the settings.
  • When ingesting Snowflake (SQL mode) via Edge or the lineage harvester, Collibra Data Lineage now harvests the metadata of stored procedures.
    Note We are currently doing the backend work to be able to generate lineage for stored procedures. We will keep you informed of our progress.
  • When ingesting Oracle data sources via the lineage harvester with a JDBC connection, you can now use the "databaseLinkMapping" property in your lineage harvester configuration file to configure, per data source, the database and schema to which a DBLink points. For complete information, example scenarios, and configuration advice, see the "databaseLinkMapping" property description in Prepare the lineage harvester configuration file.
    Note Full support for this property is not yet available, as we are finalizing the backend work. We will keep you informed of developments.
  • When no proxy is configured, the certificate (ca.pem) provided during Edge installation is now added to the trust store for communication with the Collibra Data Lineage service instances.

Data Quality & Observability Classic Integration

To find out more about all Data Quality & Observability Classic features, enhancements, and fixes included in the 2024.05 release, see the official Data Quality & Observability Classic release notes.
  • You can now view data quality scores and run jobs from the Data Quality Jobs modal on Table asset pages. You can access this modal on Table asset pages by clicking the View Monitoring link on the At a glance pane.

AI Governance

  • The Register New AI Use Case assessment is now replaced by the following four new assessments, to better enable stakeholders to assess, in parallel, the aspects of the AI use case they are most interested in:
    • Business Context
    • Data and AI Models
    • Legal and Ethics
    • Risks and Safeguards
  • We have introduced new facilities to help you find and link AI models to the AI use cases that use them.
    • You can use the new Collibra-supported Vertex AI or Databricks Unity Catalog integration to create AI Model assets.
    • If you use a different AI model provider or you want to integrate a proprietary system, you can perform a custom integration. On the Collibra Developer Portal, you can find a tutorial explaining how to use Python to create and synchronize AI Models in Collibra.
  • Improvements to the AI Use Case asset page:
    • We have added a Lifecycle Tracker to help you:
      • Monitor and drive the evolution of your use case.
      • View the assessment history of the use case, and add and start new assessments.
      • Advance the use case to the next stage in its lifecycle.
    • We have added a diagram view focused on the AI Use Case asset. The diagram depicts all of the relations that exist between the AI use case and other assets, for example linked AI Model assets.
  • You can now register a new AI use case via the Legal Reviewer landing page.
  • Any users or user groups that you assign the Business Steward resource role for a specific domain, to meet the requirements for submitting an assessment, must now also have the new “Participate in workflow” global permission.

Data Governance

  • We have simplified the permission checks for SysAdmins who are updating the operating model. SysAdmins now only need the System Administration global permission. They no longer need any additional product right permissions corresponding to the asset type to create a new custom asset type or update an asset type.
  • We have improved the performance of the asset pages.
  • In the latest UI, we have improved how images in rich text fields are shown on asset pages. This ensures that images utilize the available space.
  • To allow administrators to have a clear view on the asset type co nfiguration possibilities, we have introduced 4 new columns on the Settings > Operating Model > Asset types page in the latest UI.
    • Status Editing: If status editing it is not allowed, it means that Collibra plans to facilitate the life cycle of this asset type. Therefore, you will not be able to add custom statuses to this asset type. You will no longer see the 'Edit Model' button on the statuses tab.
    • Final Type: If an asset type is flagged as final, you will not be able to create custom subtypes or scoped assignments for this asset type. On the asset types page, you will no longer:
      • See final asset types as an option in the 'Parent asset type' field because you are not allowed to create or move a custom subtype for a final asset type.
      • See the 'Add Assignment' buttons because you are not allowed to create a scoped assignment for this asset type.
    • Activated: Indicates if you are allowed to use this asset type. If the value is false, you are not able to:
      • Create an asset of this asset type.
      • Update asset details and attributes on an asset of this asset type.
      • Use collaboration features like commenting, ratings, and tags on an asset of this asset type.
      • Create a new scoped assignment for this asset type.
      • Create a new custom subtype of this asset type.
    • Public Id: Public Ids are stable identifiers that can be used in the code of a workflow or integration instead of UUIDs.

      Note Our API's are not adjusted yet. You can see the publicId but you can not use it yet.

  • We have made the following changes to improve the protection of system (Collibra-managed) operating model component configurations. This is to guarantee that features which use these collibra-managed resources can't break.
    • For number attribute types that are Collibra-managed, it is no longer possible to change the value of the 'isInteger' parameter via API calls to update an attribute type.
    • For script attribute types that are Collibra-managed, it is no longer possible to change the value of the 'language' parameter via API calls to update an attribute type.
  • An asset view with multiple responsibility filters no longer takes an unusual amount of time to load.
  • Views with multipath hierarchies no longer take an unusual amount of time to load. (ticket #130945)
  • We introduced some additional safeguards on the maximum volume of returned objects for API calls that return a paged resultset. For new instances, the maximum value you can pass in the limit field of the request, will be set to 1000. For existing instances, customers can request to have this protection against their system going down by abnormal behaviour by creating a support ticket and asking to set 'Enable maximum paging limit' to true. We recommend to do this first on non-production instances, test integrations and workflows which create, update or query large amounts of data to make sure these integrations and workflows use paging correctly (meaning request batches of max 1000, if more results need to be fetched, a loop going over batches of 1000 should be used).

Data Marketplace

  • In the latest UI, search suggestions are now always shown in Data Marketplace. As a result, the Search Suggestions setting has been removed.
  • In the latest UI, some Search settings that impact Data Marketplace have been renamed or have moved to other sections.
    • Filters has been renamed to "Filter Facets".
      We'll refer to filter facets in the Data Marketplace documentation from now on.
    • Actions has been renamed to "Actions and Preview".
    • The options in "Shopping Basket and Data Set Management" are now available in "Actions and Preview".
    • The options in "Extra Options" are now available in other sections in the Search settings.
    • "Quick Filters" has been added, and allows admins to define quick filters for Data Marketplace in Search.

Assessments

  • The following new out-of-the-box global role and global permissions are now added: 
    • Assessments Template Manager global role: Allows users to view, create, edit, and manage assessment templates.
    • Conduct Assessments global permission: Allows users to conduct, edit, copy, delete, and make assessments obsolete, including only public assessments and those that they own or are assigned.
    • Manage Templates global permission: Allows users to view, create, edit, and manage assessment templates.
  • The Business Steward assigned to the domain of a submitted assessment now needs the following global permission to participate in the Assessments workflow: Workflow > Participate in Workflow
  • The following elements are now included in an assessment template:
    • HTML: You can insert hyperlinks and format rich text.
    • Expression: You can show a specific text based on responses or parameters assigned to questions. For example, you can use an expression to assign risk levels based on the total scores obtained from specific questions in your template. This allows for customized text to be shown based on responses.
  • You no longer need the Policy Manager global permission to use Assessments.

Data Privacy

  • Previously, the CCPA and GDPR tabs were shown on the Business Process asset page only if the asset had the following relation with Regulation assets: asset complies with Governance Asset. This dependency is now removed, meaning that the CCPA and GDPR tabs are shown on the Business Process asset page even if there are no Regulation assets.
    Note The GDPR and CCPA tabs are shown if you have the Product Rights > Privacy or Resources > Manage All Resources global permission and the Privacy landing page setting is enabled.

Workflows

  • You can no longer remove the Actions column from the Workflow Instances page, in the latest UI.
  • You can now use the Wrap text option to see all the text in the Workflow Definitions table, in the latest UI.
  • You can no longer filter by the upload date on the Workflow Definitions page, in the latest UI.
  • We have improved and consolidated the display options for the Tasks, Workflow Definitions and Instances pages, in the latest UI.
  • You now see an improved error message when adding too many start events to a workflow definition.

Workflow Designer

Note Workflow Designer enhancements become available with the upgrade of production environments.

  • You can now import packages from the lucene-analytics-common and lucene-core libraries into your Groovy scripts.

Edge

  •  Edge managed Kubernetes clusters are now compatible with k8s 1.28, in addition to 1.27. No action is required.
  • Edge sites installed on bundled k3s from the 2024.05 release are on k8s 1.28. No action is required.
  • There are 2 new installation methods for Edge sites installed on a managed Kubernetes cluster:
    • Edge CLI
    • Helm chart
  • We have added a new Out of Sync Edge site status to more clearly identify and fix Edge sites experiencing issues while upgrading. If an Edge site encounters an issue while upgrading, and is unable to continue after 60 minutes, the Edge site status will change from Upgrading to Out of Sync.

    Once in the Out of Sync status:

    1. Automatic upgrade mode: your Edge site will automatically try to upgrade again in an hour.
    2. Manual upgrade mode: You can try to upgrade your Edge site again.

    If your Edge site continues to go into the Out of Sync status, contact Collibra Support.

Search

  • When you begin typing in the global Search box, the following are now shown below the box:
    • The 3 assets you last visited or the top 3 results
    • Up to 3 of your recent searches
  • When searching for URLs, exact matches are now prioritized over partial matches, improving search relevance.
  • Tilde (~) and exclamation mark (!), which were used to represent fuzzy search and negative search, respectively, are no longer treated as reserved characters with special behavior. They are now treated as part of the search query without altering its logic.
  • If the search results span over 1000 pages, only the first 1000 pages are now shown. To view additional pages, you can use the sorting options.
  • The Uninterrupted Search setting is now enabled by default.
    Note 
    • If you are a commercial or Government customer, you can no longer enable or disable this setting yourself.
    • If you use CPSH, you can now enable or disable this setting in the Search Index Configuration section in Collibra Console only if you have the SUPER role.
  • Previously, the Maximum Batch Size for Relations setting in Collibra Console controlled the maximum batch size of both the relation reindex and the relation path preview of a relation index. Now, a new setting, Maximum Batch Size for Relation Path Preview, is added to the Search Index Configuration section for controlling the maximum batch size of the relation path preview.
  • To reduce the overall time of indexing resources and relations, Collibra has now introduced the use of multi-threaded indexing. The number of threads is automatically configured based on the available resources.

Insights Data Access

  • Insights Data Access now includes Usage Analytics events, giving Data Governance Managers ultimate flexibility to find insights within their Collibra environment and making reporting easier for them. (idea #DCC-I-1187)
  • We have introduced an additional endpoint, /reporting/insights/directDownload, to fix timeouts encountered when downloading large files via our Reporting API. This endpoint downloads the zip file containing reporting data directly from cloud storage, bypassing the Collibra network.
    Note 
    • To use the endpoint, ensure that your application allows redirects.
    • The endpoint uses a temporary pre-signed URL that's valid for 60 seconds by default.
    • The endpoint is compatible with both AWS and GCP.
    • The existing endpoint, /reporting/insights/download, will be deprecated soon.

Collibra Console

  • To prevent potential high memory usage, you can no longer set the CRON interval of the Email configuration default schedule to less than one minute. Configurations below this threshold are updated to one minute automatically. (ticket #131263)
  • We have removed the product information from the License tab in Collibra Console. You can now find this information in Settings → Users and Subscriptions → Package.
  • We have improved the backup restore error messages.

Dashboards

  • When configuring the Collibra Insights, Embedded Webpage, and Text widgets, you can now choose to remove extra padding and use the full container size to show content using the new Remove Padding and Border checkbox.
  • The results in a Bar Chart widget are now paginated to make it easier to review the results and limit the risk of memory outage. (ticket #128533)

Fixes

New look and feel (latest UI)

  • Previously, if you had only the Edge integration engineer global permission and you were on the Homepage, the Settings option in the top navigation bar wasn't shown. This issue is now fixed. (ticket #131417)
  • Due to a change in labeling on the asset pages, some bookmarked asset URLs were broken. This issue has been resolved. Bookmarked asset pages remain functional.
  • Asset pages:
    • We have increased page usage on community and domain pages so tables are spaced out better.
    • We have fixed an issue with the delete button being available to users with a role that did not allow for deletion of assets. (ticket #127068)
  • Tailored Asset Pages: Some users had issues with editing on the Tailored Asset Pages despite having the correct permissions. This issue has been resolved.
  • Text editors:
    • You can now add relative URLs in text editors.
    • You can add and format new tables in text widgets and text editors when you enable the latest UI. (ticket #136818, 140220) (idea #DCC-I-3075)
  • Views:
    • Previously, if you added the Domain field to a tile without also adding the Community field, the domain wasn't shown on the tile. This issue is now fixed.
    • When you open a drop-down list at the bottom of a table, the list is no longer cut. (ticket #139240)
    • An error no longer occurs when you pin or unpin an existing view.
    • When you use the Wrap Text option in a table, all the text within the cells is now properly aligned.
  • Dashboards: Images and multi-line headers in a Text widget created in the classic UI are no longer distorted when switched to the latest UI. (ticket #139339, 139339, 141828)
  • Customizations:
    • The On Dark logo uploaded for the top navigation bar no longer affects the color of other icons. (ticket #137395, 138963, 139382)
    • The customization of the colors in the Date Picker component now functions properly.
    • The text color of primary and secondary buttons upon hover now uses the customized text color instead of always showing white. (ticket #141752)
  • Data Marketplace: Searching with square brackets no longer causes unexpected results. (ticket #139043)
  • Search: The global search box now applies the community or domain filter when applicable.
  • The dates shown in complex relations and views are now accurate and are no longer one day ahead. (ticket #140119, 139302, 141773, 142079, 142359)

Data Catalog

  • During the Metadata Synchronization via Edge, Schema assets are now created with the default status as defined in the assignments, instead of the status Candidate. (ticket #131609) (idea # DCC-I-370)
  • In Catalog asset pages, you can now save changes to fields in views for all users. This means that, in Table asset pages, users with the correct permissions can now configure the display of the Columns section.(ticket #139771, 141128) (idea #DCC-I-5680)

    Note This update is available in the latest UI.

  • Subtypes of out-of-the-box assets, such as Table, Column, and Data Set, now display the out-of-the-box sections for these assets on the Tailored Asset Page.
  • The profiling result "Number of Distinct Values" can no longer exceed the Total Row count. (ticket #130794, 130780, 130263, 130172)
  • Integer values are now correctly displayed when viewing samples. (ticket #134036, 136414)
  • On a Schema asset page, the "Description from Source" field is now displayed by default. On a Database asset page, the Source tags field is displayed by default.

    Note This update is available in the latest UI.

  • We no longer hide data classes with connections to only assets that are not Column, Data Attribute, or Data Concepts assets from the Data Classification page. (ticket #129844)
  • Integer values are now correctly processed as such values by the Unified Data Classification method. (ticket #134036)
  • You're now much faster notified that a Unified Data Classification job has started. Before this change, it could take a few seconds for the notification to appear.
  • The Unified Data Classification process no longer fails when a table has a large number of columns or has columns with a very long name. (ticket #133729)
  • Unified Data Classification now starts classification events that can be used as workflow triggers.
  • Successful Unified Data Classification jobs that take a long to process will no longer result in an error. (ticket #140324)
  • The Unified Data Classification process now correctly enables the push-down sampling feature when available. (ticket #14324, 143723)
  • When you reimport an unmodified data class with more than 1,000 items in the list of values, the Import dialog now correctly shows "Exists (no changes)" instead of "Exists (changed)". (ticket #140337)
  • On the Data Classification page, users can no longer see the Data Categories, Data Attributes, or Data Concepts for which they don't have the View permission.
  • We have improved Collibra AI for generating asset description recommendations, allowing larger user queries to be returned without a timeout.
  • Limiting API usage issues within Data Marketplace and Catalog implementations have been resolved. (ticket #138906, #140175)

Data Lineage and BI integrations

  • We have updated an SQL query to harvest data object names when ingesting Teradata data sources, to mitigate failure during ingestion.

AI Governance

  • The Register AI Use Case workflow task to complete an assessment now has a correctly formed link that contains the host name. Without the host name, the email does not reach the recipient.

Data Governance

  • In the latest UI, when you add a relation on the asset page, we fixed an issue when loading suggestions when the ‘Filter by Organization’ option is selected. The drop-down list now correctly show all results.
  • We have fixed a caching issue related to metrics. (ticket #130890)
  • The Data Quality tab now displays faster when large amounts of data are involved in the calculation. (ticket #130890)
  • We made some performance improvements related to assignments, which impacted asset pages load times. (ticket #138695, 138397, 139380)
  • You can no longer change the scope of an assignment.
  • We updated the logic in the operating model migration import functionality for system protected complex relation types. This now ensures that the Denodo solution can be installed in all instances again.
  • Activities created during an import operation are again identified correctly in the history records.

    Note For Collibra version 2024.03 and 2024.04, such activities are incorrectly identified as Update instead of Import.

  • Exporting data in Excel that exceeds the maximum amount of a single cell value now returns a specific error message.
  • We have added options to limit paged requests. (ticket #87783)
  • We have fixed an issue with new section headers not appearing in the editor layout when using the Safari browser.

Data Marketplace

  • The "Add to Data Basket" button is now available to all users when it's configured for the asset type. (ticket #135585)
  • Opening the shopping basket if the basket can't retrieve all data no longer crashes the whole platform. (ticket #136750)
    Note This is available only in the latest UI.
  • We've fixed an issue that could cause duplicate selections to appear in the filters of Data Marketplace. (ticket #132573)
  • When using preconfigured filters, the Data Marketplace search results now correctly exclude asset types, statuses, and organizations that are not within the defined Data Marketplace scope.
  • Clicking a link in Data Marketplace no longer redirects you to the homepage. (ticket #138694, 142515)

Diagrams

  • The title on the diagram page of an asset now shows the asset's display name instead of its full name.
  • The Go to Pictures button in a diagram now functions properly.

Assessments

  • In a DPIA or PIA Threshold assessment, the button to conduct the assessment is now shown when the threshold is reached. (ticket #141066)
  • Drop-down lists in assessments no longer show inconsistent values. (ticket #133610)
    Note This issue occurred only in the classic UI.

Protect

  • Protect no longer handles data classes without any accepted matches. (ticket #141265)

Workflows

  • The error message in case of a missing recipient for an email task now reflects the cause of the error. (ticket #136071)
  • The Rich text form validation now works as expected in the latest UI. (ticket #139409, 143099)
  • The REST Core API v2 GET method of the /workflowInstances endpoint no longer returns unexpected results. (ticket #132588, 134642)
  • When configuring the workflow assignment rules in the latest UI, you can now choose the generic Asset asset type. (ticket #142814)
  • Deleting a workflow definition in the latest UI, automatically removes it from the definitions page when the process completes.
  • Completing or canceling a workflow task in the latest UI no longer causes a wfTaskNotFound error to be logged erroneously. (ticket #141150)
  • You can start again a workflow in the latest UI from a link that points to the resource the workflow should run for and the #wfid parameter. (idea #DCC-I-174)
  • Clicking outside a workflow dialog box in the latest UI no longer closes the dialog box. (idea #DCC-I-3127)
  • Emails sent by workflows created with the Workflow Designer now respect the language preferences of the recipients. (ticket #134501)

Workflow Designer

Note Workflow Designer fixes become available with the upgrade of production environments.

  • You can now set a default value for the Rich text form component. (ticket #137218)
  • You can again access the history of your Workflow Designer models and restore previous versions. (ticket #138861, 138882, 139139, 139720, 139763, 139785, 139810, 139825, 139829, 140179, 140204, 140223, 140312, 140409, 140448, 140486, 140568, 140771, 141334, 141721)
  • The Workflow Designer Text display form editor no longer saves unsupported HTML tags. (ticket #127576, 136980)

Edge

  • We fixed an issue where Edge sites could be installed successfully even if an invalid command was used. This resulted in unhealthy Edge sites. With this fix, if an invalid command is used during the Edge site installation process, the installation will fail.
  • We fixed an issue which caused the Edge diagnostic connectivity checks to fail for customers of Collibra Platform for Government when a custom helm registry was used.
  • Once you delete an Edge site, you can no longer access that deleted Edge site URL. Previously, if you deleted an Edge site and then clicked the back button, you would go back to the deleted Edge site page, but the screen would be empty.
  • You can no longer save an Edge site connection if the required fields have not been completed.
  •  We improved the clarity of the Edge site back-up validation message. Previously, this message was unclear about the success of the back up and the reasons for failure.
  •  We fixed an issue which resulted in users being unable to remove or edit JSON from Edge site capability fields. With this fix, if you enter JSON into an Edge site capability field, you can edit or remove it as expected. (ticket #137486)

Search

  • If a community or domain contains HTML tags in its description and you search for the community or domain after reindexing search, the HTML tags are no longer shown in the search results.

Browser Extension

  • Collibra Browser Extension no longer crashes when opening Power BI or Tableau Dashboards that don’t have business context relations or related assets. (ticket #142240, 142956, 143177)

Collibra Console

  • Collibra Console users with a Read role can no longer delete loggers. (ticket #140155)
  • Using a Collibra Console password containing some special characters no longer prevents you from signing in. (ticket #140155)

Security

  • After a password reset, active REST basic auth sessions are now closed.
  • User accounts that are converted to SSO are now automatically activated. (ticket #130543)
  • The latest UI no longer fails to handle the sign-in process for Collibra for Desktop and Collibra for Mobile.

API

  • An Output Module request with a ViewConfig that contains an incorrect relation type now returns a specific 400 error response instead of a generic 500.

Miscellaneous

  • Upgrading from Collibra version 5.9.1 to Collibra Platform Self-Hosted when the installation directory and collibra_data directory are on different partitions no longer fails.

Featuresin preview

A public preview is an upcoming feature or product that is made available to all customers before it is fully ready for general availability so it can be tested and evaluated early. Learn more
  • In the latest UI, you can now enable the creation of collections, ad hoc lists of assets to collect, for example, a list of favorite assets or assets that you want to review in Collibra. Because collections can be meaningful for others, you can also share collections. An icon is now available on the asset pages to add the asset to a collection. If available, the collection button is shown in the asset header. (idea # DCC-I-3123, DCC-I-27)

Collibra maintenance updates

Collibra 2024.05.1

  • A missing Lucene library no longer prevents Collibra from starting after an upgrade to version 2024.05.
  • Selecting multiple values in a multi-value Collibra data entry workflow form component no longer causes an error. (ticket #146563)
  • We have added an option for Collibra Support to enable some workflows to run in parallel with indexing jobs. (ticket #138154)
  • On the search page, the "Search in Fields" section no longer shows blank or irrelevant fields under certain conditions. (ticket #145761)
  • Reaching the maximum number of API call log entries no longer prevents Collibra from starting after an upgrade to version 2024.05.

Collibra 2024.05.2

  • We fixed a performance regression on the REST GET /relations API and the Java RelationsApi.findRelations() when filtering relations that have a specific asset as a source and/or target. (ticket #146092, 146321, 146649, 146788, 146994, 147169, 147224, 147340, 147835, 148145, 148814, 148823, 148908, 149010)

Collibra 2024.05.3

  • Opening asset pages of Business Assets, Governance Assets, Data Assets and Technology assets is now a lot faster.

Collibra 2024.05.4

  • If a workflow is started by either another workflow started from a schedule or by an asynchronous script task in another workflow, it no longer ignores form properties that its parent workflow defines.

Edge and Data Lineage updates

These updates contain security and bug fixes for Data Lineage, Edge sites and their capabilities. These releases may be planned outside the regular monthly or quarterly release. You'll see the fix versions if you are manually upgrading an Edge site or reviewing logs.

May 5, 2024
(data-lineage-2024.05.1) 

Collibra Data Lineage Service

  • The Source ID column is added in the source code list in the technical lineage graph for easier troubleshooting.
  • Previously, technical lineage for SAP HANA from calculated views was not generated for a calculated view when a mapping tag is missing. The Column mapping not defined error appeared in the Status tab. However, mapping tags are not included in SAP HANA when the source fields match the target fields exactly. Now, CollibraData Lineage processes lineage appropriately when the source fields match the target fields.
  • When ingesting Snowflake SQL data sources, the Collibra Data Lineage service instances now support COPY INTO file statements where the filename ends with -[number].[extension] (ticket # 117190)
  • When integrating Looker, two queries with the same name are now correctly ingested. Previously, one was skipped.

May 12, 2024
(data-lineage-2024.05.2) 

Collibra Data Lineage Service

  • We made various small improvements to the overall performance and user experience of Data Lineage.

May 19, 2024
(collibra-edge-2024.05.14) 
(data-lineage-2024.05.1) 

Edge infrastructure

  • We have made the following updates to the enable/disable classification commands for Edge sites:
    • We have removed the helm parameter prefix collibra_edge from the commands. For example, --set collibra_edge.collibra.classificationenabled is now --set collibra.classification.enabled
    • For new Edge sites:
      • K3s installations : use the install-master.sh command to install your Edge site with Classification enabled.
      • Managed Kubernetes installations: use the ./edgecli to install your Edge site with Classification enabled.
    • For existing Edge sites installed on either k3s or managed Kubernetes, use the .edge/cli command to enable or disable classification.
  • We fixed an issue where Edge sites failed to be restored from a backup when the backup contained old namespace secrets. With this fix, Edge sites can be restored from backups successfully, as expected.
  • We have fixed a problem where Shared Storage connection files uploaded via the CLI were evicted, or disappeared, from Edge after 30 minutes if ttlSeconds was larger than 24 days. With this fix, files can be stored for up to 180 days.
  • We fixed an issue where Edge site IDs were excluded from API calls. This resulted in problems interacting with Edge site features, for example, connection tests could not be successfully run or completed. (ticket #145960)
  • We have increased the /var/lib/kubelet partition storage size system requirements from 5 GB to 200 GB for Edge sites installed on k3s. With this increased storage size, larger capabilities, such as technical lineage, can now successfully be run at the same time.
    Note Each concurrent technical lineage capability job requires 10 GB of space on the /var/lib/kubelet partition. If you need to run more capabilities concurrently than you have space for, we recommend migrating your Edge site to a managed Kubernetes cluster.

Security

  • We have improved the security of Catalog JDBC ingestion, Catalog Data Classification, and Catalog JDBC Sampling.

Metadata integrations

  • You can now integrate Dataplex custom label properties.

Collibra Data Lineage Service

  • When ingesting Spark SQL data sources, the Collibra Data Lineage service instances now support the use of "Cluster" as an alias. (ticket #139586)
  • When ingesting Snowflake SQL data sources, the Collibra Data Lineage service instances:
    • Now support the TRIM function on a column and use the column name as an alias. (ticket #142037)
    • Now correctly handle lower-case column names. (ticket #145665)

Lineage harvester (CLI and Edge)

  • When harvesting metadata from a data source that is pointing to a folder (if you’re using the CLI lineage harvester) or via shared storage connection (if you’re using Edge) harvesting will now fail if no files are present. Previously, an empty lineage was created. If lineage already existed, it was overwritten by the empty lineage.
  • When ingesting Oracle data sources via Edge with a JDBC connection, you can now use the Database Link Mapping field in your Edge capability to configure, per data source, the database and schema to which a DBLink points.
    Note Full support for this property is not yet available, as we are finalizing the backend work. We will keep you informed of developments.

May 26, 2024
(collibra-edge-2024.05.21) 
(data-lineage-2024.05.4) 

Edge infrastructure

  • We fixed an issue which resulted in shared storage connection capabilities only being able to process up to 100 shared files. With this fix, the continuationToken now works in SharedFolderClient, as expected. (ticket #147005, 146285, 145984, 145942, 145864)
  • We fixed an issue which prevented some Edge sites from upgrading successfully. With this fix, Edge site successfully upgrade, as expected.

Security

  • We have improved the security of Edge.

Collibra Data Lineage Service

  • When integrating MicroStrategy, a relation of the type “Data Asset is source for / source BI Report” is now created between MicroStrategy reports and metrics. Previously, a relation of this type was created only between MicroStrategy reports and attributes. (ticket #142280, 142429, 144159, 145143)
  • We’ve made the following improvements to the Collibra Data Lineage service instances:
    • For MySQL data sources: Double quotation marks around the column names in the SELECT part of INSERT INTO ... WITH SELECT statements are now supported. (ticket #144427)
    • For BI tools and ETL tools with Snowflake SQL data sources: correct lineage is now generated even if there are parsing errors. (ticket #143696)
    • For Oracle data sources:
      • BULK as an alias for column names is now supported.
      • Synonyms are now correctly processed when the service name is in lowercase. (ticket #144606)
    • For custom technical lineage: Source code highlighting is now correctly generated even if the specified highlight span starts or ends on a "newline" character.
    • For Snowflake data sources: "file_format" in the FROM clause is now supported. (ticket #144333)
    • SQL statements with many nested PIVOT statements are now supported. (ticket #140141)

June 1, 2024
(collibra-edge-2024.05.27) 
(data-lineage-2024.06.1) 

Edge infrastructure

  • We fixed an issue where, when upgrading Edge sites with custom docker registries installed via the Helm chart method, the Edge site custom settings were overwritten and users received the following error: imagePullErr. With this fix, Edge sites with custom docker registries installed via the Helm chart method upgrade successfully, as expected.
  • We fixed an issue where Edge sites new installed on a managed Kubernetes cluster could not successfully upgrade and remained in the previous version. With this fix, Edge sites can upgrade successfully. (ticket #147207)

Security

  • We have improved the security of Collibra Protect, Dataplex Synchronization, ADLS synchronization, Databricks Unity Catalog synchronization, Technical Lineage for Databricks Unity Catalog.

Metadata integrations

  • We have fixed an issue for Databricks integration via Edge for no_proxy servers. (ticket #144209)

Collibra Data Lineage Service

  • We have updated MySQL driver to version 8.4.0.
  • Collibra Data Lineage now supports stored procedures when ingesting Snowflake data sources via the SQL ingestion method.
  • When integrating SSRS-PBRS, if a data object no longer exists in SSRS-PBRS, is corrupted, orphaned, or otherwise cannot be ingested, the integration process now skips the data object and continues. Previously, an HTTP 500 error was thrown. Orphaned reports are now mentioned in the log file. Lineage still can’t be created for corrupted, missing or orphaned data objects. (ticket #140506)

Lineage harvester (CLI and Edge)

  • We have made several improvements to the BigQuery Edge connection configuration and capability template:
    • You no longer need the project ID in the connection configuration.
    • The Region field has been removed from the capability.
    • You still need at least one project ID in the capability.
    • Edge harvests the metadata from all project IDs for which the service account has permissions. Harvesting is not limited to the project IDs that you include in the capability.
    Note With this feature enhancement, you need to update your driver to the newest version when creating the JDBC connection. Our testing was successful with version 23.0.8839.0. Using an older CData driver results in a "java.sql.SQLException: Cannot invoke "java.util.LinkedList.size() error".
  • When integrating SSRS-PBRS, if a data object no longer exists in SSRS-PBRS, is corrupted, orphaned, or otherwise cannot be ingested, the integration process now skips the data object and continues. Previously, an HTTP 500 error was thrown. Orphaned reports are now mentioned in the log file. Lineage still can’t be created for corrupted, missing or orphaned data objects. (Ticket #140506)

June 3, 2024
(data-lineage-2024.06.1.1) 

Collibra Data Lineage Service

  • If columns in Power BI are renamed before they are transformed via the Power Query M function Table.TransformColumnNames (Text.Upper, Text.Lower or Text.Proper), the original column names are now shown in the database node.

June 9, 2024
(collibra-edge-2024.05.35) 
(data-lineage-2024.06.2) 

Security

  • We have improved the security of Data Classification and Collibra Protect.

Collibra Data Lineage Service

  • We made various small improvements to the overall performance and user experience of Data Lineage.

June 16, 2024
(collibra-edge-2024.05.42) 
(data-lineage-2024.06.3) 

Security

  • We have improved the security of Catalog JDBC ingestion, JDBC Profiling and Data Lineage.

Metadata integrations

  • The Databricks integration can now handle foreign keys that point to tables that aren't created yet by the synchronization. Now, if the table is not created yet or if it is excluded from the integration through filters, the table will be created with basic information only. This fixes an issue that could cause problems when synchronizing Databricks data. (ticket #145990)
  • The Databricks integration lets you sync more information about your tables and views. You can now also include the following system properties via the “Extensible Properties Mappings” field:
    • tables.systemAttributes.catalog_name
    • tables.systemAttributes.schema_name
    • views.systemAttributes.catalog_name
    • views.systemAttributes.schema_name
  • The Databricks integration has been enhanced to prevent unexpected errors when syncing data that contains null characters. (ticket #142310, 146120)
  • The Synchronization Results dialog box for an S3 synchronization now includes the names of the Glue databases created by the synchronization. This information is crucial for integrating descriptions from S3 after you have synchronized the data source.

Collibra Data Lineage Service

  • Collibra Data Lineage now correctly resolves columns when the necessary database model is shared from an independent data source to a dependent data source.
    Note The previous means of sharing database models, meaning via the "useSharedDbModel" property is deprecated. You should now only use the "dependentSourceIds" property (if you use the lineage harvester) or the “Dependent On Sources” field (if you use Edge).
  • Technical lineage for Snowflake with the SQL-API ingestion mode now supports objects and attributes with \ in their names.
  • The Time Frame field in the technical lineage for Databricks Unity Catalog capability is no longer required. If no value is specified, a default value of 365 is used.
  • When ingesting Teradata data sources, Collibra Data Lineage service instances now support SQL statements that have the RECURSIVE keyword in WITH clauses.

Lineage Harvester (CLI and Edge)

  • All technical lineage and BI capabilities now correctly conform to SSLContext and work in all specified network environments. The connection to the Collibra Data Lineage service instances now also work in all network environments. The optional “Custom Certificate” field in the Power BI and Tableau Edge connections can still be used, but will soon be deprecated.
  • When integrating SSRS-PBRS, the lineage harvester now sends Transmission Control Protocol (TCP) keepalive probes, to allow the TCP connection to remain active for a period of time even if no data is exchanged. Previously, integration was failing because the TCP connection closed while the Collibra Data Lineage service instance was processing metadata.
  • When ingesting Hive data sources, Collibra Data Lineage now supports Hive instances for which concurrency is disabled. (ticket #146666)
  • We have updated the Amazon Redshift JDBC driver to version 2.1.0.29.
  • When ingesting Oracle data sources via the lineage harvester, you can now use the optional “jdbcUrl” property to override the default JDBC URL used to connect to the database.

June 23, 2024
(collibra-edge-2024.05.49) 
(data-lineage-2024.06.4) 

Infrastructure

  • We addressed a possible "classpath" conflict when trying to establish JDBC connections between Edge sites and data sources.
  • Changes to the outbound proxy authentication were completed to prevent Edge sites from being stuck in known states (for example Read-Only) for an extended amount of time.

Collibra Data Lineage Service

  • When ingesting Snowflake data sources, Collibra Data Lineage can now handle views with a high volume of indirect lineage.
  • When integrating Power BI, the Power Query M function “AddAndExpandDimensionColumn” now supports custom table names.

June 25, 2024
(collibra-edge-2024.05.51-1) 
(data-lineage-2024.06.4.1) 

Infrastructure

  • We improved how logs are sent from Edge to DataDog.

Collibra Data Lineage Service

  • We resolved an issue that caused data source synchronization to fail for some EMEA customers on June 24, 2024. The following Collibra Data Lineage service instance was affected:
    ServerIP addressDNS name
    techlin-gcp-eu35.205.146.124techlin-gcp-eu-collibra.com
    If you are experiencing errors, please open a Support ticket.

June 30, 2024
(collibra-edge-2024.05.56) 
(data-lineage-2024.07.1) 

Infrastructure

  • We have improved the file upload speed of the Edge Shared Storage connection. You can now upload up to 18,000 different file types in under 1 minute.
  • We have added new and improved existing commands for the Edge Shared Storage connection via the Edge CLI.
    • Multi-folder-upload: allows you to upload multiple folders at a time.
    • Folder-delete: now deletes all specified files or folders. Previously this was limited to the first 1,000.
    • Created date and file size are now provided in command returns.

Security

  • We have improved the security of Data Classification, Google Vertex AI, Technical Lineage for Databricks Unity Catalog and Databricks Unity Catalog synchronization.

Protect

  • If you now create multiple rules for a table without a standard and delete one of the rules, Protect now removes policies corresponding to the deleted rule from the table.
  • Protect for Databricks no longer fails if the name of the catalog begins with a number.

Collibra Data Lineage Service

  • When integrating Tableau with Snowflake data sources, Collibra Data Lineage no longer creates multiple assets if identical node paths are found in multiple Snowflake statements.
  • The shared database model feature is now generally available. Sharing database models allows you to provide table-definition details from an independent data source to a data source that is dependent on those details. This mitigates analysis errors and allows for a complete lineage that includes lineage from the SQL statements from dependent data sources.
  • When ingesting Teradata data sources, the Collibra Data Lineage service instances now support the DATE constant in BTEQ scripts.
  • When ingesting Sybase data sources, the Collibra Data Lineage service instances now support special values in SQL statements, for example "current date".
  • When integrating Power BI, the Power Query M function “Cube.AddAndExpandDimensionColumn” now supports custom table names.
  • When ingesting Snowflake data sources via the SQL ingestion method, Collibra Data Lineage now fully support the analysis of stored procedures, including variable tracking across SQL statements. This allows for complete lineage from stored procedures.
  • Previously, when you create technical lineage for Informatica PowerCenter, Collibra Data Lineage failed if a column name was empty. Now,Collibra Data Lineage continues without failing.
  • Previously, when you create technical lineage for SQL Server Integration Services (SSIS), if a DTS:ConnectionManager did not have a DTS:ConnectionString, no lineage is generated for the entire batch. Now, if the connection string is not provided, the applicable transformations will be skipped, and lineage will be generated for the rest of the batch.

Lineage Harvester (CLI and Edge)

  • When integrating MicroStrategy, Collibra Data Lineage now correctly handles "java.lang.IllegalStateException" errors. They no longer cause the integration to fail.

July 7, 2024
(collibra-edge-2024.05.63) 

Security

  • We have improved the security of Edge.

July 14, 2024
(collibra-edge-2024.05-70) 

Security

  • We have improved the security of Data classification.

August 4, 2024
(collibra-edge-2024.05.91) 

Security

  • We have improved the security of Protect.

August 11, 2024
(collibra-edge-2024.05.98) 

Security

  • We have improved the security of Data classification via Edge.

August 24, 2024
(collibra-edge-2024.05-111) 

Infrastructure

  • We have made improvements to the Edge installer to reduce future security vulnerabilities.

Security

  • We have improved the security of Databricks Unity Catalog synchronization, Technical Lineage for Databricks Unity Catalog and Azure ML.

Metadata integrations

  • The Databricks Unity Catalog integration no longer fails if tables are deleted in Databricks during the synchronization process.