Only show release notes of Collibra Platform for Government-certified features

Release 2024.06

Release information

  • Publication dates:
    • June 4, 2024: Documentation Center and release notes
  • Release dates of Collibra Platform:
    • June 9, 2024: Collibra 2024.06.0 (Non-production environments)
    • June 23, 2024: Collibra 2024.06.1 (Non-production environments)
    • June 30, 2024: Collibra 2024.06.2 (Production environments)
    • July 7, 2024: Collibra 2024.06.3
    • On demand: Collibra 2024.06.4 and Collibra 2024.06.5
  • Edge and Data Lineageupdates
    • May 5, 2024: Edge 2024.05-0 and Data Lineage 2024.05.1
    • May 12, 2024: Data Lineage 2024.05.2
    • May 19, 2024: Edge 2024.05-14 and Data Lineage 2024.05.3
    • May 26, 2024: Edge 2024.05-21 and Data Lineage 2024.05.4
    • June 1, 2024: Edge 2024.05-27 and Data Lineage 2024.06.1
    • June 3, 2024: Data Lineage 2024.06.1.1
    • June 9, 2024: Edge 2024.05-35
    • June 16, 2024: Edge 2024.05-42
    • June 23, 2024: Edge 2024.05-49
    • June 25, 2024: Edge 2024.05-51 and Data Lineage 2024.06.4.1
    • June 30, 2024: Edge 2024.05-56 and Data Lineage 2024.07.1
    • July 7, 2024: Edge 2024.05-63 and Data Lineage 2024.07.2
    • July 14, 2024: Edge 2024.05-70
    • August 4, 2024: Edge 2024.05-91
    • August 11, 2024: Edge 2024.05.98
    • August 24, 2024: Edge 2024.05.111

Enhancements

Data Catalog

  • To enhance security, regular expressions in Unified Data Classification are now validated for potential vulnerabilities before saving them.

Data Governance

  • Relation tables on asset pages now load faster.

Data Privacy

  • The Data Privacy Operating Model file is now available on our Downloads page. This file contains the entire Data Privacy operating model, which includes the following:
    • Data Privacy asset types added via the installation files.
      Examples: Contract, Remediation Plan, and Safeguard asset types.
    • Changes made by the installation files to the out-of-the-box asset types in Collibra.
      Examples: New characteristics added to Business Process, Data Category, and Data Set asset types.

Security

  • We have deployed a new Content Security Policy (CSP) to improve the security of our platform and keep your data safe. If you encounter any issues after this upgrade, such as blocked iFrames, content or images, please contact Collibra Support.
  • After a session timeout, you can now sign in on the main sign-in page, which then redirects you to the page you were previously viewing. (ticket #141585, 144603)
  • To reduce the number of log entries, we no longer check if a session is active while the user is inactive or the Collibra browser page is not in focus for more than 20 seconds. We recheck for a valid sessions as soon as the user is active or the page is in focus again. (ticket #92086, 120922, 141275)

API

  • We have improved the output module performance for complex queries that follow relations of related assets multiple times. (ticket #144186)

Fixes

Latest UI

  • The "Quality" tab on Asset pages in the latest UI now correctly works with third-party data quality integrations. This fixes a previous issue where the tab wasn't functioning as expected. (ticket #134355, 141126, 144414, 146870, 147581, 147598, 149115)

  • Dashboards:
    • In the Text widget, the source code icon, </>, is now removed from the "Rich Text Editor" box.
      Tip To paste text from external sources, you can use the "HTML Editor" box instead.
    • The visibility option you select when duplicating a dashboard is now saved in the new dashboard. (ticket #143643)
    • The size you set for a widget is now retained even after you configure the widget.
  • Views:
    • To prevent excessively long tiles, we have introduced a maximum height limit of 2,000 pixels for tiles.
    • In a table view, asset names in complex relations are now separated by commas. (ticket #143908)
    • When editing an attribute in a table view, you now have the option to clear any formatting.
    • In Stewardship, the default view is now shown in the "Select View" drop-down list. (idea #DCC-I-3157)
  • The year picker window now includes a scroll bar to allow you to find and select the required year.
  • When editing a page now, the top navigation bar is hidden and banners are shown only at the top.
  • You can now customize the background color of hovered step titles in non-linear steppers, using the new "Hover Background" field on the Custom Theme page.
    Tip A non-linear stepper allows you to move forward or back in the steps.
  • The Mentions tab is now shown only on your own profile page. (ticket #140721)
  • Search:
    • If you enter something in the search box with a community or domain page open, the top results and the recently viewed results are now filtered by the current community or domain.
    • You can now right-click a quick search result and open it in a new tab.
    • On the search page, the "Selected Filters" section now shows custom facets even if their IDs contain only lowercase letters.
    • The in preview label is no longer shown on the "Similar Data" tab of the asset preview opened from Data Marketplace in Search.
    • Filter facets that are configured to be expanded now appear expanded for Data Marketplace in Search.
    • The Certified search facet and its values are now correctly translated.

AI Governance

  • The AI Governance landing page for AI Legal Reviewers now only shows assessments to review that are related to AI Use Cases.

Data Governance

  • We have resolved an issue that prevented users from checking out the Data Basket when Data Privacy wasn't activated. (ticket #144714, 145904, 146550, 146720)
  • We have fixed a performance regression on the REST GET /relations API and the Java RelationsApi.findRelations() when filtering relations that have a specific asset as a source and/or target. (ticket #146092, 146321, 146649, 146788, 146994, 147224, 147340, 147835)
  • The layout editor now displays a correct error message when text exceeds the character limit.
  • When selecting attachments dated as the current date and then going to the history tab, there was an incorrect error message appearing. This issue has been resolved.
  • Previously, the system would log an error when attempting to view samples on a resource without proper permissions. This was confusing because it implied a technical problem, when it was a permission issue. (ticket #139034)
  • The operating model migration feature now correctly imports relation types, even if their roles are very similar. This fixes an issue that previously prevented successful import. You can now import relation types even if their roles or co-roles are substrings of existing relation types. (ticket #144715)
  • In the latest UI, the Activated column that is shown under Settings > Operating Model > Asset Types, now shows accurate information based on whether the product the asset type is associated with is part of your package or not. (ticket #146636, #147505, #148010)
  • Dialog boxes in the latest UI that require selecting a user also display the email addresses of the users. (ticket #140556)
  • You can now import attributes and relations via publicId in the import API.

Data Marketplace

Diagrams

  • Complex relation nodes are now shown in the diagram preview panel. (ticket #136993, 134629, 140840)

Protect

  • Synchronization no longer fails when multiple data access rules are created for different assets that have the same data classification with different masking types. (ticket #139275)

Workflows

  • You can start again a global workflow from a link that points to the tasks page and the #wfid parameter.
  • Using a button form property that has the writable parameter cleared no longer prevents the task form from being submitted. (ticket #142802)
  • You can again use a datetime form property in full screen mode, in the classic UI. (ticket #141105)
  • Using an out-of-the-box workflow in the latest UI to create an asset in a domain where you have no view permissions no longer prevents the workflow dialog box from closing. (ticket #143791)

Miscellaneous

  • Asset views for communities with over 32,000 domains now load properly even when filtering. (ticket #146279)
  • The "License usage monthly report" is no longer missing data from previous months. (ticket #146103, 146104)

Edge and Data Lineage updates

These updates contain security and bug fixes for Data Lineage, Edge sites and their capabilities. These releases may be planned outside the regular monthly or quarterly release. You'll see the fix versions if you are manually upgrading an Edge site or reviewing logs.

May 5, 2024
(data-lineage-2024.05.1) 

Collibra Data Lineage Service

  • The Source ID column is added in the source code list in the technical lineage graph for easier troubleshooting.
  • Previously, technical lineage for SAP HANA from calculated views was not generated for a calculated view when a mapping tag is missing. The Column mapping not defined error appeared in the Status tab. However, mapping tags are not included in SAP HANA when the source fields match the target fields exactly. Now, CollibraData Lineage processes lineage appropriately when the source fields match the target fields.
  • When ingesting Snowflake SQL data sources, the Collibra Data Lineage service instances now support COPY INTO file statements where the filename ends with -[number].[extension] (ticket # 117190)
  • When integrating Looker, two queries with the same name are now correctly ingested. Previously, one was skipped.

May 12, 2024
(data-lineage-2024.05.2) 

Collibra Data Lineage Service

  • We made various small improvements to the overall performance and user experience of Data Lineage.

May 19, 2024
(collibra-edge-2024.05.14) 
(data-lineage-2024.05.1) 

Edge infrastructure

  • We have made the following updates to the enable/disable classification commands for Edge sites:
    • We have removed the helm parameter prefix collibra_edge from the commands. For example, --set collibra_edge.collibra.classificationenabled is now --set collibra.classification.enabled
    • For new Edge sites:
      • K3s installations : use the install-master.sh command to install your Edge site with Classification enabled.
      • Managed Kubernetes installations: use the ./edgecli to install your Edge site with Classification enabled.
    • For existing Edge sites installed on either k3s or managed Kubernetes, use the .edge/cli command to enable or disable classification.
  • We fixed an issue where Edge sites failed to be restored from a backup when the backup contained old namespace secrets. With this fix, Edge sites can be restored from backups successfully, as expected.
  • We have fixed a problem where Shared Storage connection files uploaded via the CLI were evicted, or disappeared, from Edge after 30 minutes if ttlSeconds was larger than 24 days. With this fix, files can be stored for up to 180 days.
  • We fixed an issue where Edge site IDs were excluded from API calls. This resulted in problems interacting with Edge site features, for example, connection tests could not be successfully run or completed. (ticket #145960)
  • We have increased the /var/lib/kubelet partition storage size system requirements from 5 GB to 200 GB for Edge sites installed on k3s. With this increased storage size, larger capabilities, such as technical lineage, can now successfully be run at the same time.
    Note Each concurrent technical lineage capability job requires 10 GB of space on the /var/lib/kubelet partition. If you need to run more capabilities concurrently than you have space for, we recommend migrating your Edge site to a managed Kubernetes cluster.

Security

  • We have improved the security of Catalog JDBC ingestion, Catalog Data Classification, and Catalog JDBC Sampling.

Metadata integrations

Collibra Data Lineage Service

  • When ingesting Spark SQL data sources, the Collibra Data Lineage service instances now support the use of "Cluster" as an alias. (ticket #139586)
  • When ingesting Snowflake SQL data sources, the Collibra Data Lineage service instances:
    • Now support the TRIM function on a column and use the column name as an alias. (ticket #142037)
    • Now correctly handle lower-case column names. (ticket #145665)

Lineage harvester (CLI and Edge)

  • When harvesting metadata from a data source that is pointing to a folder (if you’re using the CLI lineage harvester) or via shared storage connection (if you’re using Edge) harvesting will now fail if no files are present. Previously, an empty lineage was created. If lineage already existed, it was overwritten by the empty lineage.
  • When ingesting Oracle data sources via Edge with a JDBC connection, you can now use the Database Link Mapping field in your Edge capability to configure, per data source, the database and schema to which a DBLink points. For complete information, example scenarios, and configuration advice, see the Database Link Mapping field description in Create a technical lineage via Edge.
    Note Full support for this property is not yet available, as we are finalizing the backend work. We will keep you informed of developments.

May 26, 2024
(collibra-edge-2024.05.21) 
(data-lineage-2024.05.4) 

Edge infrastructure

  • We fixed an issue which resulted in shared storage connection capabilities only being able to process up to 100 shared files. With this fix, the continuationToken now works in SharedFolderClient, as expected. (ticket #147005, 146285, 145984, 145942, 145864)
  • We fixed an issue which prevented some Edge sites from upgrading successfully. With this fix, Edge site successfully upgrade, as expected.

Security

  • We have improved the security of Edge.

Collibra Data Lineage Service

  • When integrating MicroStrategy, a relation of the type “Data Asset is source for / source BI Report” is now created between MicroStrategy reports and metrics. Previously, a relation of this type was created only between MicroStrategy reports and attributes. (ticket #142280, 142429, 144159, 145143)
  • We’ve made the following improvements to the Collibra Data Lineage service instances:
    • For MySQL data sources: Double quotation marks around the column names in the SELECT part of INSERT INTO ... WITH SELECT statements are now supported. (ticket #144427)
    • For BI tools and ETL tools with Snowflake SQL data sources: correct lineage is now generated even if there are parsing errors. (ticket #143696)
    • For Oracle data sources:
      • BULK as an alias for column names is now supported.
      • Synonyms are now correctly processed when the service name is in lowercase. (ticket #144606)
    • For custom technical lineage: Source code highlighting is now correctly generated even if the specified highlight span starts or ends on a "newline" character.
    • For Snowflake data sources: "file_format" in the FROM clause is now supported. (ticket #144333)
    • SQL statements with many nested PIVOT statements are now supported. (ticket #140141)

June 1, 2024
(collibra-edge-2024.05.27) 
(data-lineage-2024.06.1) 

Edge infrastructure

  • We fixed an issue where, when upgrading Edge sites with custom docker registries installed via the Helm chart method, the Edge site custom settings were overwritten and users received the following error: imagePullErr. With this fix, Edge sites with custom docker registries installed via the Helm chart method upgrade successfully, as expected.
  • We fixed an issue where Edge sites new installed on a managed Kubernetes cluster could not successfully upgrade and remained in the previous version. With this fix, Edge sites can upgrade successfully. (ticket #147207)

Security

  • We have improved the security of Collibra Protect, Dataplex Synchronization, ADLS synchronization, Databricks Unity Catalog synchronization, Technical Lineage for Databricks Unity Catalog.

Metadata integrations

  • We have fixed an issue for Databricks integration via Edge for no_proxy servers. (ticket #144209)

Collibra Data Lineage Service

  • We have updated MySQL driver to version 8.4.0.
  • Collibra Data Lineage now supports stored procedures when ingesting Snowflake data sources via the SQL ingestion method.
  • When integrating SSRS-PBRS, if a data object no longer exists in SSRS-PBRS, is corrupted, orphaned, or otherwise cannot be ingested, the integration process now skips the data object and continues. Previously, an HTTP 500 error was thrown. Orphaned reports are now mentioned in the log file. Lineage still can’t be created for corrupted, missing or orphaned data objects. (ticket #140506)

Lineage harvester (CLI and Edge)

  • We have made several improvements to the BigQuery Edge connection configuration and capability template:
    • You no longer need the project ID in the connection configuration.
    • The Region field has been removed from the capability.
    • You still need at least one project ID in the capability.
    • Edge harvests the metadata from all project IDs for which the service account has permissions. Harvesting is not limited to the project IDs that you include in the capability.
    Note With this feature enhancement, you need to update your driver to the newest version when creating the JDBC connection. Our testing was successful with version 23.0.8839.0. Using an older CData driver results in a "java.sql.SQLException: Cannot invoke "java.util.LinkedList.size() error".
  • When integrating SSRS-PBRS, if a data object no longer exists in SSRS-PBRS, is corrupted, orphaned, or otherwise cannot be ingested, the integration process now skips the data object and continues. Previously, an HTTP 500 error was thrown. Orphaned reports are now mentioned in the log file. Lineage still can’t be created for corrupted, missing or orphaned data objects. (Ticket #140506)

June 3, 2024
(data-lineage-2024.06.1.1) 

Collibra Data Lineage Service

  • If columns in Power BI are renamed before they are transformed via the Power Query M function Table.TransformColumnNames (Text.Upper, Text.Lower or Text.Proper), the original column names are now shown in the database node.

June 9, 2024
(collibra-edge-2024.05.35) 
(data-lineage-2024.06.2) 

Security

  • We have improved the security of Data Classification and Collibra Protect.

Collibra Data Lineage Service

  • We made various small improvements to the overall performance and user experience of Data Lineage.

June 16, 2024
(collibra-edge-2024.05.42) 
(data-lineage-2024.06.3) 

Security

  • We have improved the security of Catalog JDBC ingestion, JDBC Profiling and Data Lineage.

Metadata integrations

  • The Databricks integration can now handle foreign keys that point to tables that aren't created yet by the synchronization. Now, if the table is not created yet or if it is excluded from the integration through filters, the table will be created with basic information only. This fixes an issue that could cause problems when synchronizing Databricks data. (ticket #145990)
  • The Databricks integration lets you sync more information about your tables and views. You can now also include the following system properties via the “Extensible Properties Mappings” field:
    • tables.systemAttributes.catalog_name
    • tables.systemAttributes.schema_name
    • views.systemAttributes.catalog_name
    • views.systemAttributes.schema_name

    For information on how to configure the integration, go to Synchronize Databricks Unity Catalog.

  • The Databricks integration has been enhanced to prevent unexpected errors when syncing data that contains null characters. (ticket #142310, 146120)
  • The Synchronization Results dialog box for an S3 synchronization now includes the names of the Glue databases created by the synchronization. This information is crucial for integrating descriptions from S3 after you have synchronized the data source. For more information about integrating descriptions, go to Integrated Amazon S3 data.

Collibra Data Lineage Service

  • Collibra Data Lineage now correctly resolves columns when the necessary database model is shared from an independent data source to a dependent data source. For complete information, see Sharing database models across data sources.
    Note The previous means of sharing database models, meaning via the "useSharedDbModel" property is deprecated. You should now only use the "dependentSourceIds" property (if you use the lineage harvester) or the “Dependent On Sources” field (if you use Edge).
  • Technical lineage for Snowflake with the SQL-API ingestion mode now supports objects and attributes with \ in their names.
  • The Time Frame field in the technical lineage for Databricks Unity Catalog capability is no longer required. If no value is specified, a default value of 365 is used.
  • When ingesting Teradata data sources, Collibra Data Lineage service instances now support SQL statements that have the RECURSIVE keyword in WITH clauses.

Lineage Harvester (CLI and Edge)

  • All technical lineage and BI capabilities now correctly conform to SSLContext and work in all specified network environments. The connection to the Collibra Data Lineage service instances now also work in all network environments. The optional “Custom Certificate” field in the Power BI and Tableau Edge connections can still be used, but will soon be deprecated.
  • When integrating SSRS-PBRS, the lineage harvester now sends Transmission Control Protocol (TCP) keepalive probes, to allow the TCP connection to remain active for a period of time even if no data is exchanged. Previously, integration was failing because the TCP connection closed while the Collibra Data Lineage service instance was processing metadata.
  • When ingesting Hive data sources, Collibra Data Lineage now supports Hive instances for which concurrency is disabled. (ticket #146666)
  • We have updated the Amazon Redshift JDBC driver to version 2.1.0.29.
  • When ingesting Oracle data sources via the lineage harvester, you can now use the optional “jdbcUrl” property to override the default JDBC URL used to connect to the database.

June 23, 2024
(collibra-edge-2024.05.49) 
(data-lineage-2024.06.4) 

Infrastructure

  • We addressed a possible "classpath" conflict when trying to establish JDBC connections between Edge sites and data sources.
  • Changes to the outbound proxy authentication were completed to prevent Edge sites from being stuck in known states (for example Read-Only) for an extended amount of time.

Collibra Data Lineage Service

  • When ingesting Snowflake data sources, Collibra Data Lineage can now handle views with a high volume of indirect lineage.
  • When integrating Power BI, the Power Query M function “AddAndExpandDimensionColumn” now supports custom table names.

June 25, 2024
(collibra-edge-2024.05.51-1) 
(data-lineage-2024.06.4.1) 

Infrastructure

  • We improved how logs are sent from Edge to DataDog.

Collibra Data Lineage Service

  • We resolved an issue that caused data source synchronization to fail for some EMEA customers on June 24, 2024. The following Collibra Data Lineage service instance was affected:
    ServerIP addressDNS name
    techlin-gcp-eu35.205.146.124techlin-gcp-eu-collibra.com
    If you are experiencing errors, please open a Support ticket.

June 30, 2024
(collibra-edge-2024.05.56) 
(data-lineage-2024.07.1) 

Infrastructure

  • We have improved the file upload speed of the Edge Shared Storage connection. You can now upload up to 18,000 different file types in under 1 minute.
  • We have added new and improved existing commands for the Edge Shared Storage connection via the Edge CLI.
    • Multi-folder-upload: allows you to upload multiple folders at a time.
    • Folder-delete: now deletes all specified files or folders. Previously this was limited to the first 1,000.
    • Created date and file size are now provided in command returns.

Security

  • We have improved the security of Data Classification, Google Vertex AI, Technical Lineage for Databricks Unity Catalog and Databricks Unity Catalog synchronization.

Protect

  • If you now create multiple rules for a table without a standard and delete one of the rules, Protect now removes policies corresponding to the deleted rule from the table.
  • Protect for Databricks no longer fails if the name of the catalog begins with a number.

Collibra Data Lineage Service

  • When integrating Tableau with Snowflake data sources, Collibra Data Lineage no longer creates multiple assets if identical node paths are found in multiple Snowflake statements.
  • The shared database model feature is now generally available. Sharing database models allows you to provide table-definition details from an independent data source to a data source that is dependent on those details. This mitigates analysis errors and allows for a complete lineage that includes lineage from the SQL statements from dependent data sources.
  • When ingesting Teradata data sources, the Collibra Data Lineage service instances now support the DATE constant in BTEQ scripts.
  • When ingesting Sybase data sources, the Collibra Data Lineage service instances now support special values in SQL statements, for example "current date".
  • When integrating Power BI, the Power Query M function “Cube.AddAndExpandDimensionColumn” now supports custom table names.
  • When ingesting Snowflake data sources via the SQL ingestion method, Collibra Data Lineage now fully support the analysis of stored procedures, including variable tracking across SQL statements. This allows for complete lineage from stored procedures.
  • Previously, when you create technical lineage for Informatica PowerCenter, Collibra Data Lineage failed if a column name was empty. Now,Collibra Data Lineage continues without failing.
  • Previously, when you create technical lineage for SQL Server Integration Services (SSIS), if a DTS:ConnectionManager did not have a DTS:ConnectionString, no lineage is generated for the entire batch. Now, if the connection string is not provided, the applicable transformations will be skipped, and lineage will be generated for the rest of the batch.

Lineage Harvester (CLI and Edge)

  • When integrating MicroStrategy, Collibra Data Lineage now correctly handles "java.lang.IllegalStateException" errors. They no longer cause the integration to fail.

July 7, 2024
(collibra-edge-2024.05.63) 

Security

  • We have improved the security of Edge.

July 14, 2024
(collibra-edge-2024.05-70) 

Security

  • We have improved the security of Data classification.

August 4, 2024
(collibra-edge-2024.05.91) 

Security

  • We have improved the security of Protect.

August 11, 2024
(collibra-edge-2024.05.98) 

Security

  • We have improved the security of Data classification via Edge.

August 24, 2024
(collibra-edge-2024.05-111) 

Infrastructure

  • We have made improvements to the Edge installer to reduce future security vulnerabilities.

Security

  • We have improved the security of Databricks Unity Catalog synchronization, Technical Lineage for Databricks Unity Catalog and Azure ML.

Metadata integrations

  • The Databricks Unity Catalog integration no longer fails if tables are deleted in Databricks during the synchronization process.