Release 2023.02

• Release information
• Highlights
• Metamodel changes
• New features
• Enhancements
• Fixes
• Features in preview
• Maintenance updates

Release information

• Release date of 2023.02.0: February 5, 2023
  • Upgrade non-production environments: February 5, 2023
  • Upgrade production environments: February 26, 2023
• Release date of 2023.02.1: February 19, 2023
• Release date of 2023.02.2: February 26, 2023
• Release date of 2023.02.3: March 12, 2023
• Relevant Jobserver version: 2023.02.0-39

Highlights

• Edge is now available by default in Collibra Platform! You can install Edge sites, configure connections and capabilities without the need to enable any console settings. You are not required to make any changes and this will not impact the overall functionality of your platform, unless you want to begin using Edge. Edge allows you to safely connect your Collibra Platform platform to your data sources that are hosted either in an on-premises or cloud environment. For more information on how to set up and use Edge, go to our Edge documentation.

• Edge sites now require access to Datadog's latest release. To do so, add https://7-40-1-app.agent.datadoghq.com to the allowlist of your network. Datadog logs and monitors Edge site performance and infrastructure health, which is required for troubleshooting and support.

• When you want to migrate from Jobserver to Edge, you don't need to start from scratch. Collibra allows you to migrate Schema assets that were registered via Jobserver to Edge via a dedicated workflow.
  To learn more about this workflow and watch videos on how to use it, go to the documentation.

• To speed up the classification process, you can now activate the automatic acceptance and rejection of data class suggestions.
  For information on the configuration, go to Configuring the classification thresholds.

• The Google Cloud Storage (GCS) file system integration via Edge is now generally available. This integration allows for the registration of Google Cloud Storage (GCS) as a data source in Collibra and the synchronization of the metadata. GCS is a service provided in the Google Cloud Platform (GCP). After the synchronization, the files and directories of the GCS file system are represented in Collibra by specific asset types, retaining the original names.

• You can now create a technical lineage via Edge, for Tableau, Power BI and all supported JDBC and ETL data sources. This feature is in public preview. Benefits include:

  • Seamless integration with Data Catalog.
  • The Edge User Interface (UI), instead of Command Line Interface (CLI).
  • Connections via Edge, instead of lineage harvester drivers.
  • Job scheduling via Data Catalog.

• The lineage harvester logic is now based on the UUIDs of attribute types in the BI tool operating models, instead of the attribute type names. This means that when you integrate any of the supported BI tools, you can now change the names of the ingested attribute types.

• Collibra Data Lineage now supports a new Snowflake mode. When the mode is set to SQL-API, the effects of Snowflake queries – including stored procedures, COPY INTO, and streams – are shown in technical lineage at the column level. The queries from stored procedures are included in lineage; however the name of the stored procedure is not exposed. Snowpipes are not supported.

• The new Organization filter in Data Marketplace allows data consumers to refine search results based on community and domain. Administrators can make this filter available for use in a few clicks. Go to Settings → Data Marketplace → Filters and make the Organization filter visible.

• Snowflake enforces data access policies through tags. These tags can now be included in the Data Catalog by registering the data source via Edge. This means that the Snowflake tags become searchable in our Data Catalog and can be used in business or policy processes and in workflows. This source tags feature is in public preview.
  For information on enabling the feature, go to the documentation.

• The new import wizard is now enabled by default. The improved asset import wizard is now using version 2 of the Import API for better performance, with a refreshed interface and more intuitive mapping.

• You can now edit additional characteristics within asset types that were previously locked by the system. Removing this limitation will allow you to build custom integrations, for a better integration experience when importing data from another source.
  You should not change the attributes created by Catalog synchronization, as the changes are not pushed to the data source. If you make changes to the attributes and re-synchronize the data source again, your updates will be overwritten with the information from the data source.

Metamodel changes

• The attribute Source Tags has been added to the Schema, Table, Database View and Column asset types. This change is related to the source tags feature (in preview).

New features

Data Catalog

• When you want to migrate from Jobserver to Edge, you don't need to start from scratch. Collibra allows you to migrate Schema assets that were registered via Jobserver to Edge via a dedicated workflow.
  To learn more about this workflow and watch videos on how to use it, go to the documentation.
• To speed up the classification process, you can now activate the automatic acceptance and rejection of data class suggestions.
  For information on the configuration, go to Configuring the classification thresholds.
• The Google Cloud Storage (GCS) file system integration via Edge is now generally available. This integration allows for the registration of Google Cloud Storage (GCS) as a data source in Collibra and the synchronization of the metadata. GCS is a service provided in the Google Cloud Platform (GCP). After the synchronization, the files and directories of the GCS file system are represented in Collibra by specific asset types, retaining the original names.

Data Marketplace

• The new Organization filter in Data Marketplace allows data consumers to refine search results based on community and domain. Administrators can make this filter available for use in a few clicks. Go to Settings → Data Marketplace → Filters and make the Organization filter visible.
• We now display the Community > Domain breadcrumb in the Data Marketplace search results and in the asset previews.

Assessments

• You can now protect the sensitive information in your assessments by providing granular permissions to your assessments. When conducting an assessment, on the new Properties tab, you can set one or more assignees for the assessment. In addition, you can define if the assessment will be visible to only the owner and the assignees (private) or to everyone (public). If the assessment is set to private, only the owner and the assignees can view, edit, complete, or submit the assessment.
  Note 

  • By default, the assessments that are created in 2023.02 or later are private.
  • By default, the assessments that were created prior to 2023.02 are public. That is, everyone can view those assessments. If an assessment that you created prior to 2023.02 contains sensitive information and is in the Draft status, we recommend that you change the view properties of that assessment so that only you and the assignees that you select can view the assessment.

Data Privacy

• A new Data Privacy experience is now available in Collibra Platform in the form of the Privacy landing page. This new, preconfigured landing page is intended to help Privacy Stewards effectively perform their most frequent Privacy job functions. It also contains answers to some frequently asked questions related to the Privacy tasks. You can access the Privacy landing page if it is enabled and if you have the new Privacy User global role with the Privacy global permission.
Note This new Privacy landing page will replace the Privacy Dashboard. The Privacy Dashboard will be deprecated as of 2023.05 (May, 2023). If, however, you are an existing customer of Data Privacy, the Privacy Dashboard will continue to be available to you, but Collibra will be unable to provide support in terms of fixes or enhancements. As of 2023.05, new installations of Data Privacy will not include the Privacy Dashboard.
• Collibra Protect now supports the enforcement of data protection standards on BigQuery.
• You can now view the data protection standards and data access rules that are applied to an asset directly on the asset page via the new Data Protection tab.
  Note The Data Protection tab is shown in the asset pages of the asset types that Collibra Protect supports, which are as follows:

  • Business Process
  • Data Category
  • Data Set
  • Custom asset types such as Column, Database, Schema, and Table, derived from the aforementioned asset types via prescriptive paths

Miscellaneous

• Administrators can now customize the Homepage to suit user needs. They can modify the heading and content in each section and define what cards to show, including their order.
• The following additional cards can now be shown on the Homepage: My Quick Links and My Tasks. On the My Quick Links card, you can add links to the pages that you want to quickly access.

Enhancements

Data Catalog

• The data profiling charts available in the Catalog have been improved for better usability.
• The integration results dialog boxes, for example for S3 integration, has been improved for better user experience.

Data Lineage and BI integrations

Note Collibra Data Lineage is a cloud-only feature.

• When you integrate Tableau:
  • Performance is significantly improved for Tableau users with the Explorer role.
  • The lineage harvester now filters out data objects for which you do not have permissions. (ticket #103076)
  • You can now view the source code in the technical lineage Sources tab page. The source code shows information about the processes visible in the technical lineage and shows warnings and errors where a process has failed. This enhancement does not affect the success rate of metadata analysis.
  • UUIDs no longer appear in the names of Tableau assets in a technical lineage, with the following exception: if Tableau data objects in a technical lineage hierarchy have the same full name, Collibra Data Lineage adds the UUIDs of the corresponding assets to the names in the technical lineage, to maintain uniqueness. For complete information, including how to resolve UUIDs in the names, see Technical lineage for Tableau.
• When you synchronize any supported BI tool, if a corresponding data object of an asset in Data Catalog can no longer be found in the data source, the asset is no longer deleted from Data Catalog. Instead, the status of the asset changes to “Missing from source”.
• When you integrate Power BI, the “content provider type” for data sets is now included in the source code.
• Collibra Data Lineage now supports Power Query parameters (public preview). For complete information, including how to ensure that the Power BI APIs return all parameters that are loaded in a report, see Working with Power Query parameters.
• Collibra Data Lineage now fully supports the optional property "deleteRawMetadataAfterProcessing" in your lineage harvester configuration file. You can use this property to delete your raw metadata from the Collibra Data Lineage service after processing. This property is applicable for all supported data sources.
• Collibra Data Lineage now supports the Power Query M function Databricks.Catalogs.

Data Marketplace

• Recommendations are now provided for new users too, based on their user group.
• Filters linked to a selected filter tab are presented in a collapsed format to prevent an overload of information in the Selected area. If users want to refine their search, they can expand the filter to get the details.
• Tableau Dashboard asset previews now show the number of report views as recorded in Tableau.
• When no secondary actions are defined, we now show a meaningful message.
• The URL to manage relation indexes has changed to data-marketplace/relation-indexes.
• You can no longer create relation paths that are exactly the same in one relation index.

Data Governance

• Scope page performance has been improved with the help of lazy loading and pagination of the organization tree. We have unified the scope page into a single version.
• The Users table CSV export will now include the "Latest Workflow Participation" and "Required License" columns. Note that the "Required License" column will contain the values "AUTHOR" or "CONSUMER" for "Standard" and "Read-only" licenses respectively.
• You can now import responsibilities using the Import UI v2 wizard.

Data Privacy

• Collibra Protect now supports custom asset types, for example, the asset types that you have modified or those that you have created. In addition, you can now customize the prescriptive paths for the asset types through APIs. The custom asset types and the customized prescriptive paths are applied to data protection standards and data access rules. This enables you to use Collibra Protect on customized operating models and with customized asset types.
  Note To customize the prescriptive paths through APIs, you need the Protect > Administration global permission.

Insights Data Access and Usage Analytics

Note Insights Data Access and Usage Analytics are cloud-only features.

• A new column named relation_id is now added to the Relation table. This new column contains the UUID of the relation, and it is used to uniquely identify each row, in combination with the direction column.

Collibra Console

• The package swagger-ui-bundle has been upgraded to version 4.15.2.

Miscellaneous

• On the Most Viewed Assets and Recommended Assets cards of the Homepage, a checkmark is now shown next to the certified assets.

Fixes

Data Catalog

• We no longer create temporary files with format tmp-[DRIVER_TYPE]-driver-[UUID] when registering a data source via Jobserver. (ticket #99055)
• You can now run profiling via Edge for databases that are related to Schema assets which are not created via Edge. Note that Schema assets that are not created via Edge are not visible on the Database asset Configuration page and will not be profiled. (ticket #101810)
• You no longer receive an error when you add a column to a Table asset via the Table asset page or when you add a Data Element to a Data Set asset via the Data Set asset page. (ticket #102659, 99556, 101420, 101764, 102659, 103599)
• Columns with unsupported data types are now excluded from profiling queries. The exclusion applies to columns mapped to following java.sql.Types: ARRAY, BINARY, BLOB, CLOB, DATALINK, DISTINCT, JAVA_OBJECT, LONGVARBINARY, NCLOB, NULL, OTHER, REF, REF_CURSOR, ROWID, SQLXML, STRUCT, VARBINARY. (ticket #100005)
• The 'Request Sample Data' option in the 'Actions' menu of Table and Column asset pages is now enabled and disabled in the same way as the 'Request Sample Data' button.
• We are now properly calculating the number of rows to be profiled when running profiling via Edge on a random subset of rows for Databricks and Spark. (ticket #101777)
• We no longer fail to profile tables or create sample data for tables in Databricks if the ratio of the 'rows to profile' to the 'total row count' is smaller than 0.0005, for example (500 / 1,000,000). (ticket #100873, 101777)
• We now define all relations from child to parent assets during an S3 synchronization to ensure there are no missing relations. (ticket #96736)

Data Lineage and BI integrations

Note Collibra Data Lineage is a cloud-only feature.

• Unexpected input on the Collibra Data Lineage service instance no longer causes metadata analysis to fail.
• When you ingest Teradata data sources, the Collibra Data Lineage service instances now support the XC string suffix in SYSLIB.OTRANSLATE functions. (ticket #99256)
• When you ingest BigQuery data sources, the Collibra Data Lineage service instances now support JOIN statements with the UNNEST operator.
• Improved error messaging when Tableau site names and/or site IDs are unspecified or invalid in your lineage harvester configuration file.
• When you integrate SSRS-PBRS, you can again view a technical lineage graph from SSRS Report, SSRS Table and Power BI Report asset pages. (tickets #96611, 96686, 96688, 105064)
• When you integrate Power BI, the Collibra Data Lineage service instances no longer generate false parsing errors for the <source ID> file when there are no errors.
• When you integrate Power BI or Azure Data Factory (currently in public preview), the lineage harvester now connects to the Microsoft cloud instance, instead of the login.microsoftonline.com host. (ticket #100367)
• When you run a full-sync of a Snowflake data source, the lineage harvester automatically refreshes the authentication token, to avoid a time-out error. (ticket #100824)
• Fixed the ordering of columns for Power BI technical lineage custom queries.
• When you integrate Tableau:
  • The Collibra Data Lineage service instances now successfully process columns with names that have quotation marks in them.
  • The Collibra Data Lineage service instances now correctly split table and schema names. As a result, assets reflected in the technical lineage now have the expected full names. (ticket #103279)
  • If there are errors related to invalid Tableau sites, the errors are now correctly shown under the Tableau source in the technical lineage Sources tab page.
• The Power BI integration can now connect to Power BI for US government customers.

Data Marketplace

• Tags are now displayed on multiple lines in the asset preview.
• The data basket will now also appear for child assets of Data Set and Report.

Data Governance

• Executing validation rules no longer changes the last modified date of the asset.
• You can once again sort columns in a complex relation. (ticket #96087, #96366)
• When adding a new numeric attribute or updating an existing numeric attribute on the asset page, the save button is disabled when you only type "-" or "+".
• When checking the details of a validation result on the asset page title bar, the date of the previous validation is now localized to your timezone. (ticket #98725)
• When creating an asset via Global create, the values in the domain dropdown are now sorted case-insensitive.
• You can no longer accidentally create duplicates by clicking the save button in the Add Relations dialog multiple times as this button is now disabled after the first click. (ticket #100884)
• When adding relations or attributes on an asset page you will now see the correct validation rule for the page you are on. (ticket #100673)
• You no longer need to complete the ID field when importing new complex relations. (ticket #102334)
• When trying to import an update to an existing asset, the import no longer fails if the asset type is not mapped, when using the new import wizard.
• When trying to import a new asset in an existing community or domain, the import no longer fails if the domain type is not mapped, when using the new import wizard.
• An error message is now properly displayed on the Activities page if an error occurred during export. (ticket #90410)
• You can now click the 10,000+ count in lower right corner of the User or Group table and the full count of the respective table will be returned and paginate as expected. (ticket #103420)
• Protection against customization of assigned characteristics for the Database and Schema asset types has been corrected. (ticket #100881)
• Global create once again loads correctly, regardless of how you navigate there. (ticket #100125)
• Migrating a CMA file with only a restriction change on an assigned relation type of an asset type now properly reflects the restriction change when imported. (ticket #101597, #102124)
• It is no longer possible to assign invalid restrictions when assigning a Relation Type through the REST or Java API. Passing an invalid restriction as argument will result in an Exception.
• You will now only see the global create button if you have permissions that allow you to use it.
• Your input text will no longer be included in the text for any error messages received when getting a status name through REST API. (ticket #96592)
• When adding relations or attributes on an asset page you will no longer see content from the previous page you were on. (ticket #100673)

• You can once again update attributes without an error message that an object has been re-saved after removal. (ticket #97346, #104936, #105231).

Assessments

• When conducting an assessment, if you switch between the pages in the assessment template, the template logic settings are now followed and the expected page is shown.
• A user with the Assessments global role can no longer delete new assessments that are created by another user.
• If the text in a field in an assessment reaches a certain length, the text no longer overlaps the other fields.

Data Privacy

• Previously, in Collibra Protect, if two or more data protection standards masked the same column, you could not view the column data in the data source. This issue is now fixed.
• Time-based review rules that were configured to trigger time-based asset review workflows on an annual basis now work as expected.

Search

• After having accesssed a community, if you access a different community and then search for an asset, the Search field no longer shows the community that you previously accessed but only the current community. (ticket #100166)
• The global Search field no longer shows deleted communities.

Insights Data Access and Usage Analytics

Note Insights Data Access and Usage Analytics are cloud-only features.

• In the CSV files downloaded from the Visitors section of the Usage Analytics dashboard, the values shown in the Required License and Effective License columns now match those shown in the tables on the dashboard.
• On the Usage Analytics dashboard, if the usage period is Last 90 days, the lines in the trend line charts on the Community, Domain Type, or Asset Type card now show the date and the number instead of the label undefined and 0, respectively.
• On the Usage Analytics dashboard, on the expanded Community, Domain Type, or Asset Type card, when the category is All, the horizontal (x) axis on the trend line charts now reflects the selected usage period.

Collibra for Mobile

• Collibra for Mobile no longer runs on rooted (Android) and jailbroken devices (iOS).

Miscellaneous

• When editing the template of the Bar Chart widget, the value in the Grouping by field no longer changes to a different value. (ticket #101159)
• You can now access all the data sets on a bar chart that contains more than 7 data sets. (ticket #97414, 99810)
• When you clicked a link to another dashboard from a custom dashboard with a certain widget configuration, the dashboard was not shown. This issue is now fixed. (ticket #96671)
• On the Dashboards card of the Homepage, if no dashboards are available to you, a button to create a dashboard is now shown.
  Note To create a dashboard, you need one of the following global permissions:

  • Manage shared Views, Dashboard and Search Filter
  • Manage your own Views, Dashboards, Search filters
• The Created on and Last modified on column filters again return only the data that match the dates used in the filters. (ticket #100954)
• If the bottom banner is enabled and you load more asset views on the Browse pane on the Homepage or the Data Marketplace landing page, the bottom banner no longer hides the Show More button. (ticket #101598, 101646, 103993)

Maintenance updates

Collibra 2023.02.1

• When you view a technical lineage, the lineage iFrame again appears correctly in the web page. (ticket #107306, 107317, 107323, 107329)
• Fixed an issue with the language setting on the profile page, which prevented Collibra to start.
• The list of connections on the Edge site overview page and the Connection dropdown menu now list connections alphabetically. (ticket #100924, 103408)
• Fixed an issue in which some Collibra environments failed to upgrade to 2023.02. (ticket #107807)

Collibra 2023.02.2

• You can now edit all attributes, including those that were previously locked by the system.

Collibra 2023.02.3

• Collibra can now connect to Collibra DQ if service mesh is enabled.
• When running Edge capabilities, the data in the Edge log files is now more secured. This applies to the integrations with Amazon S3, Google Cloud Storage, and technical lineage via Edge for all supported data sources.

Edge 2023.02.4

Catalog S3 Synchronization capability update

• You can now save input metadata for your S3 capabilities on Edge. The input metadata is extracted from the data source and can be used to troubleshoot your S3 capabilities on Edge.

Edge 2023.02.5

Collibra Data Lineage capability update

• Collibra Data Lineage now supports proxy servers for technical lineage via Edge. If a proxy setting is defined for the Edge site, it is automatically taken into account. Proxy server authentication is now supported. Custom SSL certificates are not yet supported. See Configure a forward proxy.

Browser Extension 1.1.1

• Auto-matching by UUID for a Power BI dashboard works again. (ticket #107672)