• Announcement
• Enhancements
• Fixes
• Beta features

Release Information

• Release date of Collibra Data Quality & Observability 2025.01: January 27, 2025
• Release notes publication date: December 31, 2024

Announcement

Enhancements

Platform

• When you access Swagger from the Admin Console or the in the upper right corner of any Collibra Data Quality & Observability page, it now opens in a separate tab or window.

Connections

• You can now authenticate Amazon S3 connections using OAuth2 with an Okta principal as a service account. This enhancement simplifies the authentication process and improves security.
• When you add a connection, you can now use commas and equals characters in values in the Values option on the Variables tab. For example, val1=,val2 is now a supported variable value.
  • Additionally, the Driver Properties on the Properties tab now use a JSON array format instead of a comma-separated string. For example, [{"name":"prop1","value":"val1"},{"name":"prop2","value":"val2"}] is now the supported format for properties.

Jobs

• Snowflake Pushdown connections now support source-to-target validation for datasets from the same Snowflake Pushdown connection.
• Snowflake Pushdown connections now support all regex special characters in column names except for square brackets [ ].

Rules

• When you hover over the break percentage (Perc) value on the Rules tab of the Findings page, a new popover message now shows the full value. This update improves visibility and makes it easier to view detailed information.
• Tooltips are now available for the Points and Perc columns on the Rules tab of the Findings page.
• You now see a descriptive error message if your data class rule fails to save because of missing required fields or a duplicate name when using the data class rule creator.
• You no longer need to rerun a DQ Job after renaming a rule for its new name to be reflected across all relevant pages.

APIs

• The POST /v3/datasetdefs endpoint now supports creating outlier and/or pattern records directly by including the outliers or patterns element in the JSON payload (dataset name is required).
  • If an ID is not provided, a new record is created.
  • If an ID is provided and matches an existing record, the existing record will be updated.
  • Alternatively, you can use the existing /v2/upsertPatternOpt and /v2/upsertOutlierOpt endpoints to create records and then reference their IDs when you use the PUT /v3/datasetdefs endpoints for updates.
• The ControllerAdmin API now requires ROLE_ADMIN or ROLE_ADMIN_VIEWER to use the endpoints:
  • /gettotalcost
  • /gettotalbytes
• The ControllerSecurity API now requires ROLE_ADMIN, ROLE_USER_MANAGER, or ROLE_OWL_ROLE_MANAGER to use the endpoints:
  • /getalldbuserdetails
  • /getdbuserwithroles
  • getAllAdGroupsandRoles
• The ControllerUsageStats API now requires ROLE_ADMIN or ROLE_ADMIN_VIEWER to use the endpoints:
  • /getuserprofilecount
  • /getcolumnscount
  • /getusercount
  • /getdatasetcount
  • /getalertcount
  • /getemailcount
  • /getfailingdataset
  • /getpassingdataset
  • /getowlcheckcount
  • /getowlusagelast30days
  • /getowlusagemonthly
  • /getrulecount
  • /gettotalowlchecks
• We deprecated the GET /deleteappconfig. As an alternative, you can use DELETE /deleteappconfig instead.
• We deprecated the GET /deleteadminconfig. As an alternative, you can use DELETE /deleteadminconfig instead.

Fixes

Jobs

• You can again preview columns on SAP HANA tables with periods (.) in their names.
• When you run DQ Jobs from the command line, the command -srcinferschemaoff, which prevents Collibra Data Quality & Observability from inferring the schema in the validate source activity, now works as expected when files are in the source.
• When DB_VIEWS_ON is set to TRUE on the Application Configuration page of the Admin Console, the Include Views option is now also applied when using the Mapping step for a Database connection.

Findings

• The data preview of DQ Jobs created on file data sources that use pipe delimiters (|) now displays correctly on the Profile and Shapes tab of the Findings page.
• DQ Jobs with the validate source layer enabled no longer return an error when you open their Findings pages.
• When you edit the scheduled run time of a DQ Job from the Findings page, the updated scheduled time now saves correctly.
• We removed the sort icon for the Breaking Records and Passing Records columns on the Rules tab of the Findings page, because it was not functional.
• When you click the Run ID link of a dataset on the Assignments Queue, the Findings page now opens to the finding type associated with the Run ID. For example, a Rule finding links to the Rules tab on the Findings page.

Profile

• When you change an Adaptive Rule from automatic to manual scoring on the Profile page, the points value now saves correctly.
• When you manually set the lower and upper bounds of an Adaptive Rule, you can now use decimal values, such as 10.8.
• When you use the search function within the Profile page to search for a partial string of a column name, such as "an" to return results for the columns "exchange" and "company," the correct number of results are now returned. Previously, a maximum of 10 results were returned.
• When a column contains special characters, the column stats section on the Profile page now formats the results correctly, ensuring better readability.

Rules

• When you use the Archive Break Records feature and the link ID column is an alias column, Collibra Data Quality & Observability now properly identifies the link ID column to ensure the rule processes correctly.
• Complex DQ Job query joins containing SAP HANA reserved words in SQL queries are now supported. This ensures that the query compilation processes successfully. The following SAP HANA reserved words are supported in SQL queries:
  • "union", "all", "case", "when", "then", "else", "end", "in", "current_date", "current_timestamp", "current_time", "weekday", "add_days", "add_months", "add_years", "days_between", "to_timestamp", "quarter", "year", "month", "to_char"
• User-defined rules against DQ Jobs on Snowflake Pushdown connections no longer return a syntax error when "@" is present anywhere in the query.
• A rule on a DQ Job with Archive Breaking Records enabled now processes successfully without returning an exception message, even when it does not reference all link IDs in its SELECT statement.
• We temporarily removed the ability to view rule results in fullscreen on the Findings page to address an issue where rule results were missing in fullscreen. The fullscreen view will be re-added in the upcoming Collibra Data Quality & Observability 2025.03 release.
• Int Check rules using string column types no longer flag valid integers as breaking.

  Note When displaying preview break records, if a column in the dataset has an inferred data type that differs from the defined type, the data type check is performed based on the defined type rather than the inferred type.

• Break records are no longer shown for rules with an exception status. Additionally, an error message now provides details about the exception.

Dataset Manager

• When you rename a dataset from the Dataset Manager, you can now use a name that exists in a different tenant but not in your current one.
• Datasets with spaces in their names no longer lose their rules when you rename them from the Dataset Manager.

Reports

• The time zone configurations of certain Collibra Data Quality & Observability environments no longer prevent you from properly viewing the Dataset Findings Report.
• When you use the search filter on the DQ Check Summary Report, you no longer encounter an error when you reach records for Check Type - Rule.

Connections

• The CDATA driver is now supported for Athena Pushdown connections.

Admin Console

• The Security Audit Trail now correctly shows the username of the user who deleted a dataset.

APIs

• The /v3/jobs/<jobId>/breaks/rules endpoint no longer returns a 500 error when using a valid jobId. Instead, it now returns empty files when no results are found for exports without findings.
• The /v3/jobs/{jobID}/findings and /v3/jobs/{dataset}/{runDate}/findings endpoints now return the correct value for the passFail parameter.

Integration

• When you integrate a dataset into Collibra Platform, then add extra columns to it and re-run the integration, the exported JSON file now correctly shows the additional columns.
• The score tile on the Quality tab of Data Quality Rule and Data Quality Job assets now works as expected.
• Dates on the Overview - History table and chart on the Quality tab are now arranged in chronological order.
• The tooltip for Last Run Status on the Summary tab of Data Quality Job assets now contains the correct message.

Beta features

Rules

• When you edit an existing rule from the Rules tab of the Findings page, you can now edit the tolerance value.
• Rule Tolerance is now available for Pushdown connections.
• Rule filter queries applied to rules on Pullup DQ Job now support secondary datasets. Filter queries on @t1 rules are not supported at this time.