• Announcement
• New and improved
• Fixes

Release Information

• Expected release date of Data Quality & Observability Classic 2025.07: August 4, 2025
• Release notes publication date: July 8, 2025

Announcement

New and improved

Platform

• You can now use the global security setting "Require DATASET_RULES for rule template create/edit access" and assign users the "ROLE_TEMPLATE_RULES" role to restrict updates to template rules. (idea #CDQ-I-169)
  • When “Require DATASET_RULES for rule template create/edit access” is enabled, users without the "ROLE_ADMIN" or "ROLE_TEMPLATE_RULES" roles can view the list of template rules but cannot add, edit, or delete them.
  • If “Require DATASET_RULES for rule template create/edit access” is disabled, or if it is enabled and the user has "ROLE_ADMIN" or "ROLE_TEMPLATE_RULES," they can view, add, edit, and delete rule templates.
• You can now use SAML SSO to sign in from the Tenant Manager page. Additionally, the AD Security Settings page from the Admin Console is now available in the Tenant Manager, allowing you to map user roles to external groups.
• The application now has improved security when parsing SQL queries.
• When creating a job, the “Visualize” function in Explorer is now inaccessible if the global "Require DATA_PREVIEW role to see source data" security setting is enabled and the current user doesn't have the "ROLE_DATA_PREVIEW" permission.

Jobs

• You can now scan for duplicate records in Oracle Pushdown jobs.
• When you transform a column from Explorer, it is now available in the “Target” drop-down list during the Mapping step.
• When a job fails, the metadata bar remains visible on the Findings page.

Rules

• When you enable the "RULE_FILTER" setting in the Application Configuration Settings page, create a rule with a filter, and then disable the "RULE_FILTER" setting before rerunning the job, the results on the Findings page still reflect the rule using the filter.
• The “Edit Rule” dialog box now includes Description and Purpose fields. These fields allow you to add details about Collibra Platform assets associated with your rule and identify which assets should relate to your Data Quality Rule asset upon integration. (idea #CDQ-I-219)
• The Rule Workbench now supports the substitution of both string and numeric data types for $min, $average, and $max stat rule values.
• The Results Preview on the Rule Workbench now shows columns in the same order as they are listed in the rule query. For example, the query select SYMBOL, VOLUME, LOW, HIGH, CLOSE, OPEN from PUBLIC.NYSE where SYMBOL IS NOT NULL LIMIT 3 displays the columns in the following order in the Results Preview:
  • SYMBOL
  • VOLUME
  • LOW
  • HIGH
  • CLOSE
  • OPEN

Alerts

• You can now create alerts for adaptive rules in the “Breaking” state. This eliminates the need to manually search for breaking records in adaptive rules, reducing the risk of missing them. (idea #DCC-I-2713, CDQ-I-199, CDQ-I-105)
• Condition alerts now include a "Condition Value" field to provide more context about the reason for the alert. For example, the Condition Value for a row count check might be rc = 0, while for an outlier score, it could be outlierscore > 25. (idea #CDQ-I-317)
• When you create a new alert with a name already in use by another alert on the same dataset, the existing alert is no longer automatically replaced. Instead, a confirmation dialog box now appears, allowing you to choose whether to overwrite the existing alert.

Findings

• Float values with commas now retain up to 4 decimal places and remove trailing zeroes from the Findings page. For example, 3722.25123455677800000000000 is now shown as 3722.2512. When you hover your pointer over the value, the full value is shown in a tooltip.

Profile

• The correlation heatmap in the Correlation tab of the Profile page now shows a maximum of 10 columns, regardless of the number of columns returned in the API response.
• The “Quality” column on the Profile tab is now renamed to “Completeness/Consistency.”

Collibra Platform Integration

• When you rename a job from Data Quality & Observability Classic that is integrated with Collibra Platform, the corresponding domains in Collibra Platform are now renamed to match the new job name.
• When you add, update, or remove meta tags from a job in Data Quality & Observability Classic with an active integration to Collibra Platform, the import JSON and the corresponding Data Quality Job in Collibra Platform are now updated. The updated JSON can include up to 4 meta tags.

APIs

• The GET /v3/datasetdef API call now returns meta tags in the array response in the same order they are set in the Dataset Manager. For example, if the first meta tag input field in the Dataset Manager contains "DATA," the second and third fields are empty, and the fourth contains "QUALITY," the array response will return:
Copy

    “metaTags”:  [
      “DATA”, 
     null,
      null,
      “QUALITY”,
],

Admin Console

• You can now control the maximum allowed temporary file size using the new "maxfileuploadsize" admin limit. The default value is 15 MB.
• When you delete a role, any datasets mapped to it are now removed. Previously, the relations between the role, datasets, and users were still shown in the UI after deleting the role.
• Orphaned dataset-run_id pairs are now removed from the “dataset_scan” table when you use a time-based data retention purge.

Fixes

Platform

• You no longer encounter unexpected errors with SAML/SSO, rules, and Pushdown after upgrading to some Spark standalone self-hosted instances of Data Quality & Observability Classic.
• Group-based role assignments for Azure SSO now work correctly after upgrading Data Quality & Observability Classic. Users no longer receive only the default "ROLE_PUBLIC," ensuring proper access permissions.
• Users with the "ROLE_PUBLIC" and "ROLE_DATA_PREVIEW" roles can no longer create and save rules, ensuring role restrictions are enforced as intended.

Connections

• You can now run jobs on Teradata connections without requiring the DBC.RoleMembrs.RoleName permission.

Jobs

• Alias names now appear for date fields during row selection for a new job.
• You no longer see the "Pushdown Count" option on the Mapping Configuration page. This was removed for BigQuery because it is not supported by Google.

Rules

• Rule findings using a rule filter for Pushdown jobs now show the correct subset of rows applied for the rule, instead of the total rows at the dataset level.

Findings

• Boolean attributes now show their values correctly in the Result Preview on the Rule Workbench screen.
• The handling of “shape/enabled” and “deprecated profile/shape” JSON flags in dataset definitions and job runs is now consistent.
• When a column contains multiple shape findings, the data preview is now shown as expected when you expand a shape finding.

Profile

• Profiling activities now correctly support profile pushdown on BigQuery, ensuring accurate and efficient data profiling.

Reports

• Encrypted columns now appear correctly on the Dataset Findings report.

APIs

• The /v3/rules/{dataset}/{ruleName}/{runId}/breaks API now returns a “204 No Content” HTTP status code during pullup jobs when a rule has no break records linked to a specific LinkID.

• Announcement
• New and improved
• Fixes

Release Information

• Release date of Data Quality & Observability Classic 2025.06: June 30, 2025
• Release notes publication date: June 3, 2025

Announcement

New and improved

Platform

• This application, which was previously known as Collibra Data Quality & Observability, is now called Data Quality & Observability Classic. This change reflects the introduction of the new Data Quality & Observability as part of the unified Collibra Platform. For more information about Data Quality & Observability, go to About Data Quality & Observability.

Jobs

• When archiving break records for Pushdown jobs, link IDs are now optional. This change provides greater flexibility in storing and accessing break records and offers a more scalable solution for remediating records outside of Data Quality & Observability Classic. (idea # CDQ-I-155, DCC-I-2566, DCC-I-5726, CDQ-I-90, CDQ-I-137)
  • If Data Quality & Observability Classic does not have permission to write to tables in your source database, Pushdown jobs with or without link IDs selected require you to run an ALTER statement on the COLLIBRA_DQ_RULES table to add the new results column for JSON data. Data Quality & Observability Classic will attempt this alteration automatically, which will cause the rule to fail because it will attempt to write to a column that doesn’t exist in your source database. For more information, go to ALTER statement requirements for optional link IDs.

  Important If you don’t use a link ID, the results column in the COLLIBRA_DQ_RULES table will become exponentially larger, leading to unintended storage and compute costs in your source database. To minimize these costs, you can use a link ID or move these results out of your source database and into a remote file storage system, such as Amazon S3.

• Data Quality & Observability Classic now checks that a link ID is set in the settings dialog on Explorer when creating a DQ job, to ensure the archive break records feature works when enabled. If one or more link ID columns are selected, a checkbox and drop-down list appear next to the Archive Breaking Records option in the settings dialog. If no link ID columns are selected, the checkbox and drop-down list are unavailable, and a tooltip appears when you hover over the option. (idea #DCC-I-2746)
• Oracle Pushdown is now generally available.
• Oracle Pushdown connections do not support the following data types:
  • LONG RAW
  • NCHAR
  • NVARCHAR2
  • ROWID
  • BYTES
• Trino Pushdown jobs now support source-to-target analysis for datasets within the same Trino Pushdown connection.
• SAP HANA, Snowflake, SQL Server, and Trino Pushdown connections do not support the VARBINARY data type.
• The valsrclimitui admin limit, which controls the number of Validate Source records shown in the UI, now works as expected. For example, if valsrclimitui is set to 3, the UI shows 3 records from the Column Schema and 3 records from the Cell Panels.

Note The fpglimitui, histlimitui, and categoricallimitui admin limits are not working as expected today. Further, any admin limit ending “ui”, such as valsrclimitui, is intended only for updates made via the Admin Limits page of the Admin Console. Changes made to these admin limits via the command line will not be reflected.

Rules

• Rules that use $colNm in the query, such as data class, template, and data type checks, no longer perform automatic rule validation.
• The Rule Workbench now has a warning message when the preview limit exceeds the recommended maximum of 500 records.
• The maximum length of the name of a rule is now 250 characters.
• The Preview Breaks modal now indicates the name of the rule.
• The Low, Medium, and High buttons now correctly reflect your selection when you reopen the Rule Details dialog box after automatically changing a rule's points. (idea #CDQ-I-150)
  • Additionally, if you manually update the score, the button automatically adjusts to match the corresponding scoring range:
    • Low: 1-4 points
    • Medium: 5-24
    • High: 25-100

Admin Console

• Business unit names on the Business Unit page in the Admin Console cannot contain the special characters / (forward slash), `(backtick), | (pipe), or $ (dollar sign). If you use any of these special characters, an error message appears, and the Edit Business Unit dialog box remains open until the special characters are removed.
• Data retention policies now apply to the following metastore tables:
  • alert_output
  • behavior
  • dataset_activity
  • dataset_schema_source
  • hint
  • validate_source
• The Title and Story fields of the User Story section of the User Profile page cannot contain the following special characters:
  • ‘ (single quote)
  • “ (double quote)
  • -- (double dash)
  • < (less than)
  • > (greater than)
  • & (ampersand)
  • / (forward slash)
  • ` (backtick)
  • | (pipe)
  • $ (dollar sign)
  If you use any of these special characters, an error message appears, and the Edit User Story dialog box remains open until the special characters are removed.

Fixes

Rules

• You can now apply rule filter queries to rules created using rule templates on Pushdown datasets.
• Duplicated data quality rules no longer appear after you import data assets using CMA.
• Rules now run without errors when you use a single link ID to archive break records with SQL Server Pushdown.

Profile

• Divide-by-zero errors and issues with empty histograms are now resolved. Calculations and visualizations now work as expected without interruptions.

Findings

• You can now export rule break records with accurate counts. The total number of rule break records for a rule now matches the number in the Breaking Records column of the exported file.

Integration

• When you rename a rule in Data Quality & Observability Classic that is part of a dataset integrated with Collibra Platform, additional Data Quality Rule assets are no longer created during the next integration. The existing Data Quality Rule asset is also no longer unintentionally set to a “suppressed” state during the next integration, and its name is updated to match the name in Data Quality & Observability Classic.

• Announcement
• New and improved
• Fixes
• Patch release

Release Information

• Release date of Data Quality & Observability Classic 2025.05.1: June 16, 2025
• Release date of Data Quality & Observability Classic 2025.05: June 2, 2025
• Release notes publication date: May 6, 2025

Announcement

New and improved

Warning We identified a known security vulnerability (CVE-2025-48734) in version 2025.05. We addressed this issue in the 2025.05.1 patch.

Platform

• You can now authenticate users via Microsoft Azure Active Directory B2C when signing into Data Quality & Observability Classic.
• Only users with the ROLE_ADMIN role can now see the username and connection string of JDBC connections on the Connections page and in API responses, improving connection security.
• Metastore credentials are no longer stored in plain text when you create a DQ Job via a notebook. This improvement increases the security of your credentials.
• We enhanced the security of Data Category validation.
• We improved the security of our application.

Jobs

• Trino Pushdown jobs now support validate source.
• The Run Job buttons now have two enhancements:
  • The Run Job with Date button on the Jobs tab of the Findings page is now labeled Select Run Date.
  • The Run Job button on the metadata bar now includes a helpful tooltip on hover.
• SAP HANA connections now support the operators =>, ||, and && for scoped queries, the reserved word ‘LIKE_REGEXTR’, and the function REPLACE in scoped queries.
• The assignments queue is now disabled for the validate source activity on new installations of Data Quality & Observability Classic. To enable it, an admin can set valsrcdisableaq to "false" on the Admin Limits page.

Warning When you set valsrcdisableaq to "true," the ability to assign findings is disabled for all finding types, such as rules, outliers, and so on. This issue is resolved in version 2025.05.1.

Rules

• When Archive Break Records is disabled, you can now use NULL link IDs with Athena, BigQuery, Hive, Redshift, and SQL Server connections.
• When Archive Break Records is enabled, and a rule doesn't specify a link ID column but the job does, you can no longer archive break records in these scenarios, as Data Quality & Observability Classic no longer appends the link ID to the rule. When a rule does not contain the selected link ID, the break records for that rule will not be archived, and an exception will occur. The job will execute successfully, but the rule will not produce any archived break records, and the exception message will explain that the link ID is not part of the rule.
• Trino Pushdown jobs now support profiling and custom rules on columns of the ARRAY data type.
• The Rule Details dialog on the Rule Workbench page now has an optional Purpose field. This allows you to add details about Collibra Platform assets associated with your rule and see which assets should relate to your Data Quality Rule asset upon integration.

Alerts

• You can now set up Data Quality & Observability Classic to send data quality alerts to one or more webhooks, eliminating the dependency on SMTP for email alerts.
• You can now rename alerts on the Dataset Alerts page. When you rename an alert, the updated name is reflected in the alert_nm column of the alert_cond Metastore table. The updated name applies only to alerts generated in future job runs and does not affect historical data.
• In addition to the existing rule name condition variable for percentages, you can now include a rule name with a value identifier, such as test_rule.score > 0, to create condition alerts based on rule scores.

Profile

• When you retrain a breaking adaptive rule, negative values are now supported for lower and upper bounds, min, max, and mean.

Findings

• To enhance usability, we moved the data preview from the Rules results table into a dedicated modal, making it easier to navigate and view data.
  • The Actions button is now always visible on the right side of the Rules tab, and includes the following options:
    • Archived Breaks, previously called Rule Breaks, retains the same content and functionality.
    • Preview Breaks, previously embedded under the in the first column, shows all breaking records for a given rule. This option provides the same information that was previously available in the Rules tab and still requires the ROLE_DATASET_ACCESS role. Additionally, the Rule Break Preview modal reflects the preview limit of the rule.
  • Tooltips were added to various components on the Rules tab and Rule Break Preview modal.

Integration

• You can now use the Map to primary column only switch in the Connections step of the integration setup wizard. This allows you to map a rule to only the Column asset in Collibra Platform that corresponds to the primary column in Data Quality & Observability Classic.
• When you rename a custom rule in Data Quality & Observability Classic with an active integration with Collibra Platform, a new asset is no longer created in Collibra Platform. Instead, renamed rules reuse the Collibra Platform asset IDs associated with the previous rule name when their metadata is integrated.

APIs

• The /v3/datasetDefs/template API endpoint now returns the nested key-value pairs for shape settings and job schedule in the datasetDefs JSON object.

Admin Console

• You can now use the previewlimit setting on the Admin Limits page to define the maximum number of preview results shown in the Dataset Overview and Rule Workbench. The default value is 250.

Warning Large previewlimit values can negatively impact performance.

Fixes

Platform

• All datasets are now available when you run a dataset rule fetch with dataset security disabled for the admin user account.
• FIPS-enabled standalone installations of Data Quality & Observability Classic now support SAML authentication.

Jobs

• When you remove a table from a manually triggered or scheduled job, you now receive a more descriptive error stating that the table does not exist instead of a generic syntax error.
• You can now use Amazon S3 as a secondary dataset name for a rule query.
• You can now rerun a job for a selected date using the Run DQ Job button on the metadata bar of the Findings page.
• The command line now retains the created query, including the ${rd} parameter, when you run a job using the Run DQ Job button on the metadata bar.
• The controller response for queries to the alert_cond table in the Metastore now maps internal objects correctly.
• The Profile page now correctly shows the top 5 TopN and BottomN shape results for Pullup jobs.
• When you add a transform to a job with histogram enabled or set to auto, the job processes as expected, and aliased columns display correctly on the Profile page.

Rules

• You can again save rules on Pushdown jobs that exceed ⅓ of a buffer page.
• Pushdown rules that use stat variables now run with the correct information from the current job, rather than using data from previous job runs.
• The DOUBLECHECK rule on PostgreSQL and Redshift Pullup jobs now flags rows with negative double values as valid.
• The Result Preview in the Rule Workbench no longer produces an error when you use out-of-the-box templates.

Findings

• Pushdown jobs run on Snowflake connections now show as failed on the Findings page if a password retrieval issue occurs.

Dataset Manager

• When you apply an option from the Actions drop-down list, such as delete a dataset, the correct dataset now has the action applied to it.

Admin Console

• When you select a submenu option from the Admin Console menu, the submenu section now remains open.

Patch release

2025.05.1

• Columns from Db2, Oracle, SQL Server, Sybase, and Teradata connections now load correctly in Explorer for users without the "ROLE_ADMIN" or "ROLE_CONNECTION_MANAGER" roles.
• The ability to assign findings for all finding types, such as rules, outliers, and so on, is no longer disabled when the "valsrcdisableaq" limit is set to "true" on the Admin Limits page.
• We resolved a security vulnerability related to CVE-2025-48734.

• Announcement
• New and improved
• Fixes

Release Information

• Release date of Data Quality & Observability Classic 2025.04: April 28, 2025
• Release notes publication date: April 2, 2025

Announcement

New and improved

Platform

• The Platform Path of the SQL Assistant for Data Quality feature now uses the Gemini AI model version gemini-1.5-pro-002.

Important To continue using the Platform Path of the SQL Assistant for Data Quality feature, you must upgrade to Data Quality & Observability Classic 2025.04.

• FIPS-enabled standalone installations of Data Quality & Observability Classic now support SAML authentication.
• The DQ agent now automatically restores itself when the Metastore reconnects after a temporary disconnection due to maintenance or a glitch.

Jobs

• We are pleased to announce that Oracle Pushdown is now available for preview testing.
• When you add a timeslice to a timeUUID data type column in a Cassandra dataset, an unsupported data type error message now appears.
• Dremio, Snowflake, and Trino Pullup jobs now support common table expressions (CTE) for parallel JDBC processing.
• You can now archive the break records of shapes from Trino Pushdown jobs.
• Link IDs for exact match duplicates are no longer displayed on the findings page.
• We improved the insertion procedure of validate source findings into the Metastore.

Rules

• When you add or edit a rule with an active status, SQL syntax validation now runs automatically when you save it. If the rule passes validation, it saves as expected. If validation fails, a dialog box appears, asking whether you want to continue saving with errors. Rules with an inactive status save without validation checks.
• When a rule condition exceeds 2 lines, only the first 2 lines are shown in the Condition column of the Rules tab on the Findings page. You can click "more..." or hover over the cell to show the full condition.

Profile

• The Profile page now correctly shows the top 5 TopN and BottomN shape results for Pullup jobs.
• When there are only two unique string values, the histogram on the Profile page now shows them correctly.

Findings

• To improve page alignment across the application, the Findings page now has a page title.

Alerts

• Rule score-based alerts now respect rule tolerance settings. Alerts are suppressed when the rule break percentage falls within the tolerance threshold.

Dataset Manager

• If you don't have the required permissions to perform certain tasks in the Dataset Manager, such as updating a business unit, an error message now appears when you attempt the action.

Scorecards

• You can now delete scorecards with names that contain trailing empty spaces, such as "scorecard " and "test scorecard ".

Integration

• You can now search with trailing spaces when looking up a community for tenant mapping in the Data Quality & Observability Classic and Collibra Platform integration configuration.

APIs

• You now need the ROLE_ADMIN or ROLE_ADMIN_VIEWER role to access the /v2/getdatasetusagecount API endpoint.

Fixes

Platform

• We have improved the security of our application.
• The data retention process now works as expected for all tenants in multi-tenant instances.
• Permissions errors for SAML users without the ROLE_PUBLIC role are now resolved. You no longer need to assign ROLE_PUBLIC to users who already have other valid roles.

Jobs

• Native rules, secondary dataset rules, and scheduled jobs on Databricks Pullup connections authenticated via EntraID Service Principal now run as expected.
• Pushdown jobs scheduled to run concurrently now process the correlation activity correctly.
• Pushdown jobs with queries that contain new line characters now process correctly, and the primary table from the query is shown in the Dataset Manager.

Findings

• The confidence calculations for numerical outliers in Pullup jobs have been updated for negative values. Positive value confidence calculations and Pushdown calculations remain unchanged.

Alerts

• Conditional alerts now work as expected when based on rules with names that start with numbers.

Integration

• The default setting for the integration schema, table, and column recalculation service (DQ_DGC_MAPPING_STATS_SCHEDULER_ENABLED) is now false, reducing unnecessary database activity. You can enable the service or call it through the API when needed.

Important From versions 2024.11 to 2025.03 of Data Quality & Observability Classic, if you don't want queries from the recalculation of mapped and unmapped stats of total entities to run, set the DQ_DGC_MAPPING_STATS_SCHEDULER_ENABLED to false in the owl-env.sh or Web ConfigMap. You can keep DQ_INTEGRATION_SCHEDULER_ENABLED set to true.
For more information, go to the Collibra Support center.

• The Quality tab for database assets is now supported in out-of-the-box aggregation path configurations.
• The auto map feature now correctly maps schemas that contain only views.
• Dimension configuration for integration mapping no longer shows duplicate Data Quality & Observability Classic dimensions from the dq_dimension table.

API

• When you copy a rule with an incorrect or non-existent dataset name using the v3/rules/copy API, an error message now specifies the invalid dataset or rule reference. This prevents invalid references in the new dataset.

• Announcement
• Enhancements
• Fixes

Release Information

• Release date of Data Quality & Observability Classic 2025.03: March 31, 2025
• Release notes publication date: March 4, 2025

Announcement

Enhancements

Platform

• On April 9, 2025, Google will deprecate the Vertex text-bison AI model, which SQL Assistant for Data Quality uses for the "preview path" option. To continue using SQL Assistant for Data Quality, you must switch to the "platform path," which requires an integration with Collibra Platform. For more information about how to configure the platform path, go to About SQL Assistant for Data Quality.
• We removed support for Kafka streaming.

Connections

• Private endpoints are now supported for Azure Data Lake Storage (Gen2) (ABFSS) key- and service principal-based authentication and Azure Blob Storage (WASBS) key-based authentication using the cloud.endpoint=<endpoint> driver property. To do this, add cloud.endpoint=<endpoint> to the Driver Properties field on the Properties tab of an ABFSS or WASBS connection template. For example, cloud.endpoint=microsoftonline.us.
• Trino now supports parallel processing in Pullup mode. To enable this enhancement, the Trino driver has been upgraded to version 1.0.50.

Jobs

• The Jobs tab on the Findings page now includes two button options:
  •  Run Job from CMD/JSON allows you to run a job with updates made in the job's command line or JSON. This option is Run Job from JSON in Pushdown mode since the command line option is not available.
  •  Run Job with Date allows you to select a specific run date.

  Note The Run DQ Job button on the metadata bar retains its functionality, allowing you to rerun a job for the selected date.

• The order of link IDs in the rule_breaks and opt_owl Metastore tables for Pushdown jobs is now aligned.
• The options to archive the break records of associated monitors in the Explorer settings dialog box of a Pushdown job are now disabled when the Archive Break Records option is disabled at the connection-level.
• We updated the logic of the maximum global job count to ensure it only increases, rather than fluctuating based on the maximum count of the last run job's tenant. This change allows tenants with lower maximum job counts to potentially run more total jobs while still enforcing the maximum connections for individual jobs. Over time, the global job count will align with the highest limit among all tenants.
• You can now archive the break records of shapes from SAP HANA and Trino Pushdown jobs.
• You can now use the new "behaviorShiftCheck" element in the JSON payload of jobs on Pullup connections. This allows you to enable or disable the shift metric results of Pullup jobs, helping you avoid misleading mixed data type results in string columns. By default, the "behaviorShiftCheck" element is enabled (set to true). To disable it, use the following configuration: “behaviorShiftCheck”: false.

Rules

• You can now set the MANDATORY_PRIMARY_RULE_COLUMN setting to TRUE from the Application Configuration Settings page of the Admin Console to require users to select a primary column when creating a rule. This requirement is enforced when a user creates a new rule or saves an existing rule for the first time after the setting is enabled. Existing rules are not affected automatically.
• The names of Template rules can no longer include spaces.
• CSV rule export files now include a Filter Query column when a rule filter is defined. If no filter is used, the column remains empty. The Condition column has been renamed to Rule Query to better distinguish between rule and filter queries. Additionally, the Passing Records column now shows the correct values.
• You can now apply custom dimensions added to the dq_dimension table in the metastore to rules from the Rule Details dialog box on the Rule Workbench. These custom dimensions are also included in the Column Dimension Report.
• Livy caching now uses a combination of username and connection type instead of just the username. This improvement allows you to seamlessly switch between connections to access features such as retrieving the run results previews for rules or creating new jobs for remote file connections, without manually terminating sessions.

Note Manually terminating a Livy session will still end all sessions associated with that user.

Findings

• You can now work with the Findings page in full-screen view.

Scorecards

• You now receive a helpful error message in the following scenarios:
  • Create a scorecard without including any datasets.
  • Update a scorecard and remove all its datasets.
  • Add or update a scorecard page with a name that already exists.

Fixes

Platform

• We have improved the security of our application.
• We mitigated the risk of SQL injection vulnerabilities in our application.
• Helm Charts now include the external JWT properties required to configure an externally managed JWT.

Jobs

• Google BigQuery jobs no longer fail during concurrent runs.
• When you add a -conf setting to the agent configuration of an existing job and rerun it, the command line no longer includes duplicate -conf parameters.
• When you expand a Snowflake connection in Explorer, the schema is now passed as a parameter in the query. This ensures the Generate Report function loads correctly.
• Record change detection now works as expected with Databricks Pushdown datasets.
• When you select a Parquet file in the Job Creator workflow, the Formatted view tab now shows the file’s formatted data.
• When you edit a Pullup job from the Command Line, JSON, or Query tab, the changes initially appear only on the tab where you made the edits. After you rerun the job, the changes are reflected across all three tabs.
• The Dataset Overview now performs additional checks to validate queries that don’t include a SELECT statement.

Rules

• When you update the adaptive level or pass value option in the Change Detection dialog box of an adaptive rule, you must now retrain it by clicking Retrain on the Behaviors tab of the Findings page.
• @t1 rules on file-based datasets with a row filter now return only the rows included in the filter.
• @t1 rules on Databricks datasets no longer return a NullPointerException error.
• When you run Rule Discovery on a dataset with the “Money” Data Class in the Data Category, the job no longer returns a syntax error when it runs.

Findings

• We updated the time zone library. As a result, some time zone options, such as "US/Eastern," have been updated to their new format. Scheduled jobs are fully compatible with the corresponding time zones in the new library. If you need to adjust a time zone, you must use the updated format. For example, "US/Eastern" is now "America/New_York."
• Labels under the data quality score meter are now highlighted correctly according to the selected time zone of the dataset.

Alerts

• You no longer receive erroneous job failure alerts for successful runs. Additional checks now help determine whether a job failed, improving the accuracy of job status notifications.
• You can now consistently select or deselect the Add Rule Details option in the Condition Alert dialog box.

Reports

• The link to the Dataset Findings documentation topic on the Dataset Findings report now works as expected.

Connections

• Editing an existing remote file job no longer results in an error.
• Teradata connections now function properly without requiring you to manually add the STRICT_NAMES driver property.

APIs

• When you run a job using the /v3/jobs/run API that was previously exported and imported with /v3/datasetDefs, the Shape settings from the original job now persist in the new job.
• Bearer tokens generated in one environment using the /v3/auth/signin endpoint (for local users) or the /v3/auth/oauth/signin endpoint (for OAuth users) are now restricted to that specific Data Quality & Observability Classic environment and cannot be used across other environments.
• We improved the security of our API endpoints.

Integration

• You can now use the automapping option to map schemas, tables, and columns when setting up an integration between Data Quality & Observability Classic and Collibra Platform in single-tenant Data Quality & Observability Classic environments.
• The Quality tab now correctly shows the data quality score when the head asset of the starting relation type in the aggregation path is a generic asset or when the starting relation type is based on the co-role instead of the role of the relation type.
• Parentheses in column names are no longer replaced with double quotes when mapped to Collibra Platform assets. This change allows automatic relations to be created between Data Quality Rule and Column assets in Collibra Platform.