Release 2022.09

Enhancements

Rules

  • The Conditions column on the Rules tab now displays SQLG and SQLF rule definitions on hover.

DQ Job

  • The Jobs chart now shows a dotted gray line to represent jobs in Submitted status.
  • The Jobs chart now supports an hourly view option.
  • When you run a Pushdown Job that has a data set that returns 0 rows, an unclear message displays.

Schema

  • From the Config tab in Explorer, a Check Header checkbox under DQ Job is now available for when column names contain special characters. The Check Header checkbox is checked by default.
    • When checked, schema findings do not display when detected.
    • When unchecked, schema findings display when detected.

Behavior

  • Mean values are now rounded on the Findings page.

Explorer

  • SOH delimiters for files are now supported.
  • The Only checkbox on all Build Layer tabs is now removed.
  • The Profile activity is now always enabled and no longer has an on/off switch.

Alerts

  • Only one email per alert is now sent when alerts are set up for a scheduled job.
  • You can now check the logs to see when an alert does not send in order to resend the email.

Scheduler

  • The findings page now displays a green indicator next to the Schedule icon when you schedule a job to run automatically. If Scheduler is inactive, a red indicator displays.

API

  • The v2/gethoot API now properly returns rule dimension information for data sets. (ticket #89973)

Connections

  • The Databricks connection template has changed, due to an upgrade of the driver. Any existing connection that uses the old driver must be updated. Refer to the new template. (ticket #19950)
  • The drivers for Athena, BigQuery, MongoDB, GCS, Hive/Impala were also upgraded but no connection change is required.

Spark

  • The 2026.03 release uses Spark 3.2.2.

Note We recommend using Spark 3.x for standalone installs/upgrades.

Fixes

Explorer

  • Fixed an issue that prevented the Job Estimator from properly displaying row estimates when the run date was modified during a new job run. (ticket #90860)
  • Fixed an issue that prevented DQ jobs created using NFS connection types from displaying under the Remote File Connections dropdown. (ticket #92479)
  • Fixed an issue that caused the file type parser to throw an error message when the default comma delimiter was not detected. The parser now detects a file's delimiter and updates the delimiter type in the UI automatically. (ticket #89489, 92480)

Files

  • The error message for Failed Merging Schema now has extra logging to clarify the cause of failed schema merges for both Livy sessions and non-Livy paths. (ticket #92694)

Security

  • Fixed an issue with the v2/getcatalogtableshasrulesfromcxn API that triggered a 403 status code when Dataset Security was enabled. (ticket #93298, 94258)

Agent

  • Fixed an issue that caused the Agent Check to no longer attempt check-ins to the metastore on K8s deployments, which resulted in red (unhealthy) status. (ticket #92055, 92963)
  • Fixed an issue that prevented concurrent users from properly running Livy sessions. (ticket #92963, 90432)

Known Limitations

Rules

  • The Rule Builder page becomes unusable if the user creates, validates, saves a new rule and then re-edits.

    • The workaround for this limitation is to do a full page refresh.

  • When a user attempts to validate a rule that contains a stat, an exception error is returned.

Security

  • The Assignments Queue feature is only available for local users. Support for externally connected users, such as SAML and AD connector, is not currently available.

Alerts

  • When alert recipient email addresses are separated by semicolons ;, alerts emails are not sent to the intended recipients.
    • A workaround for this limitation is to separate alert recipient email addresses with commas , instead of semicolons.

Snowflake Pushdown

  • When a Job is run, which has a data set that returns 0 rows, an unclear message displays.

  • When a native rule is created that contains an embedded stat, its calculated value will not display on the Job results page.

  • Data Set security is not supported.

  • Disabling autometrics will not take effect, therefore, all autometrics are executed.

  • Creating a DQ job using only "SQL Query" workflow doesn't allow you to set the rundate value.

DQ Security Metrics

DQ security vulnerabilities over 5 months

Critical security vulnerabilities over 5 months