Release Notes

Important 

Disclaimer - Failure to upgrade to the most recent release of the Collibra Service may adversely impact the security, reliability, availability, integrity, performance or support (including Collibra’s ability to meet its service levels) of the Service. Collibra hereby disclaims all liability, express or implied, for any reduction in the security, reliability, availability, integrity, performance or support of the Service to the extent the foregoing would have been avoided had you allowed Collibra to implement the most current release of the Service when scheduled by Collibra. Further, to the extent your failure to upgrade the Service impacts the security, reliability, availability, integrity or performance of the Service for other customers or users of the Service, Collibra may suspend your access to the Service until you have upgraded to the most recent release.

Release 2023.09

Release Information

  • Expected release date of Collibra Data Quality & Observability 2023.09: October 1, 2023
  • Publication dates:
    • Release notes: September 24, 2023
    • Documentation Center: September 29, 2023

Highlights

    Pushdown
    We're delighted to announce that Pushdown processing for Amazon Athena and Redshift is now available as public betas! Pushdown is an alternative compute method for running DQ jobs, where Collibra DQ submits all of the job's processing directly to a SQL data warehouse, such as Athena and Redshift. When all of your data resides in Athena or Redshift, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirements of a DQ job.
    Job Estimator
    Collibra DQ utilizes Spark's ability to break large datasets into smaller, more manageable segments called partitions. When you run large Pullup jobs, you can now leverage the job estimator to automatically calculate and update the number of partition columns required to optimally run and write rules against them. Previously, the only way to know when a job required the scaling of resources was when it failed.

Enhancements

Capabilities

  • When running rules that reference secondary datasets, you now have the option to use serial rule processing to reduce operational costs.
    • Set -serialrule to true to leverage the Spark cache for the secondary dataset.
  • When authenticating your connection to CockroachDB with a PostgreSQL driver, you can now leverage Kerberos TGT without errors.
  • When creating a DQ job to run against a remote file data source, you can now select BEL as a delimiter.
  • When adding a name to a rule on the Rule Workbench, a helpful message displays if you use an invalid special character.
    • Rule names can only contain alphanumerical characters, underscores, and hyphens.
  • When reviewing Rules findings, the default number of rows available to preview is now 6. Previously, the Rules tab only displayed 5 preview rows.
  • When creating a Pullup job from Explorer, the Mapping step now automatically maps source columns to target columns.
  • When using Pulse View, the Lookback column is now called Last X Days.
  • We've updated the connection icons on the Explorer, Pulse View, and Admin Connections pages.
    • When you add a new connection from the Admin Connections page, the icon will also update accordingly.
  • When monitoring the Jobs page with React on, you can now right-click to open a dataset in a new tab.
  • When assigning or validating a finding to an external user whose first name, last name, and external user ID cannot be found or do not exist, you can now set a backup display name in the ConfigMap to ensure you can still validate or assign that finding to the external user.
    • Set SAML_USE_EXTERNAL_USER_ID_FOR_DISPLAY to true.

Platform

  • When deleting a user, the user is now removed from both the user and user_profile metastore tables.
  • When loading a large remote file into Explorer, a progress bar now tracks its loading status.

DQ Integration

  • When using the configuration wizard in Collibra DQ to set up an integration, your Collibra Data Intelligence Cloud credentials are now encrypted in the metastore to ensure that your information is always secure.

DQ Cloud

  • We've introduced a new endpoint to retrieve aggregated WAL (write-ahead logs) stats.
  • When deploying a new Edge site, the TenantAlignmentService no longer stops checking for new tenants in DQ Cloud after 100 attempts.

Pushdown

  • We’ve improved the memory usage to prevent large quantities of rule break records from causing out-of-memory errors.
  • When running a Pushdown job, the entire allocated connection pool is now used to extract the maximum allowed parallelism to allow profiling to run in parallel with other layers and reduce the latency of the job.
    • Only the required number of connection threads are used for an activity.
  • When creating rules to run against Pushdown datasets, you can now use cross-join queries.
  • We've added a Pendo tracking event to track the number of Pushdown jobs and columns in an environment.

Fixes

Capabilities

  • When editing DQ jobs for KDB (PostgreSQL) connections, you can now successfully execute a query with a large number of records. (ticket #113493, #116740)
  • When creating a BigQuery job, you can now create a dataset for a destination table without throwing an error. (ticket #118534, #122761)
  • When archiving break records from Pullup jobs, you can again write break records to S3 storage buckets. Previously, an invalid rule error returned which stated "Exception while inserting break records into S3: No FileSystem for scheme s3". (ticket #121509)
  • When you open the Oversized Job Report, you can again see the reports without any errors. (ticket #121752)

Platform

  • When reviewing the configuration after running a Validate Source job, you no longer receive a validation error due to lost database, schema, table, field, and query information. (ticket #113977)
  • Oracle dataset host strings no longer parse incorrectly. Previously, Oracle dataset host strings were parsed as "jdbc" instead of displaying the correct host string. To see the updated and correct host string for Oracle datasets, rerun your jobs manually via the scheduler or API. (ticket #124846)

DQ Integration

  • When completing the connection mapping for your Collibra DQ to Collibra Data Intelligence Cloud integration, you now correctly see database views from Collibra DQ to the tables and columns to which they relate in Collibra Data Intelligence Cloud. (ticket #124191, #124213, #125676)

DQ Cloud

  • When upgrading to Collibra DQ version 2023.06, you can now see entries in your List View scorecards. Previously, there was a discrepancy between Edge and the Cloud metastore. (ticket #121624)

Pushdown

  • When running a Pushdown job with the /v3/jobs/run API, the username now correctly updates to the authenticated user. (ticket #121192)
  • When upgrading to Collibra DQ version 2023.07.2, you can now see the Data Preview for breaking record count for a freeform SQL rule against a Snowflake Pushdown dataset. (ticket #122585)

Known Limitations

Capabilities

  • There is a limitation with Validate Source where source columns containing white spaces do not map properly to the target columns.
    • A workaround is to remove the white spaces from the command line and then copy/paste the command line into a new DQ job.

DQ Security Metrics

Note The medium, high, and critical vulnerabilities of the DQ Connector are now resolved.

Warning We found 1 critical and 1 high CVE in our JFrog scan. Upon investigation, these CVEs are disputed by Red Hat and no fix is available. For more information, see the official statements from Red Hat:
https://access.redhat.com/security/cve/cve-2023-0687 (Critical)
https://access.redhat.com/security/cve/cve-2023-27534 (High)

Beta UI

Beta UI Status

The following table shows the status of the Beta redesign of Collibra DQ pages as of this release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules Done
Rule Workbench Rules Done
Data Classes Rules Done
Explorer Explorer Done
Reports Reports Done
Dataset Profile Profile Done
Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Note Admin pages are not yet fully available with the new Beta UI.

Beta UI Limitations

Explorer

  • When using the SQL compiler on the dataset overview for remote files, the Compile button is disabled because the execution of data files at the Spark layer is unsupported.
  • You cannot currently upload temp files from the new File Explorer page. This may be addressed in a future release.
  • The Formatted view tab on the File Explorer page only supports CSV files.

Connections

  • When adding a driver, if you enter the name of a folder that does not exist, a permission issue prevents the creation of a new folder.
    • A workaround is to use an existing folder.

Admin

  • When adding another external assignment queue from the Assignment Queue page, if an external assignment is already configured, the Test Connection and Submit buttons are disabled for the new connection. Only one external assignment queue can be configured at the same time.

Scorecards

  • When creating a new scorecard from the Page dropdown menu, because of a missing function, you cannot currently create a scorecard.
    • While a fix for this is planned for the September (2023.09) release, a workaround is to select the Create Scorecard workflow from the three dots menu instead.

Navigation

  • The Dataset Overview function on the Metadata Bar is not available for remote files.
  • The Dataset Overview modal throws errors for the following connection types:
    • BigQuery (Pushdown and Pullup)
    • Athena CDATA
    • Oracle
    • SAP HANA
  • The Dataset Overview function throws errors when you run SQL queries on datasets from S3 and BigQuery connections.

Release 2023.08

Highlights

    Pushdown
    We're delighted to announce that Pushdown processing for Databricks is now generally available! Pushdown is an alternative compute method for running DQ jobs, where Collibra DQ submits all of the job's processing directly to a SQL data warehouse, such as Databricks. When all of your data resides in Databricks, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirements of a DQ job.

Note The legacy documentation hosted at dq-docs.collibra.com has reached its end-of-life period and now has a redirect link to the official Collibra Data Quality & Observability documentation.

New Features

Capabilities

  • When reviewing outlier findings, you can now use the Invalidate All option to invalidate all outliers from a given job run in bulk.
  • When configuring rule details on the Rule Workbench, you can now define the Scoring Type as either a Percent, which is the default scoring type, or Absolute, which deducts points for breaking rules where the percentage is greater than 0.
  • When reviewing rule break findings, you can now select Rule Breaks from the Actions dropdown menu to preview the rule break export file and copy a signed link to the external storage location, giving you more control over how you use and share break records.

DQ Cloud

  • When upgrades of DQ Edge sites are required, you can now leverage a utility script to update the Edge DQ version without reinstalling the Edge site.
  • We've added the config parameter licenseSource to the Collibra DQ Helm chart to make it easier for our internal teams to update DQ Cloud licenses.
    • "config" is the default value for DQ Cloud deployments.

Pushdown

  • When archiving break records from Databricks Pushdown jobs, you can now write them directly to a database or schema in Databricks.
  • When you archive break records to the source warehouse, records are now pruned according to specified parameters to prevent tables or schemas from growing to unreasonable sizes. When a job gets pruned from the jobs table, the source break records from the datasource get pruned as well.

Enhancements

Capabilities

  • When scheduling jobs to run automatically, jobs now reflect the runId of the time and timezone you set. Previously, the runId of scheduled jobs reflected the default UTC server time, irrespective of the timezone you set.
  • When setting up a connection to a Google BigQuery data source, you can now use the Service Account Credential option to upload a GCP JSON credential file to authenticate your connection. This enhancement means you no longer need to use the workaround of uploading a JSON credential file as a base64 encoded Kerberos secret.

Platform

  • The endpoints for the controller-email API have changed. The following endpoints are now available:
    • GET /v2/email/server
    • POST /v2/email/server
    • POST /v2/email/server/validate
    • POST /v2/email/group
    • GET /v2/email/server/status
    • GET /v2/email/groups
    • Note For more information about the new endpoints, refer to the UI Internal option in Swagger.

Integration

  • When using either the Update Integration Credentials or Add New Integration modal to map a connection, the Connections tab now only displays the full database mapping when you click Show Full Mapping, which improves the loading time and enhances the overall experience of the Connections tab.
    • Additionally, there is now a Save and Continue button on the Connections tab to ensure your mappings save before proceeding to the next step.

DQ Cloud

  • Pendo tracking events no longer contain license_key information when log files are sent to Collibra Console.
  • We've improved the performance and resilience of Collibra DQ on Edge sites.

Pushdown

  • If Archive Break Records Location is not selected when setting up archive break records from the Connections page, the default schema is now the default schema of the database platform .yaml file. Previously, when a break records output location was not specified, the default location, PUBLIC, would be used.
  • When writing custom rules with rlike (regex) operators against Pushdown datasets, exception messages no longer throw when the job runs.
  • When running a Pushdown job from the Collibra DQ app, not via API, the correct column count displays on the Findings page. Previously, the v2/run-job-json returned empty columns, which resulted in the total number of columns displayed on the Findings page as 0.

Fixes

Capabilities

  • When adding a rule with a string match to a dataset where the string contains parentheses, extra spaces around the parentheses are no longer mistakenly added. (ticket #117055, 118319)
  • When selecting a region for an Amazon S3 connection, you can now use AP_SOUTHEAST_3 and AP_SOUTHEAST_4. (ticket #119535)
  • When assessing outlier percent change calculations on Findings, the percentage now displays correctly. (ticket #114045)
  • When using the out-of-the-box Template rule, Not_In_Current_Run as a dataset rule, an exception no longer throws when the job runs. (ticket #118401)

Platform

  • When your Collibra DQ session expires, you are now redirected to the sign-in page. (ticket #111578)
  • When migrating DQ jobs from one environment to another, columns that were selected to be included in an outlier check in the source environment now persist to the target environment. Previously, some columns that were selected in the source environment did not persist to the target. (ticket #115224)
  • When attempting to edit a completed job on a Redshift connection, the preview limit is now set to 30 rows. Previously, larger datasets experienced long load times or timed out, which prevented you from editing them from Explorer. (ticket #119831, 120245)
  • Fixed the Critical CVE CVE-2023-34034 by upgrading the spring library. (ticket #122280)
  • When running a job, you no longer receive SQL grammar errors. (ticket #120691)

Known Limitations

Capabilities

  • When using the rule breaks capability on the classic Rules Findings tab and rule break records from native rules do not exist in the metastore, the preview modal displays a blank preview and sample file.
  • When using the rule breaks capability and the remote archive location does not have write permissions, the exception details of the rule being archived are only visible on the Rules Findings tab.

Pushdown

  • The archive break records capability cannot be configured from the settings modal on the Explorer page for BigQuery Pushdown connections.
  • When using the archive break records capability, BigQuery Pushdown currently only supports rule break records.
    • Additional support is planned for an upcoming release.
  • When using the archive break records capability to archive rule breaks generated from freeform rules with explicitly selected columns, and not SELECT *, you must include the Link ID column in the rule query for break records to archive correctly.

  • Example A rule query that includes the Link ID column is SELECT sales_id, cost FROM @dataset WHERE cost < 2000 where "sales_id" represents the Link ID column.


  • When you select a date column as the column of reference in the time slice filter of a BigQuery dataset, an unsupported data type message displays. While this will be resolved in an upcoming release, a temporary workaround is to use the SQL View option to manually update the source query to reference a date column. For example, select * from example.nyse where trade_date = safe_cast('${rd}' as DATE)

DQ Security Metrics

Note The medium, high, and critical vulnerabilities of the DQ Connector are now resolved.

Warning We found 1 critical and 1 high CVE in our JFrog scan. Upon investigation, these CVEs are disputed by Red Hat and no fix is available. For more information, see the official statements from Red Hat:
https://access.redhat.com/security/cve/cve-2023-0687 (Critical)
https://access.redhat.com/security/cve/cve-2023-27534 (High)

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

a table showing the number of critical security vulnerabilities over a period of 5 releases

The following image shows a table of Collibra DQ security metrics arranged by release version.

a chart showing the number of critical security vulnerabilities over a period of 5 releases

Beta UI Redesign

The following table shows the status of the Beta redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new Beta pages is planned for an upcoming release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules Done
Rule Workbench Rules Done
Data Classes Rules Done
Explorer Explorer

In Progress

Reports Reports Done
Dataset Profile Profile Done
Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Note Admin pages are not yet fully available with the new Beta UI.

Beta UI Limitations

Explorer

  • When using the SQL compiler on the dataset overview for remote files, the Compile button is disabled because the execution of data files at the Spark layer is unsupported.
  • You cannot currently upload temp files from the new File Explorer page. This may be addressed in a future release.
  • The Formatted view tab on the File Explorer page only supports CSV files.

Connections

  • When adding a driver, if you enter the name of a folder that does not exist, a permission issue prevents the creation of a new folder.
    • A workaround is to use an existing folder.

Admin

  • When adding another external assignment queue from the Assignment Queue page, if an external assignment is already configured, the Test Connection and Submit buttons are disabled for the new connection. Only one external assignment queue can be configured at the same time.

Scorecards

  • When creating a new scorecard from the Page dropdown menu, because of a missing function, you cannot currently create a scorecard.
    • While a fix for this is planned for the September (2023.09) release, a workaround is to select the Create Scorecard workflow from the three dots menu instead.

Navigation

  • The Dataset Overview function on the metabar is not available for remote files.

Release 2023.07

Highlights

  • Integrations
  • When integrating metadata from Collibra DQ with Collibra Data Intelligence Cloud, you can now access the Integration page from the Admin Console in Collibra DQ. The Integration page features an integration configuration wizard, which leads you through a guided mapping, and a dataset status table to monitor and edit existing integrations.

    The integration configuration wizard lets you:
    • Seamlessly map data quality objects, such as tables, schemas, and columns to resources in Collibra Data Intelligence Cloud.
    • Align DQ tenants with Collibra Data Intelligence Cloud communities.
    • Map DQ dimensions to their corresponding dimensions in Collibra Data Intelligence Cloud.
  • By using these new capabilities, you can now efficiently synchronize Collibra DQ metadata with Collibra DGC with confidence and ease.
  • Pushdown
  • When storing break records from Snowflake Pushdown jobs, you can now write them directly to a database or schema in your data source. By removing the reliance on the PostgreSQL metastore for the archival of break records, you can:
    • Eliminate network traffic costs for better performance.
    • Reduce storage limit restrictions.
    • Improve security by generating break records that never leave Snowflake.
    • Enhance your reporting abilities by removing the need to move break records from the PostgreSQL metastore to your data source.
    • Join datasets against your break records.

Note The legacy documentation hosted at dq-docs.collibra.com will reach its end-of-life alongside the upcoming August (2023.08) release of Collibra Data Quality & Observability.

Enhancements

Capabilities

  • The Reports landing page now has improved organization, helpful descriptions of each report and dashboard type, and links to the documentation.
  • When using the v3/jobs API, you can now specify a time range to filter the list of jobs with the following parameters:
    • startTimeBeginning
    • startTimeEnd
    • updateTimeBeginning
    • updateTimeEnd
  • When reviewing the Rule Definitions dashboard, the Dataset column now includes links to the Findings page of the most recent run of a given dataset.
  • The labels no longer overlap when using the Rule Summary charts.
  • When using React, we renamed Catalog Dataset to Dataset Manager, and Column Manager, formerly called Catalog by Column, is now generally available.

Platform

  • When adding or modifying a connection and a Connection URL string or a property in the Driver Properties field is invalid, the error message that returns now includes helpful information to help you troubleshoot the error.

Pushdown

  • When including a shapes check in a Pushdown job, the shapes query now has a limit of the maximum number of shapes per column plus one to ignore any columns that exceed the maximum number of shapes per column.
  • When scanning for outliers in a Databricks Pushdown job, you can now select the Categorical option on the Outliers Configuration modal to check for categorical outliers.

Fixes

Capabilities

  • When profiling a PostgreSQL table that contains infinity , negative infinity -∞, NaN values in numeric columns, you can once again run jobs successfully. (ticket #112693)
  • When using the % symbol as a wildcard in queries against Db2 datasets, the % symbol no longer converts to an unreadable character, and the query runs correctly. (ticket #113618)
  • When running a scheduled job during a restricted time in a multi-tenant environment, the schedule restriction functionality again works correctly. Previously, jobs on certain tenants could still run during restricted times. (ticket #113794)
  • When setting up an SAP HANA connection, you can again successfully connect. Previously, a Java incompatibility prevented successful SAP HANA connections. (ticket #114883)
  • Note When using a JDK 8 build of Collibra DQ, you must use the SAP HANA driver provided in the Collibra DQ build package.

Platform

  • When reviewing the Breaks tab of a dataset's associated rules on CockroachDB connections, you can once again use the search bar to filter break records. (ticket #105410)
  • When configuring multi-tenancy on CockroachDB connections, you can once again add new tenants because the V193 migration script is now compatible with CockroachDB environments V22.1.x or later. (ticket #119528)
  • When reviewing the web logs for Explorer, all potential personally identifiable information (PII) is now removed from the logging message. (ticket #119375)
  • When deploying a Kubernetes pod, the security admission and RBAC clusternode warnings are now resolved. (ticket #113204)
  • When upgrading a multi-tenant deployment of Collibra DQ to a newer version, the agent display name no longer reverts to the agent name if you previously set the agent display name to something else. (ticket #114355, 119704)
  • When using the Alert Definitions page without an email server configured, an SMTP configuration error message incorrectly displayed based on a user's role. (ticket #114086)
  • When a user without a role mapped to a given connection attempts to access a dataset from Explorer via the Edit Dataset button on the Findings page, their access is now restricted until a role is assigned. Previously, role restrictions were not checked from Explorer. (ticket #115217)
  • When using AssumeRole on an S3 connection, the STS server now defaults to the same region that the connection is configured in. (ticket #118448)

Pushdown

  • When running a Snowflake Pushdown job that includes the profile activity, you no longer get an error message when a binary data type column exists in your dataset. (ticket #111977)

Known Limitations

DQ Cloud

  • Because Livy is not yet available for DQ Cloud, in order to display the data preview of remote files in the UI, you need to use temporary Spark scratch disk space to momentarily cache the preview. Until Livy is available for DQ Cloud, the caching of preview data occurs within DQ Cloud instead of Edge.
  • After upgrading a DQ Cloud instance, previously updated agent display names revert to the agent name. Any previously saved agent display names need to be manually re-entered after upgrading.

DQ Security Metrics

Note The medium, high, and critical vulnerabilities of the DQ Connector are now resolved.

Warning We found 1 critical and 1 high CVE in our JFrog scan. Upon investigation, these CVEs are disputed by Red Hat and no fix is available. For more information, see the official statements from Red Hat:
https://access.redhat.com/security/cve/cve-2023-0687 (Critical)
https://access.redhat.com/security/cve/cve-2023-27534 (High)

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

a table showing the number of critical security vulnerabilities over a period of 5 releases

The following image shows a table of Collibra DQ security metrics arranged by release version.

a chart showing the number of critical security vulnerabilities over a period of 5 releases

MUI Redesign

The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules Done
Rule Workbench Rules

In Progress

Data Classes Rules Done
Explorer Explorer

In Progress

Reports Reports

In Progress

Dataset Profile Profile

In Progress

Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Note Admin pages are not yet fully available with the new React MUI.

React Limitations

Explorer

  • When using the SQL compiler on the dataset overview for remote files, the Compile button is disabled because the execution of data files at the Spark layer is unsupported.
  • You cannot currently upload temp files from the new File Explorer page. This may be addressed in a future release.

Connections

  • When adding a driver, if you enter the name of a folder that does not exist, a permission issue prevents the creation of a new folder.
    • A workaround is to use an existing folder.

Admin

  • When adding another external assignment queue from the Assignment Queue page, if an external assignment is already configured, the Test Connection and Submit buttons are disabled for the new connection. Only one external assignment queue can be configured at the same time.

Scorecards

  • When creating a new scorecard from the Page dropdown menu, because of a missing function, you cannot currently create a scorecard.
    • While a fix for this is planned for the September (2023.09) release, a workaround is to select the Create Scorecard workflow from the three dots menu instead.

Navigation

  • While the breadcrumbs at the top of all legacy pages are no longer available in the new beta pages, as of the upcoming August (2023.08) release, a metadata bar will be anchored near the top of the Finding, Profile, Dataset Preview, and Alert Builder for quick access to all essential pages.

Maintenance Updates

2023.07.2

  • Fixed an issue where users who previously used the API integration on Collibra DQ version 2023.05 to import Collibra DQ metadata into Collibra Data Intelligence Cloud received an error message for Flyway scripts when migrating records from the integration status table to the integration history table after upgrading to Collibra DQ version 2023.07.
  • Fixed an issue where preview data on the Findings page did not display fully when column values used scientific notation.

2023.07.4

  • When using the integration API, the DQ Rule governs DQ Job relation of the DQ Rule asset no longer incorrectly identifies the "SOURCE" and "TARGET" properties in the integration JSON file.
  • Fixed a critical CVE (CVE-2023-34034) in Collibra Data Quality & Observability version 2023.07 that was identified after the release date.

Release 2023.06

Highlights

  • When running rules against datasets that return many breaking records, you can now optionally export the records to an external Amazon S3 or Azure storage bucket instead of writing them to the PostgreSQL Metastore. By offloading break records to cloud storage, you have more control over how you manage the results of your DQ jobs, and you can store data in the supported remote connection of your choice.
  • When building SQL rules to run against a dataset, you can now use the new Rule Workbench to create, edit, and preview SQL conditions to help verify that your data adheres to your organization's business requirements.

Enhancements

Capabilities

  • You can now scan data stored in Trino using the Starburst Trino JDBC connection.
    • Ensure that your connection URL contains the following substring: &source=jdbc:presto. For example, jdbc:trino://example-host.trino.galaxy.starburst.io:443/sample?SSL=true&source=jdbc:presto
  • When working with remote file connections, you can now leverage the Hudi Spark connector to scan Hudi files.
  • Note The Hudi Spark connector requires a separate package to work properly due to a security vulnerability in the Hudi bundle jar. Please reach out to your CSM for more information about accessing this package.

  • When using the Estimate Job feature to gauge the resource requirements of a Spark job, you can now set partitionautocal to true on the Admin Limits settings page to automatically calculate Spark partitioning for optimal performance results.
  • The Estimate Job feature now estimates the overall memory requirements instead of the memory per executor.
  • You can now use the Data Class and Template Rules pages to manage out-of-the-box and custom Data Class and Template rules.
  • The dayswithoutData and runswithoutData stat rules now count Day 0 and Run 0 as the first day and run without data.
  • The dayswithoutData stat rule now counts days with stale data for 365 days instead of 100.
  • When using Source to check a column for mismatched data, you can now select the Strict Source Downscoring option to include -validatevaluesthresholdstrictdownscore in your query during Source setup and base the downscore value on the number of cell mismatches found across all columns. When using this option, you must also assign a key column for source validation.
  • When a large number of Dupes are found in a DQ job run, the getDataPreviewByObsType API now has improved performance and no longer times out.

Platform

  • When adding a new dataset from a Google BigQuery connection, you can now successfully import, query, and identify EXTERNAL table types.
  • To address the conflict of JAASConf secret deletion with DQ jobs having longer Spark launch times for Driver and Executor pods, we adjusted the DQ agent's clean-up routine. (ticket #109256)

Pushdown

  • When configuring the settings of a Pushdown job, you can now choose to log all SQL queries for a dataset by turning SQL Logging on or off from the Settings modal.
  • When reviewing the Value column for Outliers on the Findings page, low-precision values now truncate after 2 decimal places when all the trailing values are 0. For example, 3780.0000000 now truncates to 3780.00. High-precision values, such as 162.9583789, still display all decimal values.
  • When adding ML layers for Databricks Pushdown jobs, you can now configure numerical outliers. Support for categorical outliers is planned for an upcoming release.
  • Because Pushdown is not yet available for DQ Cloud, the Pushdown checkbox option on the Databricks and Snowflake connection templates is temporarily unavailable when setting up a connection on a DQ Cloud deployment.

DQ Cloud

  • When admin or tenant admin users need to view the state of an Edge site identifier graph, you can now use the following APIs:
    • As admin
      • GET /v3/diagnostics/edgeSiteIdCache returns a text/plain body with a graph when successful.
      • DELETE /v3/diagnostics/edgeSiteIdCache returns no response body when successful.
    • As tenant admin
      • GET /tenant/v3/diagnostics/edgeSiteIdCache/{tenant} returns a text/plain body with a graph when successful.
      • DELETE /tenant/v3/diagnostics/edgeSiteIdCache/{tenant} returns no response body when successful.
    • Note The tenant admin API works only for the {tenant} where the user is tenant admin.

Fixes

Capabilities

  • When viewing Outlier records on the Findings page, you can again drill into individual records to view the associated graph. (ticket #110281)
  • Rules with null characters in the data again return rule breaks. Previously, null characters caused rule break inserts into the Metastore to return exception messages. (ticket #103402)
  • When exporting rule breaking records to Excel, integer values from Collibra DQ no longer display as decimal values in the Excel export file. (ticket #112830)
  • Fixed an issue where Shapes detection incorrectly assigned non-numeric text as numeric scientific notation. (ticket #113704)

Platform

  • When using an SSO configuration to sign into Collibra DQ, you no longer get errors when you attempt to sign in. The refactoring in this fix initially looks up a user profile by username, then by external user ID, before attempting to lookup by externalUserId until a user is found and returned. (ticket #109171)
  • When drilling into a column on the Profile page, Min and Max string length again displays the correct values. (ticket #110044)
  • When reviewing the Data Summary Report, the Table/File Name column now displays the correct name. (ticket #111508)
  • Fixed the builds for the 2023.05 Collibra DQ version. Previously, jobs failed with an "Unknown" status during the Alerts activity and showed a "NoClassDefFoundError" in the job log. (ticket #115956, 116052)
  • When running jobs on existing datasets with alerts configured, jobs no longer fail with an error when the ALERT_SCHEDULE_ENABLED option is set to TRUE. (ticket #112837)

DQ Cloud

  • When drilling into rule break records on the Findings page, data preview records again display correctly. (ticket #114162)
  • When reviewing the Completeness Report, data now displays correctly when you load the page and update as the inputs change. (ticket #115312)

Pushdown

  • When using query wrapping in Snowflake Pushdown profiling, you can now successfully run a profile scan without errors. (ticket #111977)

Known Limitations

Capabilities

  • When using BigQuery datasets and tables with names that begin with a number, there is a limitation that causes DQ jobs to fail.
    • A workaround is to wrap the dataset or table name with back quotes ` and then update the scope before you run the DQ job. For example, select * from `321_apples.example` and select * from `samples.311_service_requests`
  • When using the Rule Template to build SQL rules, there is a limitation with SQLF type rules where an extra select clause is added into the query when:
    • SQL keywords, such as select, from, and where are not lowercase.
      • A workaround is to write all keywords in lowercase.
    • "@DATASET", "@dataset", and "DATASET" are used to reference a dataset.
      • Instead, use "dataset" in all lowercase letters without an "@" symbol.
  • When exporting break records to external storage buckets, there is a limitation that prevents support for the feature on Amazon S3 and Azure connections that use SAML and Instance Profile authentication.
    • A workaround is to use Access Key authentication instead.
  • When using the getRuleBreaksCreateTable API to create a table statement for break records exported to an external storage bucket, there is currently a limitation that prevents the return of the proper response. While this issue will be addressed in the 2023.07 release, this API is not intended to be used at this time.

Platform

  • When leveraging the Spark 3.2.2 standalone installation that comes with the AWS marketplace installation, there is a limitation due to a jar mismatch for the AWS S3 Archive feature. The issue is that the existing hadoop-aws.3.2.1.jar file is incompatible with the feature.

    • A workaround is to update the hadoop-aws-3.2.1.jar to hadoop-aws-3.3.1.jar in the spark/jars directory. The necessary .jar file can be obtained from the following link: Apache Downloads.

      Note If you encounter any difficulties locating the necessary jar file on the Apache Downloads page, contact your CS or SE for assistance.

DQ Cloud

  • DQ Cloud does not currently support SAML configuration.
  • When reviewing the Completeness Report, new data only displays correctly after you upgrade your Collibra DQ Cloud instance to version 2023.05.2 or later.
  • If an Edge site needs to be reinstalled, you must use the original PostgreSQL metastore database or metastore corruption may occur in Collibra DQ. If necessary, restore the metastore database from a backup before reinstalling. Ensure that the installation command line parameter collibra_edge.collibra.dq.metastoreUrl points to the correct database.

DQ Security Metrics

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

the number of high and critical vulnerabilities over 5 months

The following image shows a table of Collibra DQ security metrics arranged by release version.

critical vulnerabilities over the last 5 release versions

MUI Redesign

The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules Done
Rule Workbench Rules

In Progress

Data Classes Rules Done
Explorer Explorer

In Progress

Reports Reports

In Progress

Dataset Profile Profile

In Progress

Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Release 2023.05

Highlights

  • We are excited to announce that Pushdown processing for Snowflake is now generally available! Pushdown is an alternative compute method for running DQ jobs, where Collibra DQ submits all of the job's processing directly to a SQL data warehouse, such as Snowflake. When all of your data resides in Snowflake, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirement of a DQ Job.
  • We are also excited to announce that Pushdown processing for Databricks is now available as a public beta! When you use Pushdown for Databricks ensure that PUSHDOWN_FOR_DATABRICKS is set to TRUE in the Application Configuration page of the Admin Console.
  • You can now use Swagger to leverage the new Integration API to send Collibra DQ rules, ML layer findings, and associated data quality scores to Collibra Data Intelligence Cloud. With data quality details in Collibra Data Intelligence Cloud, you can develop a more robust understanding of the health and performance of your organization's data.

New Features

Capabilities

  • When you create or edit a connection with REACT_MUI off, you can now define variables from the Connections template to securely enter sensitive properties, such as user credentials.
  • When you enable Include Links for recent job runs, the email alert now includes multiple contextual links to help you access the affected areas of the application.

Platform

  • You can now use a serverless agent to submit jobs from the UI. This lets you have a Collibra DQ installation without a native DQ agent, and allows for extensive agent customization and lightweight deployment options against new compute engines. This is currently only supported for Dataproc.

Pushdown

  • You can now set the maxconcurrentjobs config on the Admin Limits page to specify a maximum number of concurrent Pushdown jobs to run at once to prevent the Job service from overloading.
  • You can now use the new /v3/rules/{dataset}/validatePushdown endpoint to validate the syntax of Pushdown rules.

DQ Cloud

  • You can now edit and delete DQ Cloud connections.
    • When you delete a connection all of the links to its agent are also deleted. When you create a new connection in DQ Cloud, ensure that in the connections template your Connection URL is unique and select an agent from the Target Agent dropdown menu. This step is required and you cannot change your agent after the connection is created.

Enhancements

Capabilities

  • When you use a key comparison for source validation of decimal values that include scales of 0, you can now select a new Ignore Precision option from the Source configuration tab or set the flag, -validatevaluesignoreprecision, to true from the command line to ignore scales of 0 for integers before decimals and fractional values after decimals.

      Note -validatevaluesignoreprecision is only available for Validate Source when you include a key.

Platform

  • The connection template is now updated to the correct Impala JDBC driver, com.cloudera.impala.jdbc.Driver
  • The Kafka Streaming Connection is no longer available from the Connections Management page because it has finished its beta cycle and has not been promoted to becoming generally available.
  • The reactor-core package now supports Spark245 for successful Azure connections.
  • When you configure AdaptiveRules, Data Type Check is now included in the Schema detection activity.
  • When you select the Include Links option on the Alerts Configuration page, you must also enter your host address from the URL of your application into the HOST_NAME input field on the Application Configuration page. For example, if your Collibra DQ URL is http://dq.collibra.com, you need to enter http://dq.collibra.com into the HOST_NAME input field.
    • The Include Links feature for alerts now has logging to show when the Include Links option is selected but HOST_NAME is not set.
      • Informational text is now available on the Alerts Configuration page to help with the configuration of alerts.

Pushdown

  • When you run a Pushdown Job with Replay on, the start time of child Jobs in the Jobs queue now reflects their actual start times instead of the start time of the parent job from which the Replay initiated.
  • The Outliers option of the Add Layers step is now disabled for Databricks Pushdown job configuration.

DQ Cloud

  • DQ Cloud is now upgraded to the Collibra DQ 2023.05 version.

Fixes

Capabilities

  • Fixed an issue with the email lookup query of password reset requests that prevented users from resetting passwords. (ticket #110075)
  • Added a new admin limit flag, valsrcdisableaq, to fix a slow performance issue for record insertion into the assignment_q metastore table. (ticket #108547)
    • valsrcdisableaq lets you turn the connection between the assignmentQ feature and Source activity on or off. When you set it to true, you cannot invalidate or retrain any Source findings.
  • Removed the scan button from Explorer Connections because it was no longer supported from the UI. You can still profile multiple tables at the same time with the /v3/datasetDefs/ or /v2/run-catalog-scan-json/ endpoint. (ticket #107687)
  • Fixed an issue where, when DQ jobs were created from a BigQuery source and a Hive source was added, the suffix ‘/core’ was appended to the end of the driver name of the secondary dataset driver path, which resulted in long run times and an unavailable agent. (ticket #109796, #111432)
  • Fixed an issue with where loading a CSV file that use Custom delimiters on S3 remote file connections caused the file to hang in the load activity. (ticket #108890)
  • Fixed an issue with an incorrect mismatch in the validate source results, when both the source and target had values of zero. (ticket #106788)
  • Fixed an issue where attempts to invalidate findings in bulk hung on the "Updating Dataset" message because of an unsupported request method. (ticket #106826, 113881)
  • Fixed a permission issue on SQL Server Kerberos-based connections for queries run in Collibra DQ on system tables in SQL Server. (ticket #110773)

Platform

  • Fixed issues where external users could not successfully run jobs. (ticket #110773, 111988, 112417, 112857)
  • Fixed an issue with Collibra DQ on AWS Cloud where Instance Profile authentication caused scheduled S3 jobs to fail. (ticket #110570, 110574)
  • Addressed all the vulnerabilities of the optional drivers from Collibra DQ version 2023.03. You can now take the images with the optional drivers, as they now pass security scans. (ticket #112617)

DQ Cloud

  • Fixed an issue where scheduled jobs did not run because they became stuck in "Staged" status. (ticket #112802)

Known Limitations

Capabilities

  • A limitation with S3 remote file connections that use an escape character as a delimiter causes columns to parse incorrectly. For example: /a
  • When you edit an existing connection and its details, another connection is created and any sensitive properties from the old connection do not persist in the new connection.
    • A workaround is to recreate the sensitive properties from the old connection on the new connection.
  • When you create or edit connections, the new sensitive properties feature is only available with React turned off.
    • This only works for datasets created before 2023.05. Permalinks to specific Job runs in alert emails only work for datasets that run after the introduction of this feature in Collibra DQ version 2023.05. If a dataset was created before this implementation and runs on a schedule, the permalinks will not work.
  • Native SQL is not currently supported on the Rules Workbench.

Platform

  • If you are an external user of Collibra DQ versions 2023.03, 2023.04, and their patches, you may experience issues where completed jobs do not send alerts. If this occurs, set ALERT_SCHEDULE_ENABLED=false and then restart web and agent.
  • If a license key value is damaged or deleted in a current app session you may experience job failures with license key errors.
    • If you know that your deployment of Collibra DQ has a valid license key, you can attempt to refresh the page, sign out of the app and then sign back in. This will sync your license key with the metastore, which will allow you to once again run jobs without license key expiration job failures.

DQ Cloud

  • Instance Profile is not supported for S3 connections.
  • When using the Completeness Report, data only appears after upgrading to 2023.06 or later.
  • When using the Findings page, you currently cannot drill into a rule break record. While there is no workaround for this limitation, a fix is planned for the 2023.06 release.
  • When using the Findings page, you currently cannot tag job runs as off-peak. This will be fixed in the 2023.07 release.

Pushdown

  • When scanning for dupes, fuzzy match duplicates display incorrectly as "NULL" values on the Findings page. This will be fixed in the 2023.06 release.

DQ Connector

  • When navigating the Admin Console menu with React turned on, an "Integrations" option leads to a non-functional page. This will be fixed in the 2023.06 release.

DQ Security Metrics

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

security vulnerabilities over 5 months

The following image shows a table of Collibra DQ security metrics arranged by release version.

critical security vulnerabilities over 5 months

MUI Redesign

The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules Done
Rule Workbench Rules

In Progress

Rule Builder Rules

In Progress

Data Classes Rules Done
Explorer Explorer

In Progress

Reports Reports

In Progress

Dataset Profile Profile

In Progress

Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Maintenance Updates

2023.05.1

  • Fixed the builds for the 2023.05 Collibra DQ version. Previously, jobs failed with an "Unknown" status during the Alerts activity and showed a "NoClassDefFoundError" in the job log. (ticket #115956, 116052)

Release 2023.04

Highlights

  • We are excited to announce that Pushdown for Snowflake is now generally available! Pushdown is an alternative compute method for running a DQ Job, where Collibra DQ submits all of the Job's processing directly to a SQL data warehouse, such as Snowflake. When all of your data resides in Snowflake, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirement of a DQ Job.
  • Six new out-of-the-box reports are now available:

Note 
With the development of the newly designed Template Rules page in the upcoming the May release, we will use the newly built v2/createcustomrule API for custom rules and will deprecate the old v2/insertcustomrule.

Warning 
When you enable Include Links on the Alerts Configuration page of the Admin Console, you must also enter your host address in the HOST_NAME input field on the Application Configuration page. If the HOST_NAME input field does not contain a host address when Include Links is enabled, your DQ Jobs will fail.

New Features

Reports

  • The User Activity, Row Count, Alert Details, Job Runs per User, Daily Job Runs per User, and Rule Updates per User reports are now available.

Enhancements

Rules

  • You can now view rows scanned and rows breaking into the rule_output table.

Reports

  • You can now view and filter reports on column dimensions with the Column Dimension Dashboard. This dashboard provides an overview of all columns, datasets, and business units, and provides insights about total current DQ Dimension scores, Dimension scores over time, and DQ scores (along with other metadata) for each column.
    • This dashboard is the first version and design adjustments are planned for a future release.
    • We also plan to redesign the Reports page in an upcoming release to better list all charts and dashboards and make it easier for you to know what we offer and potential use cases for the reports and dashboards.
    • Contact the Collibra DQ Product team or your CSM to learn more.

Platform

  • In order to further enable Kubernetes-deploying users to more easily configure certain parameters upon deployment, the following variables are now available as owl-web-configmap properties:
    • SAML_MAX_AUTH_AGE
    • SAML_ENABLED
    • SAML_ENTITY_ID
    • CORS_ALLOWED_ORIGINS
    • SAML_ENTITY_BASEURL
  • With the MUI turned on, the DQ Score Cards tab is now called Views.

Pushdown

  • Record change observations are now included on the Findings page for Pushdown Jobs.
  • The functionality of the v3/rules/{source}/copy API to work across Pull-Up and Pushdown datasets is now extended.
  • You can now add a description of your dataset from the Review page before you create a Pushdown Job.

Fixes

Rules

  • Fixed an issue with the Rule Builder that prevented rules from properly validating. (ticket #106100)
  • Fixed an issue with the Rule Builder that prevented the selection of available columns and limited the ability to scroll through the list of options on the Primary Column dropdown list. (ticket #110276)
  • Fixed an issue with the rule breaks table within the Breaks tab where sorting the table by the Rule Name column caused an error to throw. (ticket #105410)
  • Fixed an issue where, when a rule had sub-select at the beginning of the WHERE clause, the rule displayed an Exception status when it ran. (ticket #109812)

Explorer

  • Fixed an issue with the Transform option on Athena connections, the DATE() function was incorrectly included when the job was executed with an unlocked Run CMD. (ticket #105316)

Platform

  • Fixed an issue where, while editing an existing job in Explorer, Collibra DQ became stuck in the loading stage. (ticket #105573)
  • Fixed an issue that incorrectly allowed the creation of local users even with LOCAL_REGISTRATION_ENABLED set to false. (ticket #110325)
  • Fixed an issue where a role-less user was able to access the Edit Rule modal from the Rule table on the Findings page. (ticket #104129)
  • Fixed an issue where the install packages for Collibra DQ versions 2023.02 and 2023.03 did not contain the owlmanage.sh script. (ticket #110786)

Security

  • Fixed a potential security issue where the introduction of a smart pre-admin approval message on the sign-in screen in the 2023.03 release version is now reverted to the previous sign-in error message.

APIs

  • Fixed an issue the /v2/get-job-log API where the Job Log did not load and threw an error when more than two datasets were included in a rule. (ticket # 105498)

Connections

  • Fixed an issue where too many open connections prevented new connections from being opened. (ticket #107034)
  • Fixed an issue where a database connection remained idle and did not close when the /v2/getsqlresult API was called. (ticket #108365)

Source

  • Fixed an issue where Redshift source queries failed when -profilePushdown count option was used. (ticket #106085)
  • Fixed an issue with the Cell Threshold invalidation logic which caused the issue count to remain the same after the issue was invalidated and the job re-ran.

Known Limitations

Connections

  • A limitation with Hive JDBC was observed where having both JDBC41 and JDBC42 drivers caused a conflict that prevents table data from loading. As a result, this release includes only the JDBC41 driver. This driver will be upgraded in a future release.
  • Hive Direct job execution is currently not supported due to a technical issue that we plan to fix in an upcoming release.

Capabilities

  • Sub-selecting columns of targets in the Source activity results in a failed Job when the Job runs.
    • A possible workaround is to keep the query as a select all (select *) operation.
  • When you enable Include Links on the Alerts Configuration page of the Admin Console, you must also enter your host address in the HOST_NAME input field on the Application Configuration page.
    • A fix for this is planned for the May release.

Pushdown

  • Pushdown is currently only available for on-premises deployments of Collibra DQ.
    • Support for Pushdown in DQ Cloud is planned for a future release.

DQ Connector

  • There is an issue where Rest API calls from Collibra DGC to Collibra DQ result in a 403 Forbidden Error because Collibra DGC passes a cookie header which causes Collibra DQ to reject the request due to recent updates to CSRF token requirements. This limits Collibra DQ's ability to connect to Collibra Data Intelligence Cloud.
    • A possible workaround until a fix is available is to update export CSRF_TOKEN_ENABLED=false in the owl-env.sh file for Standalone deployments or update dq.security.csrf.token.enabled=${CSRF_TOKEN_ENABLED:false} in the Web ConfigMap for Kubernetes deployments.

DQ Security Metrics

The following image shows a chart of Collibra DQ security vulnerabilities arranged by release version.

Critical vulnerabilities over 5 months

The following image shows a table of Collibra DQ security metrics arranged by release version.

Critical vulnerabilities table

MUI Redesign

The following table shows the status of the MUI redesign of Collibra DQ pages as of this release. Because the status of these pages only reflects Collibra DQ's internal test environment and completed engineering work, pages marked as "Done" are not necessarily available externally. Full availability of the new MUI pages is planned for an upcoming release.

Page Location Status
Homepage Homepage Done
Sidebar navigation Sidebar navigation Done
User Profile User Profile Done
List View Views Done
Assignments Views Done
Pulse View Views Done
Catalog by Column (Column Manager) Catalog (Column Manager) Done
Dataset Manager Dataset Manager Done
Alert Definition Alerts Done
Alert Notification Alerts Done
View Alerts Alerts Done
Jobs Jobs Done
Jobs Schedule Jobs Schedule Done
Rule Definitions Rules Done
Rule Summary Rules Done
Rule Templates Rules

In Progress

Rule Workbench Rules

In Progress

Rule Builder Rules

In Progress

Data Classes Rules

To Do

Explorer Explorer

In Progress

Reports Reports

In Progress

Dataset Profile Profile

In Progress

Dataset Findings Findings Done
Sign-in Page Sign-in Page Done

Maintenance updates

2023.04.1

  • External users can now successfully run Jobs that include alert links.
    • The email address of the owner of a Job with alert links is no longer included in alert emails for both local and external users.

2023.04.2

  • External users can now successfully run Jobs that include alert links.
    • The email address of the owner of a Job with alert links is no longer included in alert emails for both local and external users.
  • Fixed an issue with Collibra DQ on AWS Cloud where Instance Profile authentication caused scheduled S3 Jobs to fail.