DQ Workflows

Benefits

The DQ Workflows package listed on Collibra Marketplace allows you to 1) create and manage Data Quality Issues, 2) receive Notifications on Rule Metrics, and 3) request Rule Creation and Modification within Collibra Data Intelligence Cloud. Data stewards will be able to organize and prioritize all requests within DIC before they take any action within Collibra Data Quality.

Once deployed, the workflows will facilitate quicker data issue remediation by involving business analysts and other personas who can now participate in your data quality workstreams.

Please note: DQ Workflows are listed on Collibra Marketplace and are templates to get customers started. Collibra-provided Marketplace listings are not subject to the same SLA obligations (https://marketplace.collibra.com/marketplace-terms/) In addition, they can only be leveraged within Collibra Data Intelligence Cloud. In the future, we will work towards releasing bi-directional workflows.

Step 0: Prerequisites

Resource Notes
Collibra Edge Site DQ Connector is a capability of Edge
Collibra Data Intelligence Cloud 2021.07 Release (or newer)
Collibra Data Quality 2.15 (or newer)
Collibra DQ Connector Synchronized Rules from Data Quality to Catalog

Note After gathering all the prerequisites, you can now proceed to the next step.

Step 1: Download, Deploy and Start DQ Workflows

1A. Download Package from Collibra Marketplace and Unzip Files

**1B. Deploy Workflows **

1C. Adjust Workflow Settings (One-Time Setup)

Workflow Configuration Setting DQ Rule / DQ Sync Request DQ Rule Modification DQ Data Remediation DQ Issue Resolution Manage DQ Subscriptions Notify of DQ Metrics
Applies To Asset Asset Asset Asset Global Global
Applies To Asset Type Column, Table Column, Table, Data Quality Rule Column, Table Issue    
Other: Any Signed In User Can Start Workflow Y Y     Y Y
Other: Perform Candidate User Check on Workflow Start Y Y Y   Y Y
Other: This Workflow Can Only Run Once At Same Time on Specific Resource         Y Y
Other: Show In Global Create         Y Y
Roles: Start Workflow         Sysadmin Sysadmin
Roles: Stop Workflow         Sysadmin Sysadmin
Roles: Reassign Tasks         Sysadmin Sysadmin

Example: DQ Rule Request

Example: Manage DQ Subscriptions

Step 2: Create Data Quality Requests / Issues

2A. Create Data Quality Issues

Workflow Main Requestor Persona Description Steward Taking Action
DQ Data Remediation

Data Steward

Business Analyst

 Tracking / management for confirmed data issues which may require underlying data remediation Data Lake Admin, ETL Engineer
DQ Rule Request Business Analyst Proposing data quality rules in plain language e.g. "flag any German phone numbers in this dataset" or "identify customers with churn risk based on engagement time with our platform" Data Steward
DQ Rule Modification Business Analyst Proposing adjustments to existing rules e.g. values, dimensions, passing thresholds Data Steward
DQ Sync Request Business Analyst Request for synchronization of the DQ Connector to synchronize and/or onboarding a new dataset with pre-populated rules Data Steward

Example: Data Quality Rule Request

Completed Rule Request Example

Example: Data Quality Rule Modification Request

Completed Data Quality Rule Modification Example

Example: DQ Synchronization Request with Rules Request optional checkbox

Step 3: Manage Data Quality Issues

3A. Setup Data Helpdesk Filter

Data Helpdesk

  • Select Issues
  • Navigate to 'Filters'
  • Properties > Attributes > Relations > Issue **categorized by **Issue Category > Input 'Data Quality Issue' > Apply
  • Save button > Save View as > 'Data Quality Issues'
  • Optional settings for View: Can pin, promote, make public, make default

3B. Manage Issues From Data Helpdesk View

3C. Alternate: Manage Issues From Tasks

Step 4: Receive Notifications Of DQ Issues And Metrics

4A. Set Up DQ Metric Subscription

Who? Anyone can set up a DQ subscription, for yourself or for your teammates.

Alerts will be sent based on reviewing rules and metrics associated with Tables or Columns that violate the specified Threshold.

Assuming an e-mail is associated with the Subscriber within Collibra, the Subscriber will receive e-mail notifications by default at 12pm local server time. This, along with other settings within the provided workflow, can be adjusted in Eclipse, Collibra's recommended workflow editor.

4B. Review DQ Metric Alerts

Ensure that the DQ alerts set for you are providing helpful details.

4C. Update Subscription Settings

For every subscription set up for a Subscriber, the Manage DQ Subscriptions modal will cycle through for your review. You can update Threshold, add or delete Notification Days, add or delete Tables or Columns, rename the Subscription title, Save the new settings, or simply Unsubscribe.