Run your DQ job

After you review your DQ job, you can run it. There are two options for running a DQ job:

  • Run with your specified parameters.
  • Run with Replay and selected back run parameters.

Running with Replay

Replay creates back runs of your DQ job for the number of days to establish a historical baseline profile of the data. When you use a Time Slice filter, Replay inserts the run date from the time slice grouping for the number of days, months, or years you set as the Back Run value.

For example, if the Start date of the Time Slice filter is 2023-03-31 and you use Month as your Back Run Time Bin option, and set 10 as your Back Run value, then there will be 11 different runs with 11 different run dates. The first run is the entire month of March 2023 and has a run date of 2023-03-31, while the eleventh run is the entire month of May 2022 and has a run date of 2022-05-31. Every run between these months follows the same pattern where the run date is the last day of the month and the time bin is the first through last day of the month.

Configuring Replay

  1. Click the down arrow button next to Run, then click Replay.
    2. The Replay modal appears.
  2. Select from the available options.
    3. Option Description
    Back Run Time Bin The grouping of the replay. The available options are Day, Month, and Year.
    Back Run The number of time that will replay from a specified number. Click and drag the slider or enter a value between 1-50 in the input field next to the slider.
    Preview Previews the Time Bin per replay run.
  3. Click Run.

    Note This is an alternative run option to the default run without Replay.

What happens when a job runs?

When a job runs, Collibra DQ assigns it a Run Id and submits it to the Jobs page. When you run a job in Pullup mode, a SparkSession is created for Spark to submit the job for processing or the job's metadata is submitted directly to the data warehouse for processing. Conversely, a job running in Pushdown mode is submitted directly to its source database for processing without the need for intervention from a Spark agent.