Before you install on self-hosted Spark Standalone

Prerequisites

• Review the Spark Standalone architecture described in Installing Data Quality & Observability Classic on self-hosted Spark Standalone.
• Verify that your system meets the System requirements.

Download the Collibra DQ install package

You can download the DQ installation package from the Collibra Product Resource Center.

Save the package in a directory on the CDQ VM, to be extracted later for the install.

Extract the file:

tar -xvf dq-full-package.tar.gz

(Optional) Clean up:

rm dq-full-package.tar.gz

Set the environment variables

Set the following environment variables:

• OWL_BASE: The directory to install DQ files
• OWL_METASTORE_USER: The default username for the PostgreSQL server
• OWL_METASTORE_PASS: The default password for the PostgreSQL server. It must adhere to the following password policy:
  • Minimum length of 8 characters
  • Maximum length of 72 characters
  • At least one upper-case letter
  • At least one numeric character
  • At least one special character (supported are !,%,&,@,#,$,^,,?,_,~)*
  • Password cannot contain the PostgreSQL username
• SPARK_PACKAGE
• SCHEUDLE_RECOVERY_ENABLED: Restarts jobs left in an unfinished state due to an unscheduled server crash. When set to TRUE, the scheduler automatically adds these unfinished jobs to the job execution queue, allowing them to run and complete without user intervention.
• DQ_ADMIN_USER_PASSWORD: The password for the admin account on the DQ Web application. It must adhere to the following password policy:
  • Minimum length of 8 characters
  • Maximum length of 72 characters
  • At least one upper-case letter
  • At least one numeric character
  • At least one special character (supported are !,%,&,@,#,$,^,,?,_,~)*
  • Password cannot contain the username (admin)

For example, execute the following commands to set the environment variables:

export OWL_BASE=$(pwd)
export OWL_METASTORE_USER=postgres
export OWL_METASTORE_PASS=H55Mt5EbXh1a%\$aiX6
export SPARK_PACKAGE=spark-3.5.6-bin-hadoop3.tgz
export DQ_ADMIN_USER_PASSWORD=<password>

What's next

Install Standalone