Complete the Initial Setup

After installing Collibra DQ Standalone, you are ready to complete the initial setup.

Log in to Collibra DQ

  1. When the setup.sh script finishes, the DQ Web application is automatically started. Verify that it is running by opening the following URL in your browser:

    2. <YOUR_SERVER_IP>:9000

    The Collibra DQ login page displays.

    Login page of DQ Web UI

  2. Log in using your credentials.

Verify that the Spark Cluster has started

  1. Open the following URL in your browser:

    2. <YOUR_SERVER_IP>:8080

    Note This port may not be open depending on your environment.

    The Spark Master page displays.

    Fig 3: Spark Master Web UI

  2. Take note of the Spark Master URL. It is required during DQ Agent configuration.

Set the license key

  1. In the DQ Web, open Admin Console and click License.

  2. Enter your license key and license name in the fields provided. The key and name are provided by Collibra in your license provision email.

  3. Click Save.

    4. License dialog

Update DQ Agent properties

In this step, you'll start the DQ Agent, which creates a file called agent.properties. You'll then stop the agent, edit the agent.properties file, and then restart DQ Agent.

  1. Start the DQ Agent. Enter the following commands:
    2. Copy 
    cd $OWL_BASE/owl/bin
    ./owlmanage.sh start=owlagent

    This process creates the file agent.properties in the $OWL_BASE/owl/config directory.

    Note To see the directory structure created by setup.sh, go to Collibra DQ Directory Structure.
  2. Stop the DQ Agent. Enter the following command:
    3. Copy 
    ./owlmanage.sh stop=owlagent
  3. Add the following lines to the agent.properties file:
    4. Copy 
    sparksubmitmode=native
    sparkhome=</your/spark/home/folder>
  4. Start the agent again. Enter the following command:
    5. Copy 
    ./owlmanage.sh start=owlagent
  5. The agent.properties file identifies the number of agents installed on your server. Since this is the first non-default agent installed, the expected number of agents is 2. Verify that agent.properties contains the correct details.
    6. Copy 
    cd $OWL_BASE/owl/config
    cat agent.properties

    The output should include agentid=2.

Edit the DQ Agent configuration

  1. In the DQ Web, open Admin Console and click Agent Configuration.
  2. In the row that displays 2 under Agent ID, click Actions and then click Edit. The Edit Agent dialog displays.

    3. Edit Agent dialog

    Notice that the paths in the following fields use /opt/owl/ as the base path, which do not match the base path to your instance of Collibra DQ:

    • Base Path
    • Collibra DQ Core JAR
    • Collibra DQ Core Logs
    • Collibra DQ Script
    • Collibra DQ Web Logs
  3. Update the paths in these fields to include the proper base path.
    4. Important  The owlmanage.sh start=owlagent script does not respect the OWL_BASE environment variable. You must edit the Agent Configuration to follow the OWL_BASE.
  4. In the Default Master Path field, enter the Spark Master URL that you copied from the Spark Master page.
  5. Update the values in the following fields based on the requirements of your instance:
    • Number of Executors(s)
    • Executor Memory (GB)
    • No. of Core(s)
    • Driver Memory (GB)
  6. In the Free form (Appended) field, enter the following command:
    7. Copy 
    -conf spark.cores.max=8
  7. In the Default Deployment Mode field, select Client.

For more information on configuring the DQ Agent, go to Set up a DQ agent.

Map a Connection to the DQ Agent

Follow the steps provided in Map data source connections to a DQ agent.

Run the first DQ job

  1. In the DQ Web, open the Explorer page.
  2. Select a connection, schema, and table.
  3. Click Create DQ Job.

For more information on DQ jobs, go to Create a DQ job.

What's next?

Troubleshooting Standalone Install