Connecting to Dremio

This section contains details for Dremio connections.

General information

Field Description
Data source Dremio
Supported versions 24.3.0
Connection string jdbc:dremio:direct=
Packaged?

No

Certified?

No

Supported features
Estimate job

Yes

Analyze data

Yes

Schedule

Yes

Processing capabilities
Pushdown

No

Spark agent

Yes

Yarn agent

Yes

Parallel JDBC

Yes

Java Platform version compatibility
JDK 8

Yes

JDK 11

Yes

Note The Dremio connection is compatible with JDK11 if you add the following to owlmanage.sh as a JVM option for the web and Spark instance:
-Dcdjd.io.netty.tryReflectionSetAccessible=true

Minimum user permissions

In order to bring your Dremio data into Collibra Data Quality & Observability, you need the following permissions.

  • Read access on your Dremio tables.
  • ROLE_ADMIN assigned to your user in Collibra DQ.

Recommended and required connection properties

Required Connection Property Type Value

Yes

Name Text The unique name of your connection. Ensure that there are no spaces in your connection name.

Yes

Connection URL String

The connection string path of your Dremio connection.

When referring to the example below, replace the ${value} sections of the connection URL with your actual value.

Example jdbc:dremio:direct=${host};schema=Samples

Yes

Driver Name String

The driver class name of your Dremio connection.

com.dremio.jdbc.Driver

Yes

Port Integer

The port number to establish a connection to the datasource.

The default port is 31010

No

Limit Schemas Option

Allows you to manage usage and restrict visibility to only the necessary schemas in the Explorer tree.

See Limiting schemas to learn how to limit schemas from the Connection Management page.

Note When you include a restricted schema in the query of a DQ Job, the query scope may be overwritten when the job runs. While only the schemas you selected when you set up the connection are shown in the Explorer menu, users are not restricted from running SQL queries on any schema from the data source.

No

Source Name String N/A

No

Target Agent Option The Agent that submits your Spark job for processing.

Yes

Auth Type Option

The method to authenticate your connection.

Note The configuration requirements are different depending on the Auth Type you select. See Authentication for more details on available authentication types.

No

Properties String

The configurable driver properties for your connection. Multiple properties must be comma delimited. For example, abc=123,test=true

Authentication

This data source supports the following authentication type:

Username/Password

Required Field Description

Yes

Username The username of your Dremio account.

Yes

Password The password of your Dremio account.

Known limitations

  • When you run a DQ job with a SQL query that attempts to perform a WHERE on a date column, the job fails. For example, select * from "@admin"."myspace" WHERE registration_dttm = '2016-02-03T07:55:29'. There is currently no known workaround for this limitation.
  • When you run a DQ job with a SQL query that does not contain an explicit WHERE clause, the job fails with an exception message that the command line does not contain a -rd. A workaround for this limitation is to add WHERE 1=1 to the query.
  • When you run a DQ job with a query that contains both a WHERE and LIMIT clause, the SQL preview returns all records and ignores both filters.