Connecting to Network File Storage (NFS)

This section contains an overview of Network File Storage (NFS).

General information

Field Description
Data source Network File Storage (NFS)
Supported versions N/A
Connection string nfs:///
Packaged?

Yes

Certified?

Yes

Supported features
Analyze data

Yes

Archive breaking records

No

Estimate job

Yes

Note Estimate job is only available for NFS connections on Standalone deployments of Collibra DQ.
Pushdown

No

Processing capabilities
Spark agent

Yes

Note 
  • Spark agent is available for NFS connections on Standalone deployments of Collibra DQ. Additional configurations are required for Kubernetes and Hadoop deployments.

  • A Spark configuration must be added to the command line to mount the NFS directory path and run the NFS remote connections in Kubernetes. Refer to the Review page for more information on appending using the command line.

    The following Spark configurations are related to mounting NFS volume in Kubernetes:

    • spark.kubernetes.driver.volumes.persistentVolumeClaim.nfs-pv-dq-dev.mount.path=/opt/owl/nfs-storage

    • spark.kubernetes.driver.volumes.persistentVolumeClaim.nfs-pv-dq-dev.options.claimName=nfs-pvc-dq-dev

    • spark.kubernetes.executor.volumes.persistentVolumeClaim.nfs-pv-dq-dev.mount.path=/opt/owl/nfs-storage

    • spark.kubernetes.executor.volumes.persistentVolumeClaim.nfs-pv-dq-dev.options.claimName=nfs-pvc-dq-dev
Yarn agent

Yes

Note Yarn agent is available for NFS connections on Standalone deployments of Collibra DQ. Additional configurations are required for Kubernetes and Hadoop deployments.

Minimum user permissions

In order for Collibra DQ to access your local file system, you need the following permissions.

  • Ensure that you can mount the NFS in your Collibra DQ service locally.
  • Ensure that the Linux user has read permissions on the path that runs Collibra DQ services.

Recommended and required connection properties

Required Connection Property Type Value

Yes

Name Text The unique name used for your connection.

Yes

Connection URL String

The connection string value of your NFS connection.

nfs:///your/directory/path/

No

Target Agent Text The Agent used to submit your DQ Job.

Yes

Auth Type Option

The method to authenticate your connection.

Note Auth Type is always NFS.

No

Driver Properties String

The configurable driver properties for your connection. Multiple properties must be comma delimited. For example, abc=123,test=true

Authentication

Auth Type must be set to NFS.