Freeform

Fully defined SQL for more detailed checks.

Where 'Simple' rules just use the condition, 'Freeform' rules use the complete SQL statement. When more complex SQL is required, you can express more with Freeform including joins, CTE's and window statements.

select * from @DATASET_NAME where 
email_address is not null and email_address != '' 

Warning Freeform rules should include a semi-colon at the end of the rule value.

Note  All built-in Spark functions for non-Pushdown connections are available for use. See (https://spark.apache.org/docs/2.3.0/api/sql/) for Simple and Freeform SQL rules.‌

Cross-Connection Libraries

Note When applying cross-connection rules please use the -addlib to submit the job with the appropriate jar files. In this example, a secondary set of jars is added through the Explorer. These files are located in the /opt/owl/drivers/mysql directory. The path should not contain double quotes or single quotes. It should point to a directory without spaces in the path.

Known Limitations

Freeform rules with fully qualified column names are currently unsupported when they use the following syntax:

select <column name> FROM @<dataset name> WHERE @<dataset name>.<column name> condition

A possible workaround to this limitation is to use aliasing instead.