Fully defined SQL for more detailed checks.
Where 'Simple' rules just use the condition, 'Freeform' rules use the complete SQL statement. When more complex SQL is required, you can express more with Freeform including joins and window statements.
Copy
select * from @DATASET_NAME where
email_address is not null and email_address != '' Note Use the built-in Spark functions for non-Pushdown connections. Native SQL will not work, unless you switch to Native SQL rules. See (https://spark.apache.org/docs/2.3.0/api/sql/) for Simple and Freeform SQL rules.
Important Column name references on the Spark dataframe do not necessarily match the column names.
For more information, go to Freeform SQL.