About interactive sessions
- Validate that your SQL rule compiles and runs against your data while you write it.
- Preview the records your rule returns, with results limited to 25, 50, or 100 rows.
- Review live break records from the Monitors tab when archive break records is not enabled.
Session lifecycle
An interactive session is a short-lived Spark Session that runs on your Edge deployment. When you select a Pullup job in the Rule Workbench, the session starts and loads the data for that job.
While the session starts, the Rule Workbench shows the following statuses in sequence: Initializing session, Session starting, Engine starting, Validating connection, and Loading new dataset. When the session is active, a green chip shows Session active.
While the session is starting, you can still use the schema preview, the primary column drop-down list, SQL AI, and all rule settings fields. The Validate SQL and Validate Rule buttons become active only when the session reaches Session active.
Once a session is active, you can run one query at a time. The Validate SQL and Validate Rule buttons become inactive while a query is running and become active again when the result returns.
A session shuts down automatically after 15 minutes of inactivity. If you navigate away from the Rule Workbench while a session is running, Collibra shows a confirmation prompt before the page closes.
If the maximum number of simultaneous sessions is reached, the Rule Workbench shows "Max sessions reached" until a session ends or you navigate away.
Validate and preview rules
With an active session, you have two ways to check your rule before saving:
- Validate Rule: Runs your SQL rule against the data in the session and returns a pass or error result. If the rule is not valid, the error message describes what to correct.
- Rule preview: Runs the rule and returns a sample of matching records. Use the drop-down list next to the preview button to set the result limit to 25, 50, or 100 rows.
Rules for Pullup jobs run in Spark SQL. The SQL editor uses Spark SQL 4.1 autocomplete suggestions, and SQL AI generates Spark SQL.
Break records in the Monitors tab
When archive break records is not enabled for a Pullup job, the Break Records tab in the rule actions drawer uses an interactive session to show matching records live.
If a session is not active when you open break records, a button lets you start one for the job's connection. If an active session belongs to a different connection, a prompt lets you restart the session for the current job's connection.
Session resources
Interactive sessions start with minimal Spark resources. When Spark can push filter and aggregation operations directly to your data source, the session uses those minimal resources throughout.