Configuring archive break records
This topic shows you how to configure archive break records to send details of breaking records from jobs to your data source. Enabling this feature allows you to automatically store row-level data for failed quality checks directly in your database.
You can configure the storage location at the connection level to align with your organization's data architecture.
Note Archive break records is currently only available for Pushdown connections.
Prerequisites
To archive break records, you need the following permissions and configurations:
| Requirement | Details |
|---|---|
| Collibra permissions |
|
| Data source privileges for break records |
Because exact permission models vary significantly across different data platforms, consult your organization's database administrator to provision the exact permissions. The service account connecting to your data source requires the following generalized privileges: On the source data (to monitor records):
On the destination schema (to archive break records):
Note You only need
create access once per break records table, or you can create it manually with explicit update access. All monitors write to the same break records table, so create access can be safely revoked after initial setup. If the break records storage location changes, create access is required again to generate a new table. |
| Connection configuration |
You have an established connection. |
Enabling archive break records
Important Ensure that the service account used for the connection has all the privileges listed above. If any privilege is missing, the system cannot archive break records.
- Open a site.
-
On the main toolbar, click
→
Settings.
The Settings page opens. -
In the tab pane, click Edge.
The Sites tab opens and shows a table with an overview of your sites. - In the site overview, click the name of a site.
The site page appears.
-
On the main toolbar, click
- Click the name of your connection in the connections list.
The Edit connection dialog box appears. - Click the Edit in the lower-right corner of the dialog box.
- Click + Add Property in the Property section.
- Enter __readOnly__ in the Name field.
- Enter false in the Value field.
- Click Save.
-
On the main toolbar, click
→ Data Quality.
Data Quality & Observability opens. - Click Connections.
The Connections page opens. - Open the connection with the recently added property.
- Click the Archive break records switch to enable archive break records.
- Select a break record archival location. Options include:
- Same location as job: The system creates break records in the job's schema. For example, If your job resides in a public schema, break records will be created in the same schema.
- Specify a location: Select a database and schema from the drop-down menu.
Note The database and schema must already exist in your connection.
- Create and run a job on this connection.
- Create a rule that will return breaking records.
- Re-run your job.
When breaking records occur, they will be archived to the location you specified in Step 11.