Data sources
A data source is an external system where your data, accounts, and groups reside, and from where the data is pulled into Data Access. An external system is also called the underlying data source. Your external system can be a database, reporting tool, or data warehouse such as BigQuery, Databricks, or Snowflake. It can also be an identity store that contains only accounts and groups (no data objects), such as Microsoft Entra ID or Okta. You can also have multiple data sources of the same type in Data Access, for example, multiple Snowflake accounts.
A data source in Data Access is an instance of your external system within Data Access. It allows Data Access to synchronize entities such as access controls, identities, and data objects.
Data Access does not automatically link a data source to an existing asset in Collibra. You can, however, manually link it to a System asset in Data Catalog.
Supported data sources
Data Access supports the following data sources:
- Data warehouses: BigQuery, Databricks, Snowflake
- Identity stores: Microsoft Entra ID, Okta
Data Access also includes the Collibra system as a data source. The Collibra data source is used to synchronize Collibrausers and groups.
How data sources work
Each data source includes metadata that describes its structure and behavior. This metadata helps Data Access understand:
- Which types of data objects exist within the data source, for example, folders, files, and tables.
- How access is managed, for example, whether the data source is Access Control List (ACL)-based or role-based.
- Which specific permissions can be set on the data objects.
- An explanation of what each permission does and whether it represents a Read, Write, or Admin permission.
- Which features the data source supports, such as column masking and row filtering, or whether it functions purely as an identity store.
Collibra as a data source
Data Access treats Collibra as a built-in data source, automatically including all users and groups from your Collibra environment. Because this data source is managed by Collibra, it is read-only in Data Access. You can still change its owners. To manage the users and groups within it, you can use the Collibra settings.
View a data source
To view a data source, on the Data Access landing page, in the left pane, click RESOURCES > Data sources, and then click the data source.
How data sources sync
In Data Access, synchronization is the process of synchronizing entities such as data objects, accounts, groups, and access controls. You set the synchronization schedule when you add a data source to Data Access. By default, Data Access runs a full synchronization daily at midnight.
What happens during a full sync
During a full data source synchronization:
- Data Access pulls all data objects, accounts, groups, and access controls from the underlying data source.
- Data Access pushes all access controls to the underlying data source.
After the first full synchronization, you can see within Data Access who has access to what. You can override the global schedule by setting a custom schedule for individual synchronization tasks. For example, you can configure specific synchronization tasks to run more frequently, or disable them completely. If you do not want to wait for the scheduled synchronization to see the data, you can manually start a synchronization.
How sync is triggered
Synchronization can be triggered in the following ways.
| Sync trigger | Description |
|---|---|
| Scheduled |
Automatically triggered based on the synchronization schedule that is set in the data source. |
| Manual | Manually triggered by clicking the Sync now button on the data source page. |
| Immediate |
Automatically triggered whenever an access control is created or updated in Data Access so that the new or updated access control is pushed to the underlying data source. |
Available sync tasks
The following synchronization tasks are available so that you can set granular schedules for specific entities, or manually start a targeted synchronization.
| Sync task | Description |
|---|---|
| Data object sync | Pulls all data objects (such as databases, schemas, tables, and columns) from the data source into Data Access. |
| Identity sync | Pulls all accounts from the data source into Data Access. Data Access automatically links the imported accounts to existing identities based on matching email addresses. |
| Access from target sync | Pulls all access controls and groups from the data source into Data Access. |
| Access to target sync | Pushes all access controls that are managed in Collibra from Data Access to the data source. |