Configure BigQuery permissions

Before setting up the BigQuery data source for Data Access, configure the underlying data source with the required permissions to allow Data Access to synchronize data objects, accounts, and access controls.

Tip To understand why and when a specific permission is required, go to BigQuery permissions.

In Data Access, first, you add a single data source at the GCP organization level that serves as an identities-only data source. That is, this GCP data source imports users and groups from GSuite and service accounts from GCP, but it does not import data objects or manage permissions. Then, you add a separate data source for each GCP project where you want to manage BigQuery access. You can either create a unique service account for each data source, or create one service account shared across all data sources. The following steps use the single shared service account approach.

1 Create a service account

Data Access requires a dedicated Google Cloud service account to securely authenticate and connect to your GCP and Google Workspace environments.

Steps

To create the service account and JSON key file:

  1. Create a service account in your preferred GCP project by using the Google Cloud Console or the command line.
    Copy
    gcloud iam service-accounts create collibra \
    --display-name="Service account for Collibra Data Access"
  2. Create and download a JSON key file for the service account.
    In the command, replace ${JSON_KEY_PATH} with the local path where you want to save the key file, and replace ${GCP_PROJECT_ID} with the ID of the project where you created the service account.
    Copy
    gcloud iam service-accounts keys create ${JSON_KEY_PATH} \
    --iam-account=collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com

2 Enable APIs

You need to enable the APIs that allow Data Access to interact with your Google resources.

Steps

To enable the APIs:

  1. In every GCP project that hosts the service account or contains the BigQuery assets that you want to manage, enable the APIs that are required by both GCP and BigQuery connectors.
    Copy
    gcloud services enable \
      cloudresourcemanager.googleapis.com \
      iam.googleapis.com \
      admin.googleapis.com \
      --project=${GCP_PROJECT_ID}
  2. Enable the BigQuery API for any project that contains BigQuery assets.
    Copy
    gcloud services enable \
      bigquery.googleapis.com \
      --project=${GCP_PROJECT_ID}
  3. If you plan to use column masking (controlled by the BigQuery catalog enabled option in Data Access), enable the additional APIs.
    Copy
    gcloud services enable \
      datacatalog.googleapis.com \
      bigquerydatapolicy.googleapis.com \
      --project=${GCP_PROJECT_ID}

3 Configure organization-wide permissions

Configuring organization-wide permissions is optional depending on which scoping approach you choose to limit what Data Access can access.

Note If you choose the Project scoping approach, skip the current section entirely, and then go to section 4 Configure Google Workspace admin role.

Scoping approaches

Choose the approach that matches how broadly you want Data Access to discover projects.

Scoping approach Description Permissions
Organization

Data Access automatically discovers all projects across your entire GCP organization.

This approach requires the full set of organization-level permissions to traverse your organization's folders and projects.

Choose this approach for comprehensive organization-wide management.

Organization-level permissions:

  • resourcemanager.organizations.get
  • resourcemanager.folders.list
  • resourcemanager.projects.list

Folder or project-level permissions: N/A

Folder

Specify which folders Data Access can manage by using the Include Paths field when adding the GCP data source to Data Access.

This approach requires only one organization-level permission. Permissions to traverse the hierarchy are applied only to the folders that you specify.

Choose this approach when you want to manage only specific folders.

Organization-level permissions: resourcemanager.organizations.get

Folder-level permissions:

  • resourcemanager.folders.list
  • resourcemanager.projects.list

Project-level permissions: iam.serviceAccounts.list

Project

Specify which projects Data Access can manage by using the GCP service accounts in identity sync projects field when adding the GCP data source to Data Access.

This approach does not require organization-level or folder-level permissions.

Choose this approach when you want to manage designated projects directly with minimal permissions.

Organization or folder-level permissions: N/A

Project-level permissions: iam.serviceAccounts.list

Steps: Organization scoping approach

To configure organization-wide permissions with the Organization scoping approach:

  1. Create a role-definition file named collibra-gcp-role.yaml .
    This role grants the minimum permissions that are required to fetch organization metadata, traverse folders and projects, and read service accounts.
    Copy
    title: "CollibraGcpRole"
    description: "Role for the Collibra Data Access GCP connector"
    stage: GA
    includedPermissions:
    - resourcemanager.organizations.get
    - resourcemanager.folders.list
    - resourcemanager.projects.list
    - iam.serviceAccounts.list
  2. Create a GCP role at the organization level.
    Copy
    gcloud iam roles create CollibraGcpRole \
    --organization=${GCP_ORGANIZATION_ID} \
    --file=collibra-gcp-role.yaml
  3. Assign the GCP role to the service account.
    Copy
    gcloud organizations add-iam-policy-binding ${GCP_ORGANIZATION_ID} \
    --member=serviceAccount:collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com \
    --role=organizations/${GCP_ORGANIZATION_ID}/roles/CollibraGcpRole

Steps: Folder scoping approach

To configure organization-wide permissions with the Folder scoping approach:

  1. Create a file named collibra-gcp-org-role.yaml for the organization-level role.
    This role grants only the single permission that is required to read the organization root.
    Copy
    title: "CollibraGcpOrgRole"
    description: "Organization-level role for the Collibra Data Access GCP connector (folder scope)"
    stage: GA
    includedPermissions:
    - resourcemanager.organizations.get
  2. Create a file named collibra-gcp-subtree-role.yaml for the folder-level role.
    This role grants the permissions that are required to traverse a folder sub-tree and read service accounts.
    Copy
    title: "CollibraGcpSubtreeRole"
    description: "Subtree role for the Collibra Data Access GCP connector (folder scope)"
    stage: GA
    includedPermissions:
    - resourcemanager.folders.list
    - resourcemanager.projects.list
    - iam.serviceAccounts.list
  3. Create both custom roles at the organization level so that they can be referenced from any resource in the organization.
    Copy
    gcloud iam roles create CollibraGcpOrgRole \
    --organization=${GCP_ORGANIZATION_ID} \
    --file=collibra-gcp-org-role.yaml

    gcloud iam roles create CollibraGcpSubtreeRole \
    --organization=${GCP_ORGANIZATION_ID} \
    --file=collibra-gcp-subtree-role.yaml
  4. Assign the organization-level role to the service account at the organization level.
    Copy
    gcloud organizations add-iam-policy-binding ${GCP_ORGANIZATION_ID} \
    --member=serviceAccount:collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com \
    --role=organizations/${GCP_ORGANIZATION_ID}/roles/CollibraGcpOrgRole
  5. Assign the sub-tree role to each target folder by repeating the following command once for each top-level folder that you specify in the Include Paths field when adding the BigQuery data source to Data Access.
    In the command, replace ${TARGET_FOLDER_ID} with the numeric folder ID. You do not need to re-bind the sub-tree role to nested folders or individual projects.
    Copy
    gcloud resource-manager folders add-iam-policy-binding ${TARGET_FOLDER_ID} \
    --member=serviceAccount:collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com \
    --role=organizations/${GCP_ORGANIZATION_ID}/roles/CollibraGcpSubtreeRole
  6. Verify that the service account has the expected role on the folder.
    Copy
    gcloud resource-manager folders get-iam-policy ${TARGET_FOLDER_ID} \
    --flatten="bindings[].members" \
    --filter="bindings.members:collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com" \
    --format="value(bindings.role)"

4 Configure Google Workspace admin role

Assigning a custom Google Workspace admin role directly to the service account allows Data Access to authenticate and call the Google Workspace Directory API. This is required to import users, groups, and group memberships into Data Access.

Steps

To configure the Google Workspace admin role:

  1. In the Google Workspace domain's admin console, click Account > Admin roles.
  2. Create an admin role with the Read users (users.read) and Read groups (groups.read) privileges.
    These privileges cover all three Directory API endpoints that the connector uses: users.list, groups.list, and members.list. The groups.read privilege authorizes all three of these calls.
  3. Open the new role, click Assign admin > Assign service accounts, and then enter the email address of the service account.
  4. Click Account > Account settings, and then record the customer ID, which begins with C.
    You need this ID for the GSuite Customer ID field when adding the GCP data source to Data Access.

5 Configure project-level permissions

For each GCP project that contains the BigQuery assets that you want to import into Data Access, you need to grant the required permissions to the service account. You can grant these BigQuery permissions at the organization level (to cover all projects at once) or at the individual project level (for a narrower scope).

Permission categories

When you configure project-level permissions, the first step is to create a CollibraBigQueryRole. The permissions in this role are grouped into the following categories so that you can grant only the permissions that you need based on the Data Access features that you intend to use.

Category Purpose
Always required Synchronize datasets, tables, and views as data objects, and import or export project-level and dataset-level Identity and Access Management (IAM) bindings.
Required if you use Data Access row filters

Apply row access policies to BigQuery tables from Collibra.

Tip Among the included permissions, bigquery.tables.getData might look unexpected. This permission is required only if you use Data Access row filters. Data Access row filters are implemented as CREATE OR REPLACE ROW ACCESS POLICY and DROP ROW ACCESS POLICY Data Definition Language (DDL) statements. BigQuery runs these statements as query jobs. Its query-job authorization model requires bigquery.tables.getData on every table that is referenced in the statement, even though no rows are read. This is a BigQuery-side requirement (not a Collibra-side requirement).

Despite the name, the bigquery.tables.getData permission does not read your table data. No code path in the connector reads row data, regardless of whether you grant this permission.

If you exclude this permission, the row filter feature stops working because the connector can no longer create, update, or remove BigQuery row access policies on your tables. If you do not use Data Access row filters, you can safely exclude this permission.
Required if you enable BigQuery catalog (column masking)

Apply column masks through Data Catalog policy tags and BigQuery data policies. This is controlled by the BigQuery catalog enabled option in Data Access, which is disabled by default.

Tip Column masking requires seven bigquery.dataPolicies.* permissions. This is because column masks are implemented as BigQuery data policies that are attached to Data Catalog policy tags, and managing them requires the full data policy life cycle, including IAM.
  • dataPolicies.create / delete / get / list / update: Create, list, modify, and remove the data policies that define masking rules, for example, ALWAYS_NULL or DEFAULT_MASKING_VALUE.
  • dataPolicies.getIamPolicy / setIamPolicy: Read and update the IAM bindings on each data policy to grant roles/bigquery.maskedReader to principals who should see masked data, and roles/datacatalog.categoryFineGrainedReader to principals who should see unmasked data.
These permissions apply only to the project's data policy resources; they do not grant access on your table data.
Required if you synchronize BigQuery usage analytics

Query INFORMATION_SCHEMA.JOBS to retrieve usage data.

Steps

To configure project-level permissions:

  1. Create a role-definition file named collibra-bigquery-role.yaml. Include only the sections that apply to your usage.
    Copy
    title: "CollibraBigQueryRole"
    description: "Role for the Collibra Data Access BigQuery connector"
    stage: GA
    includedPermissions:
      # Always required
      - bigquery.datasets.get
      - bigquery.datasets.getIamPolicy
      - bigquery.datasets.setIamPolicy
      - bigquery.datasets.update
      - bigquery.jobs.create
      - bigquery.jobs.get
      - bigquery.tables.get
      - bigquery.tables.getIamPolicy
      - bigquery.tables.list
      - bigquery.tables.setIamPolicy
      - resourcemanager.projects.get
      - resourcemanager.projects.list
      - resourcemanager.projects.getIamPolicy
      - resourcemanager.projects.setIamPolicy
      - iam.serviceAccounts.list   # Remove this line if you used Organization or Folder scoping approach

      # Required if you use Data Access row filters
      - bigquery.tables.getData
      - bigquery.rowAccessPolicies.create
      - bigquery.rowAccessPolicies.delete
      - bigquery.rowAccessPolicies.getIamPolicy
      - bigquery.rowAccessPolicies.list
      - bigquery.rowAccessPolicies.setIamPolicy
      - bigquery.rowAccessPolicies.update

      # Required if you enable BigQuery catalog (column masking)
      - bigquery.tables.setColumnDataPolicy
      - bigquery.tables.update
      - bigquery.tables.setCategory
      - bigquery.dataPolicies.create
      - bigquery.dataPolicies.delete
      - bigquery.dataPolicies.get
      - bigquery.dataPolicies.list
      - bigquery.dataPolicies.update
      - bigquery.dataPolicies.getIamPolicy
      - bigquery.dataPolicies.setIamPolicy

      # Required if you sync BigQuery usage analytics
      - bigquery.jobs.listAll
    Important When adding the BigQuery data source to Data Access, you can use the Disable project level permissions option to control whether Data Access includes or excludes project-level permissions. This option is disabled by default, meaning that Data Access includes project-level permissions.
    • To include project-level permissions: Keep the resourcemanager.projects.getIamPolicy and resourcemanager.projects.setIamPolicy permissions in the YAML file.
    • To exclude project-level permissions: Remove the resourcemanager.projects.getIamPolicy and resourcemanager.projects.setIamPolicy permissions from the YAML file.
  2. Create a BigQuery role at the organization level.
    Copy
    gcloud iam roles create CollibraBigQueryRole \
    --organization=${GCP_ORGANIZATION_ID} \
    --file=collibra-bigquery-role.yaml
  3. Assign the BigQuery role to the service account.
    The following command grants the service account access on BigQuery assets across all projects in the organization. If you want to limit access to a single project, replace the first line in the command with gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID}.
    Copy
    gcloud organizations add-iam-policy-binding ${GCP_ORGANIZATION_ID} \
    --member=serviceAccount:collibra@${GCP_PROJECT_ID}.iam.gserviceaccount.com \
    --role=organizations/${GCP_ORGANIZATION_ID}/roles/CollibraBigQueryRole

Related topics

BigQuery permissions

What's next

Create GCP connection