Add an Edge capability to an Edge site

Important

In Collibra 2024.05, we launched a new user interface (UI) for Collibra Data Intelligence Platform! You can learn more about this latest UI in the UI overview.

Use the following options to see the documentation in the latest UI or in the previous, classic UI:

Latest UI Classic UI

After you have created and installed an Edge site, you can add an Edge capability to perform specific tasks on a data source. For example, you can register a data source by using a JDBC connection that belongs to an Edge capability.

Prerequisites

You have a global role that has the System administration global permission.
You have a global role that has the Manage connections and capabilities global permission, for example, Edge integration engineer.
You have created and installed an Edge site.
You have created a JDBC connection.
Ensure the max cardinality of the asset attributes is at least 1.

Steps

Open an Edge site.
1. On the main toolbar, click , and then click Settings.
  The Collibra settings page opens.
2. In the tab pane, click Edge.
  The Sites tab opens and shows a table with an overview of the Edge sites.
3. In the table, click the name of the Edge site whose status is Healthy.
  The Edge site page opens.
In the Capabilities section, click Add capability.
The Add capability page is shown.
Select the EdgeCatalog Data ClassificationCatalog JDBC ingestionJDBC ProfilingCatalog JDBC SamplingS3 synchronizationGCS synchronizationDatabricks Unity Catalog synchronizationCatalog JDBC ingestionTechnical Lineage for dbtCollibra Protect for AWS Lake FormationCollibra Protect for BigQueryCollibra Protect for DatabricksCollibra Protect for SnowflakeGoogle Dataplex Catalog synchronization capability template you want to use.
Note When you select a capability template, you may need to add required custom properties. For example, if you select the S3 synchronization capability template, you have to add credentials to configure the S3 connection.

Enter the required information.

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `S3 synchronization`	Yes
S3 service account	This section contains information about how to connect to Amazon S3.
AWS Connection	The AWS connection to be used.	Yes
IAM role	The IAM role to be used by the AWS Glue crawlers.	Yes
Delete Glue database left after previous synchronization of the file system	Select the checkbox if you want the capability to delete the Glue databases created by previous runs of the capability, before the capability starts the synchronization. If you deselect this checkbox, the Glue databases created by previous runs are not removed. This can be useful for troubleshooting. By default, this checkbox is selected.	No
Save input metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. The Collibra Support team can provide the location of the saved ZIP files after the S3 synchronization. By default, this checkbox is not selected.	No
Finalization Strategy	Define what you want to do if an asset has been deleted from the S3 data source after an initial synchronization. The possible values are: Change Status (default): If an asset has been deleted from the S3 data source after an initial synchronization, we update the status of the asset in Collibra to "Missing from source". Remove Resources: If an asset has been deleted from the S3 data source after an initial synchronization, we remove the asset from Collibra. Ignore: If an asset has been deleted from the S3 data source after an initial synchronization, we don't change anything for the asset in Collibra.	Yes
Logging parameter	You can use this field to customize the debug logging. Important Only complete this field on request of or together with Collibra Support.	No
Custom parameter	Use this field to define that you want to ingest File Group assets as File assets. Name: file-group-as-file Type: Text Encryption: Not encrypted (Plain text) Value: true Type: Text Value Type: Plaintext Name: file-group-as-file Value: true	No
Glue database configuration Glue database configuration	Text in JSON format to define the Glue database names, regions, and domain IDs that you want to integrate. Tip Use this parameter if the current S3 synchronization crawler configuration doesn’t meet your needs. With this parameter, you can integrate an AWS Glue database for which you defined crawlers in AWS Glue itself. This allows you to use all crawler options from the AWS Glue Console. If you use this parameter, you don't need to create crawlers in Collibra. Important If you use this parameter, any crawlers you create in Collibra will not be taken into account during the S3 synchronization. You, however, will need to create a dummy crawler in Collibra to start the synchronization. A dummy crawler is a crawler with an invalid include path, such as s3://dummy. In a future release, we'll remove the need for a dummy crawler. The text must be in JSON format and can contain a block per database that you want to integrate. You can use any JSON validator to verify the format. Collibra is not responsible for the privacy, confidentiality, or protection of the data you submit to such JSON validators, and has no liability for such use. In a block, you can specify the Glue database name, region, and domain ID that must be ingested. The format is: `"glueDbName": “the name of the AWS Glue database”` `"glueDbRegion": “the region of the AWS Glue database”` `"dgcDomainId": “the domain ID in Collibra where assets of the AWS Glue database must be added”` If you don't add the domain ID, the assets are added in the same domain as the S3 File System asset. Example [ { "glueDbName": "integrations-auto-1", "glueDbRegion": "eu-west-1", "dgcDomainId": "a3fe0607-65af-43d6-bc2c-7c3adae6e162" }, { "glueDbName": "integrations-auto-2", "glueDbRegion": "eu-west-1" } ] In this example: Assets from the AWS Glue database "integrations-auto-1" will be ingested into the domain with ID "a3fe0607-65af-43d6-bc2c-7c3adae6e162". Assets from the AWS Glue database "integrations-auto-2" will be ingested into the same domain as the S3 File System asset.	No
Advanced Configuration Logging configuration Memory JVM arguments	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This setting is not valid for this integration. It should be set to false. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	An option to determine the verbosity of the log files. The default value is `No logging`.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Catalog Data Classification`	Yes
Connection	This section contains information to connect to the data source.
JDBC connection	The connection to the data source.	Yes
General	This section contains general information about logging.
Debug	An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Databricks Unity Catalog synchronization`	Yes
Databricks Connection
Databricks Connection	The Databricks connection to be used.	Yes
Configuration	This section contains information on how to connect to Databricks Unity Catalog.
Save input metadata	If you select this option the metadata extracted from the data source will be saved in a file that can be used for troubleshooting. Select this option only on request of Collibra Support.	No
Exclude Schemas (will be removed soon, use domain mapping instead)	Comma-separated list of the schemas that you don't want to integrate.	No
(deprecated) Filters and Domain Mapping	Important This field is deprecated in the latest UI. You can now define the mappings in the integration configuration. If you have existing mappings here, they will continue to work. However, we advise you to move them to the integration configuration. Text in JSON format to include or exclude databases and schemas, and to configure domain mappings. The text must be in JSON format and can contain an include and an exclude block. You can use any JSON validator to verify the format. Collibra is not responsible for the privacy, confidentiality, or protection of the data you submit to such JSON validators, and has no liability for such use. In the include block, you can specify the domain in which specific catalogs or schemas must be ingested. The format is: `“Catalog/Database > schema ”: “domain ID”`. For example, `"HR > address-schema": "30000000-0000-0000-0000-000000000000"`. In the exclude block, you can specify the catalogs or schemas that you don't want to ingest. For example, `"* > test"`. The exclude block has priority over the include block. If the include block is not present, we ingest all assets into the same domain as the System asset. If there is no explicit domain mapping for a schema, we use the domain specified for the database. You can use the keyword `default` as a domain ID. In that case, the catalog or schema will be ingested in the same domain as the System asset. A match with a database has priority over a match with a schema. The integration fails before the synchronization starts, if one or more domain IDs specified in the include block don't exist. The integration fails before the synchronization starts if a domain ID is left empty in the include block. You can use the ? and * wildcards in the catalog and schema names. If a catalog or schema matches multiple lines, the most detailed match is taken into account. Show example Example `{` `"include": {` `"HR": "20000000-0000-0000-0000-000000000000", "HR > address-schema": "30000000-0000-0000-0000-000000000000",` `"Orders > fk": "40000000-0000-0000-0000-000000000000", "Orders > ": "50000000-0000-0000-0000-000000000000", "* > profiling": "60000000-0000-0000-0000-000000000000", "sales": "default"` `}`, `"exclude": [` `"testDB", " * > information_schema"` `]` `}` In this example: The "HR" Database asset will be ingested into the domain with ID "20000000-0000-0000-0000-000000000000". Assets from the "HR" database will be ingested into the domain with ID "20000000-0000-0000-0000-000000000000". However, all assets from the "HR > address-schema" schema will be ingested into the domain with id "30000000-0000-0000-0000-000000000000". The "Orders" Database asset will be ingested in the same asset as the System asset. All assets from the "Orders” database with schemas starting with fk (fk*) will be ingested into the domain with ID "40000000-0000-0000-0000-000000000000", and all other assets from the "Orders” database will be ingested into the domain with ID "50000000-0000-0000-0000-000000000000". The "sales" Database asset and all assets from the "sales" database will be ingested in the same domain as the System asset. Assets from the "profiling" schema will be ingested into the domain with ID "60000000-0000-0000-0000-000000000000". However, the "profiling" schema in the database "Orders" will be ingested in the domain with ID "50000000-0000-0000-0000-000000000000" because a database match has priority over a schema match. All assets from the "testDB” database will be excluded. All assets from the “information_schema” schema in all databases will be excluded.	No
(deprecated) Extensible Properties Mapping	Via the Extensible Properties Mapping field, Databricks Unity Catalog allows you to add additional properties to Catalog, Schema, and Table objects. Important This field is deprecated in the latest UI. You can now define the mappings in the integration configuration. If you have existing mappings here, they will continue to work. However, we advise you to move them to the integration configuration. If you use this feature, make sure to set up all required characteristic assignments for the asset types. Three possible JSON formats are available. Version 0.1: This version allows you to ingest custom properties only. You can ingest the values from the Properties field from Catalog, Schema, and Table objects into specific attributes in Collibra assets. You do this by adding the mapping between the Properties fields for the objects in Databricks Unity Catalog and the Collibra attribute IDs to ingest the data in, using a JSON string. The text must be in JSON format and can contain a Catalogs, Schemas, and Tables block. The Catalogs block refers to Database assets, the Schemas block to Schema assets, and the Tables block to Table assets. In each block, you specify the property name and the attribute ID to which you want to map the value in the property. The format is: `"[property name]": "[attribute resource ID]"`. For example, `"Description from source system": "00000000-0000-0000-0001-000500000074"`. Example `{` `"catalogs": {` `"color": "00000000-0000-0000-0000-000000001234"`, `"Description from source system": "00000000-0000-0000-0001-000500000074"` `}`, `"schemas": {` `"File Location": "00000000-0000-0000-0001-000500000004"` `}`, `"tables": {` `"delta.lastCommitTimestamp": "00000000-0000-0000-0000-000000003114"` `}` `}` In this example: In the Database assets that we create, we'll add the Color value in attribute 00000000-0000-0000-0000-000000001234, and the Description from Source value in attribute 00000000-0000-0000-0001-000500000074. In the Schema assets that we create, we'll add the File Location value in attribute 00000000-0000-0000-0001-000500000004. In the Table assets that we create, we'll add the delta.lastCommitTimestamp value in attribute 00000000-0000-0000-0000-000000003114. Version 0.2: This version allows you to ingest both default system properties and custom properties. You can ingest most values from the Details page from Catalog, Schema, and Table objects into specific attributes in Collibra assets. You do this by adding the mapping between the fields for the objects in Databricks Unity Catalog and the Collibra attribute IDs to ingest the data in, using a JSON string. The text must be in JSON format. A Version block referencing 0.2 must be added. A Catalogs, Schemas, and Tables block can be added. The Catalogs block refers to Database assets, the Schemas block to Schema assets, and the Tables block to Table assets. Inside a Catalogs, Schemas, or Tables block, you can add a systemAttributes and a customParameters block. systemAttributes refers to the default system properties. customParameters refers to the custom properties. In each block, you specify the property name and the attribute ID to which you want to map the value in the property. The format is: `"[property name]": "[attribute resource ID]"`. For example, `"Description from source system": "00000000-0000-0000-0001-000500000074"`. Following system properties are supported: Catalogs: "browse_only", "catalog_type", "connection_name", "created_at", "created_by", "isolation_mode", "metastore_id", "provider_name", "provisioning_info", "securable_kind", "securable_type", "share_name", "storage_location", "storage_root", "updated_at" , and "updated_by". Schemas: "catalog_type", "created_at", "created_by", "metastore_id", "securable_type", "securable_kind", "storage_location", "storage_root", "updated_at", and "updated_by". Tables: "access_point", "created_at", "created_by", "data_access_configuration_id", "data_source_format", "deleted_at", "metastore_id", "securable_type", "securable_kind", "sql_path", "storage_credential_name", "storage_location", "table_type", "updated_at", "updated_by", and "view_definition". Tables mapping apply to tables and views. Example `{` `"version": 0.2,` `"catalogs": {` `"systemAttributes": {` `"metastore_id": "00000000-0000-0000-0000-000000004224"` `}`, `"customParameters": {` `"color": "00000000-0000-0000-0000-000000001234"`, `"Description from source system": "00000000-0000-0000-0001-000500000074"` `}` `}`, `"schemas": {` `"customParameters": {` `"File Location": "00000000-0000-0000-0001-000500000004"` `}` `}`, `"tables": {` `"systemAttributes": {` `"metastore_id": "00000000-0000-0000-0000-000000004224"` `}`, `"customParameters": {` `"delta.lastCommitTimestamp": "00000000-0000-0000-0000-000000003114"` `}` `}` `}` In this example: In the Database assets that we create, we'll add the metastore_id value in attribute "00000000-0000-0000-0000-000000004224", the Color value in attribute 00000000-0000-0000-0000-000000001234, and the Description from Source value in attribute 00000000-0000-0000-0001-000500000074. In the Schema assets that we create, we'll add the File Location value in attribute 00000000-0000-0000-0001-000500000004. In the Table and View assets that we create, we'll add the metastore_id value in attribute "00000000-0000-0000-0000-000000004224" and the delta.lastCommitTimestamp value in attribute 00000000-0000-0000-0000-000000003114. Version 0.3: This version allows you to ingest both default system properties and custom properties, and define separate decisions for tables and views. You can ingest most values from the Details page from Catalog, Schema, Table, and View objects into specific attributes in Collibra assets. You do this by adding the mapping between the fields for the objects in Databricks Unity Catalog and the Collibra attribute IDs to ingest the data in, using a JSON string. The text must be in JSON format. A Version block referencing 0.3 must be added. A Catalogs, Schemas, Tables, and Views block can be added. The Catalogs block refers to Database assets, the Schemas block to Schema assets, the Tables block to Table assets, and the Views block to Database View assets. Inside a Catalogs, Schemas, Tables, or Views block, you can add a systemAttributes and a customParameters block. systemAttributes refers to the default system properties. customParameters refers to the custom properties. In each block, you specify the property name and the attribute ID to which you want to map the value in the property. The format is: `"[property name]": "[attribute resource ID]"`. For example, `"Description from source system": "00000000-0000-0000-0001-000500000074"`. Following system properties are supported: Catalogs: "browse_only", "catalog_type", "connection_name", "created_at", "created_by", "isolation_mode", "metastore_id", "provider_name", "provisioning_info", "securable_kind", "securable_type", "share_name", "storage_location", "storage_root", "updated_at" , and "updated_by". Schemas: "catalog_type", "created_at", "created_by", "metastore_id", "securable_type", "securable_kind", "storage_location", "storage_root", "updated_at", and "updated_by". Tables: "access_point", "created_at", "created_by", "data_access_configuration_id", "data_source_format", "deleted_at", "metastore_id", "securable_type", "securable_kind", "sql_path", "storage_credential_name", "storage_location", "table_type", "updated_at", "updated_by", and "view_definition". Views: "access_point", "created_at", "created_by", "data_access_configuration_id", "data_source_format", "deleted_at", "metastore_id", "securable_type", "securable_kind", "sql_path", "storage_credential_name", "storage_location", "table_type", "updated_at", "updated_by", and "view_definition". Example `{` `"version": 0.3,` `"catalogs": {` `"systemAttributes": {` `"metastore_id": "00000000-0000-0000-0000-000000004224"` `}`, `"customParameters": {` `"color": "00000000-0000-0000-0000-000000001234"`, `"Description from source system": "00000000-0000-0000-0001-000500000074"` `}` `}`, `"schemas": {` `"customParameters": {` `"File Location": "00000000-0000-0000-0001-000500000004"` `}` `}`, `"tables": {` `"systemAttributes": {` `"metastore_id": "00000000-0000-0000-0000-000000004224"` `}`, `"customParameters": {` `"delta.lastCommitTimestamp": "00000000-0000-0000-0000-000000003114"` `}` `}` `"views": {` `"systemAttributes": {` `"metastore_id": "00000000-0000-0000-0000-000000004224"` `}`, `"customParameters": {` `"view.sqlConfig.spark.sql.session.timeZone": "018cedbf-37fc-7da3-9ea8-da2af754222e"` `}` `}` `}` In this example: In the Database assets that we create, we'll add the metastore_id value in attribute "00000000-0000-0000-0000-000000004224", the Color value in attribute 00000000-0000-0000-0000-000000001234, and the Description from Source value in attribute 00000000-0000-0000-0001-000500000074. In the Schema assets that we create, we'll add the File Location value in attribute 00000000-0000-0000-0001-000500000004. In the Table assets that we create, we'll add the metastore_id value in attribute "00000000-0000-0000-0000-000000004224" and the delta.lastCommitTimestamp value in attribute 00000000-0000-0000-0000-000000003114. In the Database View assets that we create, we'll add the metastore_id value in attribute "00000000-0000-0000-0000-000000004224" and the view.sqlConfig.spark.sql.session.timeZone value in attribute 018cedbf-37fc-7da3-9ea8-da2af754222e.	No
Compute Resource HTTP Path	The HTTP path of the compute resource in Databricks Unity Catalog that we can process to extract the source tags. You can find the HTTP path in the connection details of your cluster. For details, go to Get connection details for a cluster in Databricks documentation.	No
Advanced Configuration Logging configuration Memory JVM arguments	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This setting is not valid for this integration. It should be set to false.	No
Log level	This setting is not valid for this integration. It should be set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `ADLS synchronization`	Yes
ADLS service account	This section contains the information on how to connect to Azure Data Lake Storage.
Azure Connection	The ADLS connection to be used.	Yes
Synchronization Source	Choose which Microsoft data sources you want to integrate from.	No
Microsoft Purview Account Name	The name of your Microsoft Purview account. If you enter a Purview account name, the integration uses Microsoft Purview for the integration.	No
Save Input Metadata	If you select this option the metadata extracted from the data source will be saved in a file that can be used for troubleshooting. Select this option only on request of Collibra Support.	No
Max Schema Level	For columns that have a structured technical data type, Array or Struct, you can register the structure of the data. This is supported for AVRO, CSV, JSON, ORC, PARQUET, PSV, SSV, TSV, TXT, and XML. In this field, enter the maximum level of the structure you want to see. For example, 3. Note If you include a high number of levels, this can have an impact on the integration performance.	No
Advanced Configuration Logging configuration Memory JVM arguments	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This setting is not valid for this integration. It should be set to false. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This setting is not valid for this integration. It should be set to No logging. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Azure Connection	The Azure connection to be used.	Yes
Subscription ID	The ID of your Azure subscription.	Yes
Save Input Metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from Azure ML. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the Azure ML. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following capability template to ingest Collibra Data Quality & Observability user-defined rules, metrics, and dimensions into Collibra Data Catalog: `DQ Connector`	Yes
DQ	This section contains information about the Collibra Data Quality & Observability connection.
Base URL	Your Collibra Data Quality & Observability URL	Yes
Username	The Collibra Data Quality & Observability username for this connection.	Yes
Password	The Collibra Data Quality & Observability password for this connection.	Yes
Encryption options	Select the type of encryption to use. Default: To be encrypted by Edge management server.
Issuer of the JWT	If you have selected Encrypted with public key, enter your JWT issuer.	No
Collibra metadata model	This section contains information about where to ingest Collibra Data Quality & Observability assets.
DQ Rules domain id	The UUID of the Rulebook Domain for the ingested Collibra Data Quality & Observability rules.	Yes
DQ Metrics domain id	The UUID of the Rulebook Domain for the ingested Collibra Data Quality & Observability metrics.	Yes
DQ Dimensions domain id	The UUID of the Governance Asset Domain for the ingested Collibra Data Quality & Observability dimensions.	Yes
Default DQ Dimension name	The default Data Quality Dimension, for example Accuracy, Completeness, Consistency and so on. Default: Completeness.	Yes
DQ Metric classified by DQ Dimension relation type id	The UUID of the Data Quality Metric classified by / classifies Data Quality Dimension relation. If left unspecified, this relation will not be added.	No
Assets are imported in batches of this size	The batch size of the ingestion. Default: 5000.	Yes

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select your Edge capability template. Note When you select a capability template, you may need to add required custom properties. For example, if you select the S3 synchronization capability template, you have to add credentials to configure the S3 connection.	Yes
General	This section contains general information about logging.
Debug	An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `GCS synchronization`	Yes
GCP service account	This section contains information on how to connect to Google Cloud Storage.
GCP Connection	The GCP connection to be used.	Yes
Configuration	This section contains information on the configuration of the crawlers.
Maximum number of files per crawler	The maximum number of files that can be registered per crawler. The default value is 1,000.	Yes
Save input metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. The Collibra Support team can provide the location of the saved ZIP files after the synchronization. This checkbox is not selected by default.	No
Integrate Schemas from Dataplex	Select the checkbox if you want to integrate the schemas from Dataplex based on the crawler path that will be specified in the GCS integration configuration. If the checkbox is not selected, no Dataplex data will be ingested. This checkbox is selected by default.	No
Project IDs	Add a comma-separated list of the Project IDs where Dataplex is enabled. The capability will search in these projects for schemas based on the crawler path that will be specified in the GCS integration configuration. If the Project IDs field is empty, the integration will search in the project included in the provided GCP Service Account Credentials JSON.	No
Advanced Configuration Logging configuration Memory JVM arguments	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This setting is not valid for this integration. It should be set to false. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This setting is not valid for this integration. It should be set to No logging. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Catalog JDBC ingestion

Yes

Connection

This section contains information to connect to the data source.

JDBC connection

The connection to the data source.

Yes

JDBC data source type (Deprecated)

Deprecated field. The field was used to indicate the type of the data source. You no longer need to change this field. The required value is automatically identified.

Note The automatically identified value is not shown in this page.

Yes

Supports schemas

A text field where you have to enter True to enable database registration of data sources that have no schema. If the data source has schemas, you can ignore this field.

Tip If the data source does not have a schema, Data Catalog creates a Schema asset with the same name as the full name of the database.

Other Settings

Others

This section can contain additional capability properties.

Click Add propertyAdd Other Settings to add a property.

Show possible properties

Name	Description	Type	Encryption	Default value
Warning The following additional properties can have a significant impact on your Edge site. Only add or update them together with Collibra Support.
ingest-schema-garbage-collector	The garbage collector that is used by the capability. For information about other possibilities, see AZUL documentation.	Text	Not encrypted (plain text)	-XX:+UseParallelGC
ingest-schema-requests-cpu	The minimum amount of CPU computing power requested by the capability. The amount is expressed in milliCPU.	Text	Not encrypted (plain text)	100
ingest-schema-limits-cpu	The maximum amount of CPU computing power requested by the capability. The amount is expressed in milliCPU.	Text	Not encrypted (plain text)	950
ingest-schema-requests-memory	The minimum amount of memory requested by the capability. The amount is expressed in mebibytes (Mi). Important If you add this property, you also need to add the properties: ingest-schema-limits-memory and ingest-schema-jvm-max-memory.	Text	Not encrypted (plain text)	128
ingest-schema-limits-memory	The maximum amount of memory requested by the capability. The amount is expressed in mebibytes (Mi). Important If you add this property, you also need to add the properties: ingest-schema-requests-memory and ingest-schema-jvm-max-memory.	Text	Not encrypted (plain text)	2048
ingest-schema-jvm-max-memory	The maximum amount of memory that can be used by the Java virtual machine (jvm). The amount is expressed in mebibytes (Mi). Warning Make sure this amount is lower than the ingest-schema-limits-memory amount. Important If you add this property, you also need to add the properties: ingest-schema-requests-memory and ingest-schema-limits-memory.	Text	Not encrypted (plain text)	2048
http-connect-timeout-seconds	The maximum amount of time allowed to create a connection with Collibra. The value must be set to 30 or higher.	Text	Not encrypted (plain text)	30
http-read-timeout-seconds	The maximum amount of time allowed to wait for a response before closing the connection. The value must be set to 300 or higher.	Text	Not encrypted (plain text)	300

Note No validation is performed on the values you add.

General

This section contains general information about logging.

Debug

An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false.

Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.

For more information, go to logging.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

JDBC Profiling

Yes

Connection

This section contains information to connect to the data source.

JDBC connection

The connection to the data source.

Yes

Other Settings

Others

This section can contain additional capability properties.

Warning Adding additional properties can have a significant impact on your Edge site. Only add or update them together with Collibra Support.

Click Add propertyAdd Other Settings to add a property.
The possible properties are:

Show properties

Name	Description	Type	Encryption	Default value
garbage-collector	The garbage collector that is used by the capability. For information about other possibilities, see AZUL documentation.	Text	Not encrypted (plain text)	-XX:+UseParallelGC
requests-cpu	The minimum amount of CPU computing power requested by the capability. The amount is expressed in milliCPU.	Text	Not encrypted (plain text)	1000
limits-cpu	The maximum amount of CPU computing power requested by the capability. The amount is expressed in milliCPU.	Text	Not encrypted (plain text)	2000
requests-memory	The minimum amount of memory requested by the capability. The amount is expressed in mebibytes (Mi). Important If you add this property, you also need to add the properties: limits-memory and jvm-max-memory.	Text	Not encrypted (plain text)	2048
limits-memory	The maximum amount of memory requested by the capability. The amount is expressed in mebibytes (Mi). Important If you add this property, you also need to add the properties: requests-memory and jvm-max-memory.	Text	Not encrypted (plain text)	4096
jvm-max-memory	The maximum amount of memory that can be used by the Java virtual machine (jvm). The amount is expressed in mebibytes (Mi). Warning Make sure this amount is lower than the limits-memory amount. Important If you add this property, you also need to add the properties: requests-memory and limits-memory.	Text	Not encrypted (plain text)	3072
table-profiling-queries-timeout	The maximum amount of time, in the defined time unit, after which the query on the data source side stops.	Integer number	Not encrypted (plain text)	604800
table-profiling-queries-timeout-unit	The unit of time used to calculate the maximum amount of time for one query attempt. The possible values are: Nanos, Micros, Millis, Seconds, Minutes, Hours, and Days.	Text	Not encrypted (plain text)	604800
table-profiling-processing-timeout	The maximum amount of time, in the defined time unit, after which the data transfer and calculation of profiling statistics stops.	Integer number	Not encrypted (plain text)	604800
table-profiling-processing-timeout-unit	The unit of time used to calculate the maximum amount of time for one processing attempt. The possible values are: Nanos, Micros, Millis, Seconds, Minutes, Hours, and Days.	Text	Not encrypted (plain text)	Seconds
dgc-client-max-attempts	The maximum number of times the capability tries to send the profiling results to Collibra. The value must be between one and four. The first attempt also counts.	Integer number	Not encrypted (plain text)	2
dgc-client-read-timeout-amount	The maximum amount of time, in the defined time unit, that the capability tries to send the profiling results to Collibra.	Integer number	Not encrypted (plain text)	1800
dgc-client-read-timeout-unit	The unit of time used to calculate the maximum amount of time for one attempt. The possible values are: Nanos, Micros, Millis, Seconds, Minutes, and Hours.	Text	Not encrypted (plain text)	Seconds

Note No validation is performed on the values you add.

General

This section contains general information about logging.

Debug

An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false.

For more information, go to logging.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
AWS Connection	The AWS connection to be used.	Yes
Save Input Metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from Amazon SageMaker. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the Amazon SageMaker. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
AWS Connection	The AWS connection to be used.	Yes
Save Input Metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from Amazon Bedrock. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the Amazon Bedrock. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
SAP AI Core Connection	The SAP AI Core connection to be used.	Yes
Save Input Metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from SAP AI Core. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the SAP AI Core. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Catalog JDBC Sampling`	Yes
Connection	This section contains information to connect to the data source.
JDBC connection	The connection to the data source.	Yes
General	This section contains general information about logging.
Debug	An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h. For more information, go to logging.	No
Log level	An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Amazon Redshift

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

We strongly recommend that you not edit the full name of your System, Database and Schema assets in Data Catalog. Doing so can lead to errors during the technical lineage creation process. If stitching is missing specifically because you edited the full name of your Database asset, you can use this field to specify the current name of your Database asset in Data Catalog.

Queries

The queries to download all the data that is required to create technical lineage. The queries vary depending on the data source you use. The query code is automatically available. However, you can modify the query code if needed.

Example Enter the following filter in a Views query: where v.table_schema not in ('pg_catalog', 'information_schema');. This query excludes the pg_catalog and information_schema schemas, which don't contain customer data. If you want to exclude other schemas, adjust the query to, for example where v.table_schema not in ('pg_catalog', 'information_schema', 'another_schema');.

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Note On occasion, to improve performance, we update the default queries. When that happens, the next time your data sources are synchronized, the new default queries are checked against the previous default queries. If there is a difference, the new queries are used. However, Collibra Data Lineage can only check for changes between the new default queries and the previous set of default queries. If the queries in your Oracle Edge capability are older than the previous set of queries (or if you have customized them) they are recognized as customized queries and cannot be updated. Therefore, you won't benefit from the performance improvements.

To benefit from the performance improvements, you can create a new capability and copy the set of default queries from that into your existing capabilities. You can, then, modify them to suit your needs.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

This option allows you to provide table-definition details from an independent data source to a data source that is dependent on those details. This is needed to avoid analysis errors and to have a complete lineage that includes lineage from the SQL statements from dependent data sources.

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Technical lineage via Edge harvests raw metadata from specified data sources and uploads it in a ZIP file to a Collibra Data Lineage service instance. This option indicates whether the raw metadata should be deleted from the Collibra Data Lineage service instance after the metadata that is targeted for ingestion in Data Catalog is processed.

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

If you synchronize multiple data sources and don't select this option, each data source will be synchronized as a separate job. This can lead to failed sync jobs. For complete information, see Tips for successful lineage synchronization.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Harvest metadata from the data source and upload it to your Collibra environment. This allows you to inspect and, if necessary, edit the harvested metadata before uploading it to the Collibra Data Lineage service instance for analysis.

Analyze

Load and analyze the metadata on the Collibra Data Lineage service instance. Synchronization does not start after analysis; it starts only after you trigger the synchronization of another data source for which you specify the "Sync" value in this drop-down list.

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Important If you synchronize multiple data sources and don't select this option, each data source will be synchronized as a separate job. This can lead to failed sync jobs. For complete information, see Tips for successful lineage synchronization.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

An option to enable logging of a JDBC job. If you enable logging, you can download the output file of the JDBC job in the Edge Jobs dashboard (beta). The output file contains the logs of the JDBC driver. For more information about downloading the output file, go to Download job output files.

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Note The database and schema names in the SQL statements in your SQL files take precedence over the values that you provide for the Database and Schema fields in the technical lineage for SqlDirectory capability. If your SQL statements contain database and schema names, Collibra Data Lineage uses them for stitching. If your SQL statements do not contain database and schema names, Collibra Data Lineage uses the values of the Database and Schema fields in the capability for stitching. Fore more information, go to Prepare the SQL directory and Automatic stitching for technical lineage.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

This optional field allows you to map databases to their rightful systems, to obtain stitching. This resolves missing stitching, which occurs when Collibra Data Lineage associates multiple databases with the default system name that you provide in the Collibra System Name field.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Authentication Type

The authentication details for signing in to Azure Data Factory. You can select one of the following values:

Service Principal: When you select this authentication type, ensure that you entered the application secret for the Service Principal in the Service Principal Secret field when you created the Azure connection.
Resource Owner Password Credentials: When you select this authentication type, ensure that you specify the username field in this capability and also entered the password in the Service Principal Secret field when you created the Azure connection.

Yes

ADF Connection

The Azure connection that you created.

Yes

Username

The email address of your Azure Active Directory user.

This field applies only when you selected Resource Owner Password Credentials for the Authentication Type field.

Resource Group Name

The name of the resource group that the data factory belongs to.

Yes

Subscription ID

The subscription ID of the resource group.

Yes

Factories

The Azure Data Factory factories that Collibra Data Lineage collects and processes. Specify this property with an array of Azure Data Factory factory names. This property is optional.

The following rules apply when you specify this property:

Enter the factory names in square brackets ([ ]), enclose each factory name in double quotes (" "), and separate them by a comma, for example, ["MyFirstFactory", "MySecondFactory"].
The factory name is not case-sensitive. For example, the MyFactory and myfactory factories are considered the same by Azure Data Factory and Collibra Data Lineage.
If you do not specify any factory name, Collibra Data Lineage collects and processes all factories that have datasets and piplelines in them.

Source Configuration

The source configuration for database mapping, system mapping, schema mapping, and filtering. Specify the following properties in JSON format and enter the content in this field.

If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the <sourceId>.conf file in this field.

Connection definition properties

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name> | found_dbname=<datafactory_name>_<linkedservice_name>;found_hostname=*

The information of the supported data sources in Azure Data Factory to be collected by Collibra Data Lineage. You can specify any of the following values for the found_dbname property:

A database name. And then you can specify the following properties:
- found_hostname=<server name>, where <server name> is the name of the server that the database is running on.
- found_schema=<schema name>, where <schema name> is the name of the schema. This property is optional.

The combination of <datafactory_name>_<linkedservice_name>, where <datafactory_name> is a data factory name and <linkedservice_name> is a linked service name. If you use this combination, specify * for the found_hostname property.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The name of the database asset in Data Catalog. Specify this property with the database name that you created when you registered the data source.

schema

The name of the schema asset in Data Catalog. Specify this property with the schema name that you created when you registered the data source.

If the Collibra Data Lineage fails to find the schema that you specify, it uses the default schema.

dialect

If you specify a database name for the found_dbname property, select one of the following dialects. If you specify a linked service name for the found_dbname property, ignore this property.

collibraSystemName

The system or server name of the data source.

Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source.

Specify this property with the same name as the name of the System asset that you created when you registered the data source.

If you don't specify a value for this property, DEFAULT is shown in the technical lineage.

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for ADF

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Authentication Type

The authentication details for signing in to Azure Data Factory. You can select one of the following values:

Service Principal: When you select this authentication type, ensure that you entered the application secret for the Service Principal in the Service Principal Secret field when you created the Azure connection.
Resource Owner Password Credentials: When you select this authentication type, ensure that you specify the username field in this capability and also entered the password in the Service Principal Secret field when you created the Azure connection.

Yes

ADF Connection

The Azure connection that you created.

Yes

Username

The email address of your Azure Active Directory user.

This field applies only when you selected Resource Owner Password Credentials for the Authentication Type field.

Resource Group Name

The name of the resource group that the data factory belongs to.

Yes

Subscription ID

The subscription ID of the resource group.

Yes

Factories

The Azure Data Factory factories that Collibra Data Lineage collects and processes. Specify this property with an array of Azure Data Factory factory names. This property is optional.

The following rules apply when you specify this property:

Enter the factory names in square brackets ([ ]), enclose each factory name in double quotes (" "), and separate them by a comma, for example, ["MyFirstFactory", "MySecondFactory"].
The factory name is not case-sensitive. For example, the MyFactory and myfactory factories are considered the same by Azure Data Factory and Collibra Data Lineage.
If you do not specify any factory name, Collibra Data Lineage collects and processes all factories that have datasets and piplelines in them.

Source Configuration

The source configuration for database mapping, system mapping, schema mapping, and filtering. Specify the following properties in JSON format and enter the content in this field.

If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the connection_definitions.conf file in this field.

Connection definition properties

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name> | found_dbname=<datafactory_name>_<linkedservice_name>;found_hostname=*

The information of the supported data sources in Azure Data Factory to be collected by Collibra Data Lineage. You can specify any of the following values for the found_dbname property:

A database name. And then you can specify the following properties:
- found_hostname=<server name>, where <server name> is the name of the server that the database is running on.
- found_schema=<schema name>, where <schema name> is the name of the schema. This property is optional.

The combination of <datafactory_name>_<linkedservice_name>, where <datafactory_name> is a data factory name and <linkedservice_name> is a linked service name. If you use this combination, specify * for the found_hostname property.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The name of the database asset in Data Catalog. Specify this property with the database name that you created when you registered the data source.

schema

The name of the schema asset in Data Catalog. Specify this property with the schema name that you created when you registered the data source.

If the Collibra Data Lineage fails to find the schema that you specify, it uses the default schema.

dialect

If you specify a database name for the found_dbname property, select one of the following dialects. If you specify a linked service name for the found_dbname property, ignore this property.

collibraSystemName

The system or server name of the data source.

Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source.

Specify this property with the same name as the name of the System asset that you created when you registered the data source.

If you don't specify a value for this property, DEFAULT is shown in the technical lineage.

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

GCP Connection

The GCP connection that you created.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Save Input Metadata

Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.

Logging configuration, Memory (MiB), and JVM arguments

These fields contain configuration options that can help when investigating issues with the capability.

Important Only complete these fields on request of or together with Collibra Support.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Azure

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Azure

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Azure

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Database Name Override

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Custom Technical Lineage

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Databricks Connection

The Databricks connection that you created.

Yes

Compute Resource HTTP Path

The HTTP path of the compute resource in Databricks Unity Catalog that Collibra Data Lineage collects and processes to create technical lineage.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Time Frame

Specify the duration for data collection. The default value is 365, which means that Collibra Data Lineage collects the data of the past 365 days.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Save Input Metadata

Filters

Use this section to include or exclude databases and schemas to be ingested. Enter the filters in JSON format. If you used filters when you integrated Databricks Unity Catalog, you can enter in this field the content from the Filters and Domain Mapping field in the Databricks Unity Catalog capability. Noted that Collibra Data Lineage ignores the UUIDs that are specified in the content.

Text in JSON format to include or exclude databases and schemas, and to configure domain mappings.

The text must be in JSON format and can contain an include and an exclude block. You can use any JSON validator to verify the format. Collibra is not responsible for the privacy, confidentiality, or protection of the data you submit to such JSON validators, and has no liability for such use.
In the include block, you can specify the domain in which specific catalogs or schemas must be ingested. The format is: “Catalog/Database > schema ”: “domain ID”. For example, "HR > address-schema": "30000000-0000-0000-0000-000000000000".
In the exclude block, you can specify the catalogs or schemas that you don't want to ingest. For example, "* > test".
The exclude block has priority over the include block.
If the include block is not present, we ingest all assets into the same domain as the System asset.
If there is no explicit domain mapping for a schema, we use the domain specified for the database.
You can use the keyword default as a domain ID. In that case, the catalog or schema will be ingested in the same domain as the System asset.
A match with a database has priority over a match with a schema.
The integration fails before the synchronization starts, if one or more domain IDs specified in the include block don't exist.
The integration fails before the synchronization starts if a domain ID is left empty in the include block.
You can use the ? and * wildcards in the catalog and schema names. If a catalog or schema matches multiple lines, the most detailed match is taken into account.

Logging configuration, Memory (MiB), and JVM arguments

These fields contain configuration options that can help when investigating issues with the capability.

Important Only complete these fields on request of or together with Collibra Support.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Databricks Unity Catalog

Yes

Main Properties

This section contains the information for creating a technical lineage.

Databricks Connection

The Databricks connection that you created.

Yes

Compute Resource HTTP Path

The HTTP path of the compute resource in Databricks Unity Catalog that Collibra Data Lineage collects and processes to create technical lineage.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Time Frame

Specify the duration for data collection. The default value is 365, which means that Collibra Data Lineage collects the data of the past 365 days.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Save Input Metadata

Filters

Text in JSON format to include or exclude databases and schemas, and to configure domain mappings.

The text must be in JSON format and can contain an include and an exclude block. You can use any JSON validator to verify the format. Collibra is not responsible for the privacy, confidentiality, or protection of the data you submit to such JSON validators, and has no liability for such use.
In the include block, you can specify the domain in which specific catalogs or schemas must be ingested. The format is: “Catalog/Database > schema ”: “domain ID”. For example, "HR > address-schema": "30000000-0000-0000-0000-000000000000".
In the exclude block, you can specify the catalogs or schemas that you don't want to ingest. For example, "* > test".
The exclude block has priority over the include block.
If the include block is not present, we ingest all assets into the same domain as the System asset.
If there is no explicit domain mapping for a schema, we use the domain specified for the database.
You can use the keyword default as a domain ID. In that case, the catalog or schema will be ingested in the same domain as the System asset.
A match with a database has priority over a match with a schema.
The integration fails before the synchronization starts, if one or more domain IDs specified in the include block don't exist.
The integration fails before the synchronization starts if a domain ID is left empty in the include block.
You can use the ? and * wildcards in the catalog and schema names. If a catalog or schema matches multiple lines, the most detailed match is taken into account.

Logging configuration, Memory (MiB), and JVM arguments

These fields contain configuration options that can help when investigating issues with the capability.

Important Only complete these fields on request of or together with Collibra Support.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions, where you specify relevant translations for each data source. Specify the following properties in JSON format and enter the content in this field.

If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the <sourceId>.conf file in this field.

Connection definition properties

Property	Description
OdbcDataSources	Open Database Connectivity data sources in IBM InfoSphere DataStage for which you want to create a technical lineage.
<data-source-name>	The ODBC data source name that you use in your DataStage projects. This section contains the properties to translate the database, schema and dialect.
dbname	The name of your database, to which the ODBC data source connection refers.
schema	The name of your schema, to which the ODBC data source connection refers.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
NonOdbcConnectors	Other data source connectors in IBM InfoSphere DataStage for which you want to create a technical lineage. For example, DB2, Oracle or Netezza. Note This section is optional.
<data-source-connector-ID>	The data source username and database of the connector that you use in your DataStage projects. This usually looks like for example admin@database-name. The combination of the username and database name should be unique. The following section contains the properties to translate the database, schema and dialect.
dbname	The name of your database, to which the data source connection refers.
schema	The name of your schema, to which the data source connection refers.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
Jobs	The jobs that you want technical lineage via Edge to collect and process to create the technical lineage. This section is optional. The following rules apply when you specify this section: Specify jobs that are executed so that the technical lineage graph does not include any job parameters with undefined values. Specify only the first and parent jobs in a sequence of executed jobs. Technical lineage via Edge automatically collects all jobs that are called by the parent jobs. For details about how CollibraData Lineage parses DataStage jobs and resolves parameters, see Transformation logic and common errors for DataStage.
JobParameters	The runtime parameters that are not in the DSX and ENV files. You can specify multiple job parameters.
name	The name of the job parameter. You can specify any of the following values: A parameter name A user variable A parameter set Important Do not enclose the name between "#" characters, for example `"name": "#name#"`
value	The value of the job parameter. You can specify one of the following values, depending on the value of the `name` property: If a parameter name is specified for the `name` property, specify one of the following values: A parameter value A parameter reference If a user variable is specified for the `name` property, specify one of the following values: A parameter value A parameter set reference If a parameter set is specified for the `name` property, specify this property with a value file name. For details about how the values are resolved, see the Parameter resolution section in Transformation logic and common errors for DataStage.
perJobParameters	The parameters of a specific job. For example, you ingest multiple jobs where the parameters have the same name, but different values. Note This value takes precedence over the values specified in the JobParameters property. Otherwise, the original jobParameters field is used as the “default” option.
jobID	The ID of the job.
name	The name of the job parameter. You can specify any of the following values: A parameter name A user variable A parameter set Important Do not enclose the name between "#" characters, for example `"name": "#name#"`
value	The value of the job parameter. You can specify one of the following values, depending on the value of the `name` property: If a parameter name is specified for the `name` property, specify one of the following values: A parameter value A parameter reference If a user variable is specified for the `name` property, specify one of the following values: A parameter value A parameter set reference If a parameter set is specified for the `name` property, specify a value file name as the value. For details about how the values are resolved, see the Parameter resolution section in Transformation logic and common errors for DataStage.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for DataStage

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions, where you specify relevant translations for each data source. Specify the following properties in JSON format and enter the content in this field.

Connection definition properties

Property	Description
OdbcDataSources	Open Database Connectivity data sources in IBM InfoSphere DataStage for which you want to create a technical lineage.
<data-source-name>	The ODBC data source name that you use in your DataStage projects. This section contains the properties to translate the database, schema and dialect.
dbname	The name of your database, to which the ODBC data source connection refers.
schema	The name of your schema, to which the ODBC data source connection refers.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
NonOdbcConnectors	Other data source connectors in IBM InfoSphere DataStage for which you want to create a technical lineage. For example, DB2, Oracle or Netezza. Note This section is optional.
<data-source-connector-ID>	The data source username and database of the connector that you use in your DataStage projects. This usually looks like for example admin@database-name. The combination of the username and database name should be unique. The following section contains the properties to translate the database, schema and dialect.
dbname	The name of your database, to which the data source connection refers.
schema	The name of your schema, to which the data source connection refers.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
Jobs	The jobs that you want technical lineage via Edge to collect and process to create the technical lineage. This section is optional. The following rules apply when you specify this section: Specify jobs that are executed so that the technical lineage graph does not include any job parameters with undefined values. Specify only the first and parent jobs in a sequence of executed jobs. Technical lineage via Edge automatically collects all jobs that are called by the parent jobs. For details about how CollibraData Lineage parses DataStage jobs and resolves parameters, see Transformation logic and common errors for DataStage.
JobParameters	The runtime parameters that are not in the DSX and ENV files. You can specify multiple job parameters.
name	The name of the job parameter. You can specify any of the following values: A parameter name A user variable A parameter set Important Do not enclose the name between "#" characters, for example `"name": "#name#"`
value	The value of the job parameter. You can specify one of the following values, depending on the value of the `name` property: If a parameter name is specified for the `name` property, specify one of the following values: A parameter value A parameter reference If a user variable is specified for the `name` property, specify one of the following values: A parameter value A parameter set reference If a parameter set is specified for the `name` property, specify this property with a value file name. For details about how the values are resolved, see the Parameter resolution section in Transformation logic and common errors for DataStage.
perJobParameters	The parameters of a specific job. For example, you ingest multiple jobs where the parameters have the same name, but different values. Note This value takes precedence over the values specified in the JobParameters property. Otherwise, the original jobParameters field is used as the “default” option.
jobID	The ID of the job.
name	The name of the job parameter. You can specify any of the following values: A parameter name A user variable A parameter set Important Do not enclose the name between "#" characters, for example `"name": "#name#"`
value	The value of the job parameter. You can specify one of the following values, depending on the value of the `name` property: If a parameter name is specified for the `name` property, specify one of the following values: A parameter value A parameter reference If a user variable is specified for the `name` property, specify one of the following values: A parameter value A parameter set reference If a parameter set is specified for the `name` property, specify a value file name as the value. For details about how the values are resolved, see the Parameter resolution section in Transformation logic and common errors for DataStage.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Db2

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

dbt Connection

The dbt connection that you created.

Yes

Environment Ids

The IDs of the environments that Collibra Data Lineage uses to download job artifacts.

Enter an array of environment IDs, for example [123456, 987654]. This field is required if you do not enter a value for the Admin URL field in the dbt connection.

If you enter values for both the Admin URL and Environment Ids fields, the Environment Ids field takes precedence.

Source Configuration

The source configuration to reduce the amount of data objects to be downloaded and enhance the performance of CollibraData Lineage in the following ways:

Filter the projects and jobs to be downloaded. Include projects and jobs to be downloaded by specifying the filter property.
Specify different Collibra system names for different projects by specifying the collibraSystemNames property .
Map a materialization as a view instead of a table by specifying the materializedMapping property.

Specify the following properties in JSON format and enter the content in this field.

Connection definition properties

Property	Description	Required?
collibraSystemNames	You can use this section to specify the Collibra System Name for each project.	No
projects	This section contains the project names and the Collibra system names.	No
project_id	Your project ID. You can find the project ID in the dbt URL right after `projects`. For example, if your dbt URL is `https://cloud.getdbt.com/develop/54321/projects/12345` , your project_id is `12345`.	No
collibraSystemName	The system or server name of the data source. This is also the name of your System asset in Data Catalog: Specify this property with the same name as the name of the System asset that you created when you registered the data source. See an example. In this code example, the project with the `12345` project ID is stitched to the `systemname1` System asset in Data Catalog. { "collibraSystemNames":{ "projects":[ {"project_id":"12345","collibraSystemName":"systemname1"} ] }, }	No
filter	You can use this section to include projects and jobs to be downloaded. Collibra Data Lineage downloads and processes only the specified jobs and projects. See an example. In this code example, the job with the 1234 job ID and the projects with the 98 and 5678 project IDs are downloaded. { "filter": { "jobIds": [ 1234 ], "projectIds": [ 98, 5678 ] } }	No
jobIds	The job IDs of the jobs that you want to include. Specify an integer. Do not specify a string. To get your job ID, in your dbt, select Deploy and then Jobs. Select a job and you can find your job ID in the URL. For example, if your URL is `cloud.getdbt.com/deploy/65432/projects/23456/jobs/123456`, `123456` is your job ID.	No
projectIds	The project IDs of the projects that you want to include. Specify an integer. Do not specify a string. You can find the project ID in the dbt URL right after `projects`. For example, if your dbt URL is `https://cloud.getdbt.com/develop/54321/projects/12345` , your project_id is `12345`.	No
materializedMapping	Indicates how materializations in dbt are mapped. If you do not specify this property, CollibraData Lineage maps materializations to tables by default. You can change the mapping of a materialization to view. In the following example, the ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES materialization is mapped to a view. "materializedMapping":{ "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW" }	No

Tip If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the <sourceId>.conf file in this field.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for dbt Cloud

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

dbt Connection

The dbt connection that you created.

Yes

Environment Ids

The IDs of the environments that Collibra Data Lineage uses to download job artifacts.

Enter an array of environment IDs, for example [123456, 987654]. This field is required if you do not enter a value for the Admin URL field in the dbt connection.

If you enter values for both the Admin URL and Environment Ids fields, the Environment Ids field takes precedence.

Source Configuration

The source configuration to reduce the amount of data objects to be downloaded and enhance the performance of Collibra Data Lineage in the following ways:

Filter the projects and jobs to be downloaded. Include projects and jobs to be downloaded by specifying the filter property.
Specify different Collibra system names for different projects by specifying the collibraSystemNames property .
Map a materialization as a view instead of a table by specifying the materializedMapping property.

Specify the following properties in JSON format and enter the content in this field.

Connection definition properties

Property	Description	Required?
collibraSystemNames	You can use this section to specify the Collibra System Name for each project.	No
projects	This section contains the project names and the Collibra system names.	No
project_id	Your project ID. You can find the project ID in the dbt URL right after `projects`. For example, if your dbt URL is `https://cloud.getdbt.com/develop/54321/projects/12345` , your project_id is `12345`.	No
collibraSystemName	The system or server name of the data source. This is also the name of your System asset in Data Catalog: Specify this property with the same name as the name of the System asset that you created when you registered the data source. See an example. In this code example, the project with the `12345` project ID is stitched to the `systemname1` System asset in Data Catalog. { "collibraSystemNames":{ "projects":[ {"project_id":"12345","collibraSystemName":"systemname1"} ] }, }	No
filter	You can use this section to include projects and jobs to be downloaded. Collibra Data Lineage downloads and processes only the specified jobs and projects. See an example. In this code example, the job with the 1234 job ID and the projects with the 98 and 5678 project IDs are downloaded. { "filter": { "jobIds": [ 1234 ], "projectIds": [ 98, 5678 ] } }	No
jobIds	The job IDs of the jobs that you want to include. Specify an integer. Do not specify a string. To get your job ID, in your dbt, select Deploy and then Jobs. Select a job and you can find your job ID in the URL. For example, if your URL is `cloud.getdbt.com/deploy/65432/projects/23456/jobs/123456`, `123456` is your job ID.	No
projectIds	The project IDs of the projects that you want to include. Specify an integer. Do not specify a string. You can find the project ID in the dbt URL right after `projects`. For example, if your dbt URL is `https://cloud.getdbt.com/develop/54321/projects/12345` , your project_id is `12345`.	No
materializedMapping	Indicates how materializations in dbt are mapped. If you do not specify this property, CollibraData Lineage maps materializations to tables by default. You can change the mapping of a materialization to view. In the following example, the ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES materialization is mapped to a view. "materializedMapping":{ "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW" }	No

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The source configuration to reduce the amount of data objects to be processed and enhance the performance of Collibra Data Lineage.

Specify the following properties in JSON format and enter the content in this field.

Connection definition properties

Property	Description	Required?
Collibra System Name	The system or server name of the data source. This field is also the full name of your System asset in Data Catalog. The value of this field must be the same as the full name of the System asset that you created when you registered the data source.	No
projects	This section contains the Collibra system names.	No
collibraSystemName	The system or server name of the data source. This is also the name of your System asset in Data Catalog: Specify this property with the same name as the name of the System asset that you created when you registered the data source. See an example. In this code example, the project is stitched to the `systemname1` System asset in Data Catalog. { "collibraSystemNames":{ "projects":[ {"collibraSystemName":"systemname1"} ] }, }	No
materializedMapping	Indicates how materializations in dbt are mapped. If you do not specify this property, CollibraData Lineage maps materializations to tables by default. You can change the mapping of a materialization to view. In the following example, the ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES materialization is mapped to a view. "materializedMapping":{ "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW" }	No

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for dbt

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The source configuration to reduce the amount of data objects to be processed and enhance the performance of Collibra Data Lineage.

Specify the following properties in JSON format and enter the content in this field.

Connection definition properties

Property	Description	Required?
Collibra System Name	The system or server name of the data source. This field is also the full name of your System asset in Data Catalog. The value of this field must be the same as the full name of the System asset that you created when you registered the data source.	No
projects	This section contains the Collibra system names.	No
collibraSystemName	The system or server name of the data source. This is also the name of your System asset in Data Catalog: Specify this property with the same name as the name of the System asset that you created when you registered the data source. See an example. In this code example, the project is stitched to the `systemname1` System asset in Data Catalog. { "collibraSystemNames":{ "projects":[ {"collibraSystemName":"systemname1"} ] }, }	No
materializedMapping	Indicates how materializations in dbt are mapped. If you do not specify this property, CollibraData Lineage maps materializations to tables by default. You can change the mapping of a materialization to view. In the following example, the ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES materialization is mapped to a view. "materializedMapping":{ "ELS_MATERIALIZE_MULTIPLE_EXTERNAL_TABLES":"VIEW" }	No

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for BigQuery

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Billing ID

Important This field is currently optional. In a future version of Collibra it will become mandatory.

The billing ID is a JDBC connection parameter that is required to execute the SQL statements to harvest the metadata. Enter the project ID of any single project for which you want to harvest metadata.

Tip You can then use the Project ID field to specify all of the other projects from which you want to harvest metadata.

Project ID

Use this field to specify (by project ID) the project or projects from which you want to harvest metadata. Leave this field empty if you want to harvest the metadata from all projects for which the service account has permissions.

Tip Each field can only contain a single project ID. To list mutiple project IDs, click Add property, and then add the next project ID.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Columns Tail	This query retrieves all columns tails.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Billing ID

Important This field is currently optional. In a future version of Collibra it will become mandatory.

Tip You can then use the Project ID field to specify all of the other projects from which you want to harvest metadata.

Project ID

Tip Each field can only contain a single project ID. To list mutiple project IDs, click Add property, and then add the next project ID.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Columns Tail	This query retrieves all columns tails.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Greenplum

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Hive

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

The database value to be used as the database name in the full path (system -> database -> schema -> table). Use this field to ensure successful stitching for a database-less data source. You can specify one of the following values:

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Hive) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Object Names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Hive) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Object Names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

IICS connection

The Informatica Intelligent Cloud Services (IICS) connection that you created.

Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Informatica Intelligent Cloud Services (IICS) connection.

Yes

Objects

The objects that you want to retrieve.

Each object requires a path and a type as shown in the following example, where,

path

The path to the object, which is relative to the Explore directory in IICS, for example, Sales.

type

The type of the object, for example, Taskflow.

IICS scanner's starting point is a Taskflow or Linear Taskflow (Workflow). Therefore the only meaningful types to retrieve are: Taskflow, Workflow, Project and Folder.

The types are not case sensitive.

Tip For more information about the objects that you can retrieve and the required information, go to the Informatica documentation.

Yes

Parameter Files

Upload a ZIP file that contains Informatica Intelligent Cloud Services parameter files. You can name the ZIP file as you prefer. Ensure that the ZIP file contains all parameter files that you want CollibraData Lineage to collect.

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

Connection definitions

Property	Description	Required?
collibraSystemNames	This section contains the system information for Informatica Intelligent Cloud Services.
connections	This section contains the system connection information. This is required to reference to the system or server of the connection.
connectionName	The name of the connection. The name must match the System asset name in Data Catalog for stitching.	Yes
collibraSystemName	The system or server name of the data source. Use this property with the `useCollibraSystemName` property in the lineage harvester configuration file to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you create when you prepare the physical data layer in Data Catalog. If you don't prepare the physical data layer, Collibra Data Lineage cannot stitch the data objects in your technical lineage to the assets in Data Catalog. Specify this property with the same name as the name of the System asset that you created when you registered the data source.	No
connectionDefinitions	This section contains the database, schema and dialect information for each connection in Informatica Intelligent Cloud Services. Note You can add connection information for each connection in the `connections` section.
connectionName	The name of the connection. The name must match with the name in a connection name in the `connections` section. This property is required.	Yes
databaseName	The name of your database. The name must match the Database asset name in Data Catalog for stitching.	Yes
schemaName	The name of your schema. The name must match the Schema asset name in Data Catalog for stitching.	Yes
dialect	The dialect of the connection. Specify this property for Collibra Data Lineage to properly extract and parse queries that are related to this connection. You can enter one of the following values: `bigquery` `db2` `hana` `hive` `greenplum` `mssql` `mysql` `netezza` `oracle` `postgres` `redshift` `snowflake` `spark` `teradata`	No

Tip If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the source ID configuration file in this field.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for Informatica Intelligent Cloud Services (IICS)

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

IICS connection

The Informatica Intelligent Cloud Services (IICS) connection that you created.

Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Informatica Intelligent Cloud Services (IICS) connection.

Objects

The objects that you want to retrieve.

Each object requires a path and a type as shown in the following example, where,

path

The path to the object, which is relative to the Explore directory in IICS, for example, Sales.

type

The type of the object, for example, Taskflow.

IICS scanner's starting point is a Taskflow or Linear Taskflow (Workflow). Therefore the only meaningful types to retrieve are: Taskflow, Workflow, Project and Folder.

The types are not case sensitive.

Tip For more information about the objects that you can retrieve and the required information, go to the Informatica documentation.

Yes

Parameter Files

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

Connection definitions

Property	Description	Required?
collibraSystemNames	This section contains the system information for Informatica Intelligent Cloud Services.
connections	This section contains the system connection information. This is required to reference to the system or server of the connection.
connectionName	The name of the connection. The name must match the System asset name in Data Catalog for stitching.	Yes
collibraSystemName	The system or server name of the data source. Use this property with the `useCollibraSystemName` property in the lineage harvester configuration file to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you create when you prepare the physical data layer in Data Catalog. If you don't prepare the physical data layer, Collibra Data Lineage cannot stitch the data objects in your technical lineage to the assets in Data Catalog. Specify this property with the same name as the name of the System asset that you created when you registered the data source.	No
connectionDefinitions	This section contains the database, schema and dialect information for each connection in Informatica Intelligent Cloud Services. Note You can add connection information for each connection in the `connections` section.
connectionName	The name of the connection. The name must match with the name in a connection name in the `connections` section. This property is required.	Yes
databaseName	The name of your database. The name must match the Database asset name in Data Catalog for stitching.	Yes
schemaName	The name of your schema. The name must match the Schema asset name in Data Catalog for stitching.	Yes
dialect	The dialect of the connection. Specify this property for Collibra Data Lineage to properly extract and parse queries that are related to this connection. You can enter one of the following values: `bigquery` `db2` `hana` `hive` `greenplum` `mssql` `mysql` `netezza` `oracle` `postgres` `redshift` `snowflake` `spark` `teradata`	No

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

If the connection definitions are provided but certain properties are not specified, an analyze error called CONFIGURATION is displayed in the transformations table on the Sources tab page when the technical lineage is created. The unspecified properties are marked as UNDEFINED in the analyze error. For more information about the analyze errors, go to Analyze errors and possible solutions in Technical lineage Sources tab page.

Connection definitions

Copy code

Property	Description
connectionDefinitions	This section contains the connection properties to a source in Informatica PowerCenter.
<connectionName>	The type of your source or target data source. This section contains the connection properties to a source or target in Informatica PowerCenter. Note Define a connection in the connection definitions only once; specifically, define a data source with the `<connectionName>` property specified only once in the connection definitions. If you define a connection multiple times, unexpected lineage and stitching issues might occur.
dbname	The name of your source or target database.
schema	The name of your source or target schema.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemNames	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
databases	This section contains the database information. This is required to connect directly to the system or server of the database.
dbname	The name of the database. The database name is the same as the name you entered in the <connectionName> section.
collibraSystemName	The system or server name of the database.
connections	This section contains the connection information. This is required to reference to the system or server of the connection.
connectionName	The name of the connection.
collibraSystemName	The system or server name of the connection.

Important If you are using variables in Informatica PowerCenter, add the value of the variable instead of the name in the connection definitions. For example, if the parameter file contains $DBConnection_dwh=DWH_EXPORT then you add the following connection definitions:

{
	"DWH_EXPORT":

		{ "dbname": "DWH", "schema": "DBO" }
}

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Informatica PowerCenter

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

Connection definitions

Copy code

Property	Description
connectionDefinitions	This section contains the connection properties to a source in Informatica PowerCenter.
<connectionName>	The type of your source or target data source. This section contains the connection properties to a source or target in Informatica PowerCenter. Note Define a connection in the connection definitions only once; specifically, define a data source with the `<connectionName>` property specified only once in the connection definitions. If you define a connection multiple times, unexpected lineage and stitching issues might occur.
dbname	The name of your source or target database.
schema	The name of your source or target schema.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemNames	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.
databases	This section contains the database information. This is required to connect directly to the system or server of the database.
dbname	The name of the database. The database name is the same as the name you entered in the <connectionName> section.
collibraSystemName	The system or server name of the database.
connections	This section contains the connection information. This is required to reference to the system or server of the connection.
connectionName	The name of the connection.
collibraSystemName	The system or server name of the connection.

{
	"DWH_EXPORT":

		{ "dbname": "DWH", "schema": "DBO" }
}

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for Looker

Yes

Main Properties

The required information for creating a technical lineage.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Looker instance. Ingesting the same Looker instance under different source IDs will fail.
Any single Looker instance can be ingested only once. If you create more than one connection for the same Looker instance, integration will fail. If you want to ingest from multiple unique Looker instances, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Looker connection

The Looker connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Looker.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Looker assets.

Yes

Paging limit

Optional property for customizing the Looker API pagination settings. The default value of "50" is sufficient in most cases; however, you can decrease it to help mitigate node limit errors, or increase it to speed up API calls.

Note The paging limit option is known to cause issues when used with Looker Core instances. If you experience issues, for example a Received RST_STREAM: Protocol error, we recommend disabling pagination by setting the value to "0".

Concurrency level

This optional property is intended to help if you are experiencing HTTP 401 Unauthorized errors due to too many concurrent HTTP calls, using the same token. It allows you to specify the internal sizing, meaning the amount of tasks that can be executed at the same time.

The default value is "15", meaning as many as 15 HTTP requests can take place in parallel. Consider reducing the value if you are experiencing HTTP 401 Unauthorized errors. Setting the value to "1" effectively disables the concurrency level, so that HTTP requests will be run in a synchronous manner, instead of in parallel.

Connection timeout

This optional property is intended to help avoid timeout errors, when Edge attempts to connect to your Looker instance. The default value is "30", meaning a timeout error is thrown if a connection is not established within 30 seconds.

If timeout errors persist, try setting the value to "60" or "90".

Source configuration

This field allows you to provide JSON code, to:

Filter on the Looker folders from which you want to ingest metadata.
If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in Looker.
Collibra Data Lineage uses the system names to match the structure of databases in Looker to assets in Data Catalog.

If you previously integrated Looker via the lineage harvester, you can copy and paste in this field the JSON code from your Looker <source ID> configuration file.

Connection definition properties

Property

Description

Mandatory?

Connections

This section contains all Looker connections for which you want to create a technical lineage.

Yes

The name of a connection object in Looker.

Yes

schema

The name of the default schema of a supported data source in Looker.

If the lineage harvester fails to find a specific schema, it uses the default schema.

dbname

The name of the database of a supported data source in Looker.

collibraSystemName

The system or server name of a database.

If you set the useCollibraSystemName property to true in your lineage harvester configuration file, but you either don't create a <source ID> configuration file, or don't specify a value for the collibraSystemName property in your <source ID> configuration file, the system name in the technical lineage is "DEFAULT".

Yes

filters

Optionally, use this section to specify the Looker folders from which you want to ingest metadata.

Note You can filter on Looker folders, but not on Looker data sets. That's because Looker data sets are linked directly to the server, instead of a folder, as shown in the Looker metadata overview. Looker data sets are ingested in the default domain, regardless of any filtering.

Let’s say, for example, you filter on folder B. A Looker Folder asset is created in the specified domain in Collibra, and all of the metadata in folder B is ingested. If folder B has a parent folder A, then a Looker Folder asset is created (in the domain specified for folder B) to preserve the hierarchy, but no metadata from folder A is ingested.

You can specify more than one Looker folder for ingestion into a single domain in Collibra.

Warning If you don't want to filter on Looker Folders, you must completely remove this filters section.

Tip There are significant benefits to filtering by folder ID. For information, see the filters > folderIdsproperty description.

Tip

You can use wildcards to capture multiple connection string combinations:

domainId

The unique resource ID of the domain (or domains), in Collibra, in which you want to ingest data objects from one or more Looker Folders.

Tip You can find the domain ID by clicking the domain type. Then look in the URL of your browser to find the ID. The URL looks like https://<yourcollibrainstance>/domain/<domain ID>?<view>.

description

Any description, as you see fit.

folderNames

The name (or names) of the Looker Folders from which you want to ingest.

Note You must specify either a folder name, a folder ID, or both.

folderIds

The ID (or IDs) of the Looker Folder you want to ingest.

Note You must specify either a folder ID, a folder name, or both.

Tip If you filter by folder ID, filtering is carried out via the API, instead of on the Collibra Data Lineage service instances.

When you filter by folder ID, the lineage harvester accesses only the folders you specify via this property, and sends only that metadata to the Collibra Data Lineage service instance for processing and ingestion in Data Catalog. Conversely, if you filter by folder name (via the folderNames property), metadata from all Looker folders is sent to the Collibra Data Lineage service instance. Only then is filtering applied.

Example

Custom Properties

This section identifies to which Collibra Data Lineage service instance you want to upload the Looker metadata.

Warning This applies only for Collibra Cloud for Government customers.

techlinKey

The unique API key to connect to the Collibra Data Lineage service instance.

A unique user key is needed for each Collibra environment. If you're not sure what your user key is, please contact your Collibra Customer Success Manager.

Warning This applies only for Collibra Cloud for Government customers.

techlinHost

The URL of he Collibra Data Lineage service instance to which you want to upload Tableau metadata, for example "techlin-gcp-eu.collibra.com".

Warning This applies only for Collibra Cloud for Government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Looker instance. Ingesting the same Looker instance under different source IDs will fail.
Any single Looker instance can be ingested only once. If you create more than one connection for the same Looker instance, integration will fail. If you want to ingest from multiple unique Looker instances, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

We highly recommend that you specify only one source ID per Looker service account.

Yes

Looker connection

The Looker connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Looker.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Looker assets.

This is the default domain.

If you want to ingest the contents of specific Looker Folders into specific domains in Collibra, you specify the domain reference IDs in the filters section of your source configuration. See the Source Configuration field below.

Yes

Paging limit

Concurrency level

Connection timeout

If timeout errors persist, try setting the value to "60" or "90".

Source configuration

This field allows you to provide JSON code, to:

Filter on the Looker folders from which you want to ingest metadata.
If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in Looker.
Collibra Data Lineage uses the system names to match the structure of databases in Looker to assets in Data Catalog.

If you previously integrated Looker via the lineage harvester, you can copy and paste in this field the JSON code from your Looker <source ID> configuration file.

Connection definition properties

Property

Description

Mandatory?

Connections

This section contains all Looker connections for which you want to create a technical lineage.

Yes

The name of a connection object in Looker.

Yes

schema

The name of the default schema of a supported data source in Looker.

If the lineage harvester fails to find a specific schema, it uses the default schema.

dbname

The name of the database of a supported data source in Looker.

collibraSystemName

The system or server name of a database.

Yes

filters

Optionally, use this section to specify the Looker folders from which you want to ingest metadata.

You can specify more than one Looker folder for ingestion into a single domain in Collibra.

Warning If you don't want to filter on Looker Folders, you must completely remove this filters section.

Tip There are significant benefits to filtering by folder ID. For information, see the filters > folderIdsproperty description.

Tip

You can use wildcards to capture multiple connection string combinations:

domainId

The unique resource ID of the domain (or domains), in Collibra, in which you want to ingest data objects from one or more Looker Folders.

Tip You can find the domain ID by clicking the domain type. Then look in the URL of your browser to find the ID. The URL looks like https://<yourcollibrainstance>/domain/<domain ID>?<view>.

description

Any description, as you see fit.

folderNames

The name (or names) of the Looker Folders from which you want to ingest.

Note You must specify either a folder name, a folder ID, or both.

folderIds

The ID (or IDs) of the Looker Folder you want to ingest.

Note You must specify either a folder ID, a folder name, or both.

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for SAP Analytics Cloud

Yes

Main Properties

The required information for creating a technical lineage.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per SAP Analytics Cloud system. Ingesting the same SAP Analytics Cloud system under different source IDs will fail.
Any single SAP Analytics Cloud system can be ingested only once. If you create more than one connection for the same SAP Analytics Cloud system, integration will fail. If you want to ingest from multiple unique SAP Analytics Cloud systems, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Yes

SAP Datasphere connection

The SAP Datasphere Catalog connection that you created for ingestion in CollibraData Catalog.

Tip Select the name that you provided in the Name field when you created a connection to SAP Datasphere Catalog.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the SAP Analytics Cloud assets.

This is the default domain.

If you want to ingest the contents of specific SAP Analytics Cloud systems, you must also specify the same domain reference ID in the filters section of your source configuration. See the Source Configuration field below.

Yes

SAP Analytics Cloud tenant ID

If you have multiple SAP Analytics Cloud systems, but you only want to ingest metadata from one of them, this optional field allows you to specific from which system you want to integrate. If you leave this field empty, Edge ingests the metadata from all SAP Analytics Cloud systems.

Note Collibra Data Lineage can only ingest assets in SAP Datasphere that are published. For information on how to publish SAP Analytics Cloud assets in SAP Datasphere Catalog, see Set up SAP Analytics Cloud.

Source Configuration

This field allows you to provide JSON code to specify the SAP Analytics Cloud containers and folders from which you want to ingest metadata.

Connection definition properties

Property

Description

Mandatory?

filters

You can use this section to specify the SAP Analytics Cloud containers and folders from which you want to ingest metadata.

You can specify one domain in Collibra to which assets will be ingested.

The container and folder names in your filter are not case-sensitive.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

domainId

The unique reference ID of the domain in Collibra in which you want to ingest data objects from one or more SAP Analytics Cloud containers and folders.

Important The domain ID that you specify here must the match the domain ID that you specify in the Domain ID field in your Edge capability.

Yes

folderPath

The name (or names) of the containers or folders from which you want to ingest. See the following filter examples for guidance on how to find the folder path.

Yes

Example

Custom Properties

This section identifies to which Collibra Data Lineage service instance you want to upload the Looker metadata.

Warning This applies only for Collibra Cloud for Government customers.

techlinKey

The unique API key to connect to the Collibra Data Lineage service instance.

A unique user key is needed for each Collibra environment. If you're not sure what your user key is, please contact your Collibra Customer Success Manager.

Warning This applies only for Collibra Cloud for Government customers.

techlinHost

The URL of he Collibra Data Lineage service instance to which you want to upload Tableau metadata, for example "techlin-gcp-eu.collibra.com".

Warning This applies only for Collibra Cloud for Government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per SAP Analytics Cloud system. Ingesting the same SAP Analytics Cloud system under different source IDs will fail.
Any single SAP Analytics Cloud system can be ingested only once. If you create more than one connection for the same SAP Analytics Cloud system, integration will fail. If you want to ingest from multiple unique SAP Analytics Cloud systems, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

We highly recommend that you specify only one source ID per SAP Analytics Cloud system.

Yes

SAP Datasphere connection

The SAP Datasphere Catalog connection that you created for ingestion in Collibra Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to SAP Datasphere Catalog.

Yes

Domain ID

Warning

This field is deprecated and should be left empty.
You need to specify the relevant domain ID in the Domain field in the Integrations page of the Data Catalog UI.
If you correctly specify one or more domains in the Data Catalog UI, but also specify a different domain in this field, integration will fail.

Yes

SAP Analytics Cloud tenant ID

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for SSRS/PBRS

Yes

Main Properties

The required information for creating a technical lineage.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per SQL Server Reporting Service (SSRS) or Power BI Report Server (PBRS). Ingesting the same SQL Server Reporting Service (SSRS) or Power BI Report Server (PBRS) under different source IDs will fail.
Any single SSRS or PBRS can be ingested only once. If you create more than one connection for the same SSRS or PBRS, integration will fail. If you want to ingest from multiple unique SSRS or PBRS, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Microsoft SSRS/PBRS connection

The Microsoft SSRS/PBRS connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to SSRS-PBRS.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the SSRS assets.

Yes

Folder Filter

This field allows you to include only specific folders that contain reports or KPIs in the ingestion process.

Important This field is mandatory. If you want to ingest all folders, enter *.

You can filter on multiple folders by:

Specifying folder names.
Specifying the full path to folders.
Using a wildcard.
Using a combination of these approaches. For example: folder1, /database/folder2, /folder3/*

Show me some examples

Scenario	Configuration
Ingest all folders with the name Folder3, anywhere in the folder hierarchy.	`Folder3` Note Reports in child folders of Folder3 are not included in the ingestion. As such: Reports in `/Folder1/Folder2/Folder3` are included in the ingestion. Reports in `/Folder3/ChildFolder` are not included in the ingestion.
Ingest Folder1 and Folder2.	`Folder1, Folder2`
Ingest two folders that are both named Folder1.	In this case, specify the full paths to the folders, for example: `/Database1/Folder1, /Database2/Database3/Folder1`
Use a wildcard to ingest all child folders of Folder1.	`/Folder1/*` Note The reports in all child folders of Folder1 are ingested, but the reports in Folder1 itself are not ingested.
Ingest all reports from Folder1 and all of the reports in the child folders of Folder1.	`/Folder1/*, /Folder1`

Tip For more information about connecting to a SSRS or PBRS folder, see the Microsoft documentation.

Yes

Source configuration

This field allows you to provide JSON code, to:

The <source ID> configuration file allows you to:

If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in SSRS and PBRS.
Provide additional information about databases in SSRS and PBRS, which is necessary if the databases do not contain all information to process the SQL source code correctly.

If you previously integrated SSRS-PBRS via the lineage harvester, you can copy and paste in this field the JSON code from your SSRS-PBRS <source ID> configuration file.

Connection definition properties

Property	Description	Required?
DataSources	This section contains all connections for which you want to create a technical lineage. The `DataSources` section refers to shared data sources in SSRS and PBRS. For more information about shared data sources, see the Microsoft documentation.	Yes
<data source type>	The path of a connection object in SSRS and PBRS.	Yes
dbname	The name of the database of a supported data source in SSRS and PBRS.	No
schema	The name of the default schema of a supported data source in SSRS and PBRS.	No
dialect	The dialect of the supported data source in SSRS and PBRS.	No
collibraSystemName	The system or server name of the database. If you set the `useCollibraSystemName` property to `true` in your lineage harvester configuration file, but you either don't create a <source ID> configuration file, or don't specify a value for the `collibraSystemName` property in your <source ID> configuration file, the system name in the technical lineage is "DEFAULT". How do I configure this property if I have two databases with the same name? Let's assume you have two databases named Customers. When you prepare the physical data layer in Data Catalog, you create a System asset for each of these databases. Let's say you named them Customers-Europe and Customers-USA. You can then configure this property as follows. "Redshift": { "dbname": "Customer", "schema": "redshift-schema-name", "dialect": "redshift", "collibraSystemName": "Customers-Europe" }, "Oracle": { "dbname": "Customer", "schema": "oracle-schema-name", "dialect": "oracle", "collibraSystemName": "Customers-USA" }	Yes
CustomDataSources	You can use custom data processing extensions that are used to support embedded data sources of which the data source definition is specified locally in a report or embedded data set. The `CustomDataSources` section refers to embedded data sources in SSRS and PBRS. For more information about embedded data sources, see the Microsoft documentation.	No
<path to report>/<custom data source name>	The full path to the report and the custom data source name. You can use wildcards to match multiple folders, reports or data sets. The connection information is this section is used to add missing information or to overwrite parsed information.	No
dbname	The name of the database of a custom data source in SSRS and PBRS.	No
schema	The name of the schema of a custom data source in power. If you don't provide the schema name, the default schema is used.	No
dialect	The dialect of the custom data source in SSRS and PBRS. Click for possible values: azure, for an Azure SQL Server data source. bigquery, for a Google BigQuery data source. db2, for an IBM DB2 data source. hana, for a SAP Hana data source. hive, for a HiveQL data source. greenplum, for a Greenplum data source. mssql, for a Microsoft SQL Server data source. mysql, for a MySQL data source. netezza, for a Netezza data source. oracle, for an Oracle data source. postgres, for a PostgreSQL data source. redshift, for an Amazon Redshift data source. snowflake, for a Snowflake data source. spark, for a Spark SQL data source. sybase, for a Sybase data source. teradata, for a Teradata data source.	No

Example

Custom Properties

This section identifies to which Collibra Data Lineage service instance you want to upload the SSRS-PBRS metadata.

Warning This applies only for Collibra Cloud for Government customers.

techlinKey

The unique API key to connect to the Collibra Data Lineage service instance.

A unique user key is needed for each Collibra environment. If you're not sure what your user key is, please contact your Collibra Customer Success Manager.

Warning This applies only for Collibra Cloud for Government customers.

techlinHost

The URL of he Collibra Data Lineage service instance to which you want to upload Tableau metadata, for example techlin-gcp-eu.collibra.com.

Warning This applies only for Collibra Cloud for Government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per SQL Server Reporting Service (SSRS) or Power BI Report Server (PBRS). Ingesting the same SQL Server Reporting Service (SSRS) or Power BI Report Server (PBRS) under different source IDs will fail.
Any single SSRS or PBRS can be ingested only once. If you create more than one connection for the same SSRS or PBRS, integration will fail. If you want to ingest from multiple unique SSRS or PBRS, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

We highly recommend that you specify only one source ID per SSRS or PBRS service account.

Yes

Microsoft SSRS/PBRS connection

The Microsoft SSRS/PBRS connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to SSRS-PBRS.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the SSRS assets.

Yes

Folder Filter

This field allows you to include only specific folders that contain reports or KPIs in the ingestion process.

Important This field is mandatory. If you want to ingest all folders, enter *.

You can filter on multiple folders by:

Specifying folder names.
Specifying the full path to folders.
Using a wildcard.
Using a combination of these approaches. For example: folder1, /database/folder2, /folder3/*

Show me some examples

Scenario	Configuration
Ingest all folders with the name Folder3, anywhere in the folder hierarchy.	`Folder3` Note Reports in child folders of Folder3 are not included in the ingestion. As such: Reports in `/Folder1/Folder2/Folder3` are included in the ingestion. Reports in `/Folder3/ChildFolder` are not included in the ingestion.
Ingest Folder1 and Folder2.	`Folder1, Folder2`
Ingest two folders that are both named Folder1.	In this case, specify the full paths to the folders, for example: `/Database1/Folder1, /Database2/Database3/Folder1`
Use a wildcard to ingest all child folders of Folder1.	`/Folder1/*` Note The reports in all child folders of Folder1 are ingested, but the reports in Folder1 itself are not ingested.
Ingest all reports from Folder1 and all of the reports in the child folders of Folder1.	`/Folder1/*, /Folder1`

Tip For more information about connecting to a SSRS or PBRS folder, see the Microsoft documentation.

Yes

Source configuration

This field allows you to provide <source ID> configuration file JSON code.

The <source ID> configuration file allows you to:

If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in SSRS and PBRS.
Provide additional information about databases in SSRS and PBRS, which is necessary if the databases do not contain all information to process the SQL source code correctly.

If you previously integrated SSRS-PBRS via the lineage harvester, you can copy and paste in this field the JSON code from your SSRS-PBRS <source ID> configuration file.

Connection definition properties

Property	Description	Required?
DataSources	This section contains all connections for which you want to create a technical lineage. The `DataSources` section refers to shared data sources in SSRS and PBRS. For more information about shared data sources, see the Microsoft documentation.	Yes
<data source type>	The path of a connection object in SSRS and PBRS.	Yes
dbname	The name of the database of a supported data source in SSRS and PBRS.	No
schema	The name of the default schema of a supported data source in SSRS and PBRS.	No
dialect	The dialect of the supported data source in SSRS and PBRS.	No
collibraSystemName	The system or server name of the database. If you set the `useCollibraSystemName` property to `true` in your lineage harvester configuration file, but you either don't create a <source ID> configuration file, or don't specify a value for the `collibraSystemName` property in your <source ID> configuration file, the system name in the technical lineage is "DEFAULT". How do I configure this property if I have two databases with the same name? Let's assume you have two databases named Customers. When you prepare the physical data layer in Data Catalog, you create a System asset for each of these databases. Let's say you named them Customers-Europe and Customers-USA. You can then configure this property as follows. "Redshift": { "dbname": "Customer", "schema": "redshift-schema-name", "dialect": "redshift", "collibraSystemName": "Customers-Europe" }, "Oracle": { "dbname": "Customer", "schema": "oracle-schema-name", "dialect": "oracle", "collibraSystemName": "Customers-USA" }	Yes
CustomDataSources	You can use custom data processing extensions that are used to support embedded data sources of which the data source definition is specified locally in a report or embedded data set. The `CustomDataSources` section refers to embedded data sources in SSRS and PBRS. For more information about embedded data sources, see the Microsoft documentation.	No
<path to report>/<custom data source name>	The full path to the report and the custom data source name. You can use wildcards to match multiple folders, reports or data sets. The connection information is this section is used to add missing information or to overwrite parsed information.	No
dbname	The name of the database of a custom data source in SSRS and PBRS.	No
schema	The name of the schema of a custom data source in power. If you don't provide the schema name, the default schema is used.	No
dialect	The dialect of the custom data source in SSRS and PBRS. Click for possible values: azure, for an Azure SQL Server data source. bigquery, for a Google BigQuery data source. db2, for an IBM DB2 data source. hana, for a SAP Hana data source. hive, for a HiveQL data source. greenplum, for a Greenplum data source. mssql, for a Microsoft SQL Server data source. mysql, for a MySQL data source. netezza, for a Netezza data source. oracle, for an Oracle data source. postgres, for a PostgreSQL data source. redshift, for an Amazon Redshift data source. snowflake, for a Snowflake data source. spark, for a Spark SQL data source. sybase, for a Sybase data source. teradata, for a Teradata data source.	No

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Matillion connection

The Matillion connection that you created.

Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Matillion connection.

Group Name

The name of your group in Matillion.

Yes

Project Name

The name of your project in Matillion.

You can only add the name of one project. If you want to create a technical lineage for other projects, add a technical lineage for Matillion capability for each project.

Note Each capability requires a separate Matillion connection.

Yes

Environment Name

The name of your environment in Matillion.

You can only add the name of one environment. If you want to create a technical lineage for other environments, add a technical lineage for Matillion capability for each environment.

Yes

Dialect

The dialect of the database.

Yes

Start timestamp

The timestamp of tasks in Matillion, which indicates the amount of metadata that technical lineage via Edge collects.

Specify this field with a UNIX timestamp in milliseconds. The default value is 1, which gets as much history as Matillion provides. Matillion provides 7 days of history by default.

Yes

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

Connection definitions

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>

The information of the supported data sources in Matillion to be collected by Collibra Data Lineage.

<database name>: The database name in Matillion.
<server name>: The name of the server that the database is running on. You can specify found_hostname=* to include all servers.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The name of the database asset in Data Catalog.

If you leave this property blank, the database is stitched to the database of DEFAULT in Data Catalog.

schema

The name of the schema asset in Data Catalog. Specify this property with the schema name that you created when you registered the data source.

If you leave this property blank, the schema is stitched to the schema of DEFAULT in Data Catalog.

dialect

Select one of the following dialects for your data source

collibraSystemName

The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source.

If you leave this property blank, the system is stitched to the system of DEFAULT in Data Catalog. If you are missing lineage or your lineage objects aren’t stitching to Catalog assets in Data Catalog as you expect, ensure this property is specified properly.

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

Note A value is required, but it is not used when technical lineage for Matillion is created.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Matillion

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Matillion connection

The Matillion connection that you created.

Note Collibra Data Intelligence Platform 2023.03 or newer is required to use the Matillion connection.

Group Name

The name of your group in Matillion.

Yes

Project Name

The name of your project in Matillion.

You can only add the name of one project. If you want to create a technical lineage for other projects, add a technical lineage for Matillion capability for each project.

Note Each capability requires a separate Matillion connection.

Yes

Environment Name

The name of your environment in Matillion.

You can only add the name of one environment. If you want to create a technical lineage for other environments, add a technical lineage for Matillion capability for each environment.

Yes

Dialect

The dialect of the database.

Yes

Start timestamp

The timestamp of tasks in Matillion, which indicates the amount of metadata that technical lineage via Edge collects.

Specify this field with a UNIX timestamp in milliseconds. The default value is 1, which gets as much history as Matillion provides. Matillion provides 7 days of history by default.

Yes

Source Configuration

The connection definitions and system names. Specify the following properties in JSON format and enter the content in this field.

Connection definitions

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>

The information of the supported data sources in Matillion to be collected by Collibra Data Lineage.

<database name>: The database name in Matillion.
<server name>: The name of the server that the database is running on. You can specify found_hostname=* to include all servers.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The name of the database asset in Data Catalog.

If you leave this property blank, the database is stitched to the database of DEFAULT in Data Catalog.

schema

The name of the schema asset in Data Catalog. Specify this property with the schema name that you created when you registered the data source.

If you leave this property blank, the schema is stitched to the schema of DEFAULT in Data Catalog.

dialect

Select one of the following dialects for your data source

collibraSystemName

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

Note A value is required, but it is not used when technical lineage for Matillion is created.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for MicroStrategy

Yes

Main Properties

The required information for creating a technical lineage.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per MicroStrategy Intelligence Server. Ingesting the same MicroStrategy Intelligence Server under different source IDs will fail.
Any single MicroStrategy Intelligence Server can be ingested only once. If you create more than one connection for the same MicroStrategy Intelligence Server, integration will fail. If you want to ingest from multiple unique MicroStrategy Intelligence Servers, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

MicroStrategy connection

The MicroStrategy connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to MicroStrategy.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the MicroStrategy assets.

Yes

URL for reports

This optional property ensures that the correct URL to data objects in MicroStrategy is included on the asset pages of corresponding MicroStrategy assets. The required value depends on which platform you run MicroStrategy:

For J2EE, use: "MicroStrategy/servlet/mstrWeb"
For .NET, use: "MicroStrategy/asp/Main.aspx"

MicroStrategy Library URL

If you are using a custom URL to connect to the MicroStrategy Library Server, use this field to specify the custom library URL.

Important You only need to specify the URL if both of the following are true:

You are connecting to a proxy server.
You are not using the default, hardcoded URL to the MicroStrategy Library Server.
Example If the URL to your MicroStrategy Library is https://collibra.microstrategy.com/MicroStrategyLibrary/api, you don't need to use this field, as that is the default, hardcoded URL. However, if the URL is something like https://collibra.microstrategy.com/MicroStrategyLibraryProd/api, then use this field and configure it as follows:
"microStrategyLibraryUrl": "MicroStrategyLibraryProd"

Source configuration

This field allows you to provide JSON code, to:

Specify the default domain, meaning the domain in Collibra in which the corresponding assets of MicroStrategy metadata will be ingested if domain mapping is not configured.
Note If you do configure domain mapping, the default domain is still the destination domain of the MicroStrategy Server asset.
Optionally, specify from which MicroStrategy projects you want to ingest metadata, and into which domains you want to ingest the corresponding assets.
Optionally, configure data source mapping, to map the name of a data source returned by the lineage harvester to the true name of the data source.
Note Mapping doesn't work for custom SQL.

If you previously integrated MicroStrategy via the lineage harvester, you can copy and paste in this field the JSON code from your MicroStrategy <source ID> configuration file.

Connection definition properties

Property	Description	Mandatory
default_domain_id	The domain in which you want the corresponding assets of MicroStrategy metadata to be ingested. Note If you configure filtering, only the MicroStrategy Server asset is ingested into this default domain.	Yes
filters	This section allows you to specify: From which MicroStrategy projects you want to harvest metadata. Into which domains in Collibra you want to ingest the corresponding assets. If you don't want to filter on projects, don't include this section in your <source ID> configuration file.	No
domainId	The unique resource ID of the domain (or domains) in Collibra in which you want to ingest the MicroStrategy assets. Tip If you use a `filters` section, you must include the `domainId` property in the section. If, by chance, you want to filter on certain projects, but you want to ingest all assets into the default domain, then the value of the `domainId` property must match the value of the `default_domain_id` property. Show me an example "default_domain_id": "1234567890", "filters": [ { "domainId": "1234567890", "projectNames": ["MicroStrategy Tutorial","Testing_MSTR"] }, How do I find a domain reference ID? Open the relevant domain in Collibra. The URL looks like: https://<yourcollibrainstance>/domain/22258f64-40b6-4b16-9c08-c95f8ec0da26?view=00000000-0000-0000-0000-000000040001. In this example, the reference ID is in bold.	No
projectIds	The IDs of the MicroStrategy projects from which you want to ingest metadata.	No
projectNames	The project names of the MicroStrategy projects from which you want to ingest metadata.	No
datasourceMapping	This optional section allows you to configure data source mapping. Include this section only if you need to differentiate between multiple data sources that have the same name. Note Mapping doesn't work for custom SQL.	No
found_datasource	The name of the data source that was returned by the lineage harvester, as shown in the technical lineage. Note The data source name is case-sensitive.	Yes
found_project	The name of the project in which the data source information resides. You can specify an asterisk (*) to search for data source information across all projects.	Yes
mapping	Use this section to map the data source name that was returned by the lineage harvester to the true name of the data source. Example You have a Redshift data source named "RD_pearl", but the lineage harvester has returned the name "Redshift_connection". You can configure the `datasourceMapping` section as follows: { "datasourceMapping": [ { "found_datasource": "REDSHIFT", "found_project": "*", "mapping": { "dbname": "RD_pearl", "collibraSystemName": "TV_dev" } } ] }	Yes
dbname	The name of the database to which you want to map the found data source.	Yes
schema	The name of the schema in MicroStrategy.	No
dialect	The dialect of the data source in MicroStrategy.	No
collibraSystemName	The system or server name of a database. If you set the `useCollibraSystemName` property to `true` in your lineage harvester configuration file, but you either don't create a <source ID> configuration file, or don't specify a value for the `collibraSystemName` property in your <source ID> configuration file, the system name in the technical lineage is "DEFAULT". If you set the `useCollibraSystemName` property to `false` in your lineage harvester configuration file, leave this property empty as follows: `"collibraSystemName": ""`. How do I configure this property if I have two databases with the same name? Let's assume that you have a data source named Customers. You use this data source connection in two different projects, Project_A and Project_B, but they are actually two different databases. When you prepare the physical data layer in Data Catalog, you create a System asset for each of these databases. Let's say you named them Customers-North and Customers-South. You can then configure this property as follows. "datasourceMapping": [ { "found_datasource": "Customers", "found_project": "Project_A", "mapping": { "dbname": "Customers", "collibraSystemName": "Customers_North" } }, { "found_datasource": "Customers", "found_project": "Project_B", "mapping": { "dbname": "Customers", "collibraSystemName": "Customers_South" } } ] Warning The values of this property must exactly match the name of your System asset in Collibra.	Yes

Example

Custom Properties

This section identifies to which Collibra Data Lineage service instance you want to upload the MicroStrategy metadata.

Warning This applies only for Collibra Cloud for Government customers.

techlinKey

The unique API key to connect to the Collibra Data Lineage service instance.

A unique user key is needed for each Collibra environment. If you're not sure what your user key is, please contact your Collibra Customer Success Manager.

Warning This applies only for Collibra Cloud for Government customers.

techlinHost

The URL of he Collibra Data Lineage service instance to which you want to upload Tableau metadata, for example "techlin-gcp-eu.collibra.com".

Warning This applies only for Collibra Cloud for Government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Maximum parallel requests

This optional property allows you to specify the internal sizing, meaning the amount of tasks that can be executed at the same time.

The default value is "1", which means that HTTP requests are run in a synchronous manner, instead of in parallel. As value of "5", for example, means that as many as 5 HTTP requests can take place in parallel.

A lower value reduces the chances of experiencing HTTP 401 Unauthorized errors.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per MicroStrategy Intelligence Server. Ingesting the same MicroStrategy Intelligence Server under different source IDs will fail.
Any single MicroStrategy Intelligence Server can be ingested only once. If you create more than one connection for the same MicroStrategy Intelligence Server, integration will fail. If you want to ingest from multiple unique MicroStrategy Intelligence Servers, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

We highly recommend that you specify only one source ID per MicroStrategy Intelligence Server.

Yes

MicroStrategy connection

The MicroStrategy connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to MicroStrategy.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the MicroStrategy assets.

Yes

URL for reports

For J2EE, use: "MicroStrategy/servlet/mstrWeb"
For .NET, use: "MicroStrategy/asp/Main.aspx"

MicroStrategy Library URL

If you are using a custom URL to connect to the MicroStrategy Library Server, use this field to specify the custom library URL.

Important You only need to specify the URL if both of the following are true:

You are connecting to a proxy server.
You are not using the default, hardcoded URL to the MicroStrategy Library Server.
Example If the URL to your MicroStrategy Library is https://collibra.microstrategy.com/MicroStrategyLibrary/api, you don't need to use this field, as that is the default, hardcoded URL. However, if the URL is something like https://collibra.microstrategy.com/MicroStrategyLibraryProd/api, then use this field and configure it as follows:
"microStrategyLibraryUrl": "MicroStrategyLibraryProd"

Source configuration

This field allows you to provide JSON code, to:

Specify the default domain, meaning the domain in Collibra in which the corresponding assets of MicroStrategy metadata will be ingested if domain mapping is not configured.
Note If you do configure domain mapping, the default domain is still the destination domain of the MicroStrategy Server asset.
Optionally, specify from which MicroStrategy projects you want to ingest metadata, and into which domains you want to ingest the corresponding assets.
Optionally, configure data source mapping, to map the name of a data source returned by the lineage harvester to the true name of the data source.
Note Mapping doesn't work for custom SQL.

If you previously integrated MicroStrategy via the lineage harvester, you can copy and paste in this field the JSON code from your MicroStrategy<source ID> configuration file.

Connection definition properties

Property	Description	Mandatory
default_domain_id	The domain in which you want the corresponding assets of MicroStrategy metadata to be ingested. Note If you configure filtering, only the MicroStrategy Server asset is ingested into this default domain.	Yes
filters	This section allows you to specify: From which MicroStrategy projects you want to harvest metadata. Into which domains in Collibra you want to ingest the corresponding assets. If you don't want to filter on projects, don't include this section in your <source ID> configuration file.	No
domainId	The unique resource ID of the domain (or domains) in Collibra in which you want to ingest the MicroStrategy assets. Tip If you use a `filters` section, you must include the `domainId` property in the section. If, by chance, you want to filter on certain projects, but you want to ingest all assets into the default domain, then the value of the `domainId` property must match the value of the `default_domain_id` property. Show me an example "default_domain_id": "1234567890", "filters": [ { "domainId": "1234567890", "projectNames": ["MicroStrategy Tutorial","Testing_MSTR"] }, How do I find a domain reference ID? Open the relevant domain in Collibra. The URL looks like: https://<yourcollibrainstance>/domain/22258f64-40b6-4b16-9c08-c95f8ec0da26?view=00000000-0000-0000-0000-000000040001. In this example, the reference ID is in bold.	No
projectIds	The IDs of the MicroStrategy projects from which you want to ingest metadata.	No
projectNames	The project names of the MicroStrategy projects from which you want to ingest metadata.	No
datasourceMapping	This optional section allows you to configure data source mapping. Include this section only if you need to differentiate between multiple data sources that have the same name. Note Mapping doesn't work for custom SQL.	No
found_datasource	The name of the data source that was returned by the lineage harvester, as shown in the technical lineage. Note The data source name is case-sensitive.	Yes
found_project	The name of the project in which the data source information resides. You can specify an asterisk (*) to search for data source information across all projects.	Yes
mapping	Use this section to map the data source name that was returned by the lineage harvester to the true name of the data source. Example You have a Redshift data source named "RD_pearl", but the lineage harvester has returned the name "Redshift_connection". You can configure the `datasourceMapping` section as follows: { "datasourceMapping": [ { "found_datasource": "REDSHIFT", "found_project": "*", "mapping": { "dbname": "RD_pearl", "collibraSystemName": "TV_dev" } } ] }	Yes
dbname	The name of the database to which you want to map the found data source.	Yes
schema	The name of the schema in MicroStrategy.	No
dialect	The dialect of the data source in MicroStrategy.	No
collibraSystemName	The system or server name of a database. If you set the `useCollibraSystemName` property to `true` in your lineage harvester configuration file, but you either don't create a <source ID> configuration file, or don't specify a value for the `collibraSystemName` property in your <source ID> configuration file, the system name in the technical lineage is "DEFAULT". If you set the `useCollibraSystemName` property to `false` in your lineage harvester configuration file, leave this property empty as follows: `"collibraSystemName": ""`. How do I configure this property if I have two databases with the same name? Let's assume that you have a data source named Customers. You use this data source connection in two different projects, Project_A and Project_B, but they are actually two different databases. When you prepare the physical data layer in Data Catalog, you create a System asset for each of these databases. Let's say you named them Customers-North and Customers-South. You can then configure this property as follows. "datasourceMapping": [ { "found_datasource": "Customers", "found_project": "Project_A", "mapping": { "dbname": "Customers", "collibraSystemName": "Customers_North" } }, { "found_datasource": "Customers", "found_project": "Project_B", "mapping": { "dbname": "Customers", "collibraSystemName": "Customers_South" } } ] Warning The values of this property must exactly match the name of your System asset in Collibra.	Yes

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Maximum parallel requests

This optional property allows you to specify the internal sizing, meaning the amount of tasks that can be executed at the same time.

A lower value reduces the chances of experiencing HTTP 401 Unauthorized errors.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for MySQL

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Netezza

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Oracle

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Database Links	This query retrieves links to other databases.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Materialized Views	This query retrieves materialized view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Database Links	This query retrieves links to other databases.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Materialized Views	This query retrieves materialized view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Database Link Mapping

If you are using DBLinks, this optional field allows you to configure, per data source, the database and schema to which DBLink points.

The configuration format is as follows:

{"<dblink_name>": {"database":"<database>","schema":"<schema>"}, ...}

Tip If you’re using a DBLink to target another source, you need to share the database model between the targeted (independent) source and the dependent source. Use the Dependent On Sources option to configure that dependency and share the database model.

Important If the same DBLink, for example dblink.example.com, exists in multiple databases, the formatting shown in the previous example still applies, but you need to enclose it in curly brackets and specify the relevant database, as follows:

Basic formatting, as shown in the previous example:
"dblink.example.com": {"database":"Database_A","schema":"Schema_A1"}
Formatting if the DBLink exists in multiple databases and you want to apply it only in a database named "dbScope1":
"dbScope1": {"dblink.example.com": {"database":"Database_A","schema":"Schema_A1"}}

If a DBLink is referenced in multiple mappings, as shown in the following example, the first mapping is used.

"dbScope1": {
   "dblink.example.com": {"database":"DevDB_A","schema":"DevSch_A1"}
}, 
   "dblink.example.com": {"database":"Database_A","schema":"Schema_A1"}}

In this case, occurrences of dblink.example.com in the database named "dbScope1" are mapped to:

"database":"DevDB_A","schema":"DevSch_A1"

Database-System Mapping

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for PostgreSQL

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Important This field is mandatory, but the value you specify is not taken into consideration. We will remove this field in a future Collibra version.

Yes

Database Name Override

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for Power BI

Yes

Main Properties

The required custom properties are based on the selected capability template.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Power BI service. Ingesting the same Power BI service under different source IDs will fail.
Any single Power BI service can be ingested only once. If you create more than one connection for the same Power BI service, integration will fail. If you want to ingest from multiple unique Power BI services, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Power BI connection

The Power BI connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Power BI.

Yes

API URL

The API URL of your Power BI service.

The default value is https://api.powerbi.com.

Important This property is only relevant for US government or national cloud Power BI customers, in which case you must include and specify values for both this property and the scope property. For complete information, consult Microsoft's documentation on Power BI for US government customers.

Scope

Optional property that is intended only for customers with a different scope, such as Chinese tenants.

Example https://analysis.chinacloudapi.cn/powerbi/api/.default

Important If you are a US government or national cloud Power BI customer, you must include and specify values for both this property and the apiUrl property. For complete information, consult Microsoft's documentation on Power BI for US government customers.

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Power BI assets.

Yes

Source configuration

This field allows you to provide JSON code for database mapping, workspace filtering and specifying the name of a System asset in Collibra.

Map the names of the server, database and schema that were collected by the lineage harvester to their true names.
Note Mapping doesn't work for custom SQL.
Configure workspace filtering.
Tip We highly recommend that you read through Filtering Power BI workspaces for important information and guidance before configuring your filters.
If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in Power BI. Collibra Data Lineage uses the system names to match the structure of databases in Power BI to assets in Data Catalog.

If you previously integrated Power BI via the lineage harvester, you can copy and paste in this field the JSON code from your Power BI <source ID> configuration file.

Connection definition properties

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name>

The database information of supported data sources in Power BI that is typically collected by the lineage harvester. Specify the name of the database (found_dbname), on which server a database is running (found_hostname), and optionally, the name of the schema (found_schema). You then use the child properties to map the names collected by the lineage harvester to the true names.

Important Schema mapping is available for schemas that come from Power Query connections. It is not available, however, if a Power Query connection is created with SQL (or MDX) statements and the schema is specified in those statements.

Important The keys that you specify must be unique.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The true name (display name) of the database collected by the lineage harvester.

schema

The true name (display name) of the schema collected by the lineage harvester.

If the lineage harvester fails to find a specific schema, it uses the schema you specify in this property.

dialect

The dialect of the supported data source in Power BI.

collibraSystemName

The system or server name of a database.

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Important If you are using a <source ID> configuration file for the purpose of providing the true system name of an ODBC database in Power BI, you are not required to:

Set the useCollibraSystemName property in the lineage harvester configuration file to true.
Specify a Collibra system name in the <source ID> configuration file.

However, if the useCollibraSystemName property is set to true in the lineage harvester configuration file, then you must specify a Collibra system name in the <source ID> configuration file.

Yes (unless you are using the <source ID> file to provide the true system names of ODBC databases in Power BI.)

filters

This section allows you to specify the Power BI workspaces from which you want to ingest metadata.

The filters work as "workspace AND workspace AND capacity AND capacity", meaning that if you specify a capacity, all of the workspaces in that capacity are also ingested.

Warning If you don't want to specify the Power BI workspaces from which to ingest, you must completely remove this filters section.

Tip

You can use wildcards to capture multiple connection string combinations:

domainId

The unique resource ID of the domain (or domains), in Collibra Data Intelligence Platform, in which you want to ingest the Power BI assets.

Tip You can find the domain ID by clicking the domain type. Then look in the URL of your browser to find the ID. The URL looks like https://<yourcollibrainstance>/domain/<domain ID>?<view>.

Yes

description

Any description, as you see fit.

Yes

workspaceNames

The names of Power BI workspaces from which you want to ingest metadata.

Important Any meta-characters in the name of a workspace must be enclosed in square brackets "[ ]". For example, a workspace with the name "Sale and Marketing [automobiles]" should be formatted as follows:
Sale and Marketing [[]automobiles[]]

workspaceIds

The IDs of Power BI workspaces from which you want to ingest metadata.

Tip We highly recommend that you read through Filtering Power BI workspaces for important information and guidance before configuring your filters.

capacityNames

The names of capacities on which you want to filter.

capacityIds

The IDs of capacities on which you want to filter.

Warning Any letters in a capacity ID must be in upper case.

excludeWorkspaceNames

The names of Power BI workspaces that you want to exclude from the ingestion job.

This is useful if you want to exclude, for example, dedicated development and testing workspaces.

Note The metadata of inactive and personal workspaces is not harvested or uploaded to the Collibra Data Lineage service instance. An inactive workspace is one for which no reports or dashboards have been viewed in the past 60 days. My workspace is the personal workspace for any Power BI customer to work with their own, personal content.

For complete details on the advantages, limitations and configuration considerations of this property, see Filtering Power BI workspaces.

excludeWorkspaceIds

The IDs of Power BI workspaces that you want to exclude from the ingestion job.

This is useful if you want to exclude, for example, dedicated development and testing workspaces.

For complete details on the advantages, limitations and configuration considerations of this property, see Filtering Power BI workspaces.

Example

Custom Properties

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Metadata is harvested and uploaded in a ZIP file to a Collibra Data Lineage service instance, for processing.

Use this optional property to specify whether or not the raw metadata should be deleted after it has been processed.

If you select this option, the raw metadata is deleted after processing. If you don't select this option, it is stored in an Amazon S3 bucket.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Use HTTP/1.1 protocol

Option to use HTTP/1.1 streams, in case file-size limitations are resulting in timeout errors when using the default HTTP/2 streams.

Enable lineage for DAX queries

Note This feature is not available on Collibra Cloud for Government.

Option to enable DAX analysis via Collibra AI. This feature:

Creates column-level lineage that includes your calculated columns and measures in Power BI.
Enables stitching between calculated columns in the technical lineage and the corresponding Power BI Column assets in Data Catalog.

Select this option to enable DAX analysis.

Clear the checkbox to disable DAX analysis.

For complete information on this feature, go to DAX analysis via Collibra AI.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Power BI service. Ingesting the same Power BI service under different source IDs will fail.
Any single Power BI service can be ingested only once. If you create more than one connection for the same Power BI service, integration will fail. If you want to ingest from multiple unique Power BI services, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Power BI Connection

The Power BI connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Power BI.

Yes

API URL

The API URL of your Power BI service.

The default value is https://api.powerbi.com.

Scope

Optional property that is intended only for customers with a different scope, such as Chinese tenants.

Example https://analysis.chinacloudapi.cn/powerbi/api/.default

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Power BI assets.

Yes

Source Configuration

This field allows you to provide JSON code for database mapping, workspace filtering and specifying the name of a System asset in Collibra.

Map the names of the server, database and schema that were collected by the lineage harvester to their true names.
Note Mapping doesn't work for custom SQL.
Configure workspace filtering.
Tip We highly recommend that you read through Filtering Power BI workspaces for important information and guidance before configuring your filters.
If useCollibraSystemName in the lineage harvester configuration file is set to true, use the collibraSystemName property to specify the system name of databases in Power BI. Collibra Data Lineage uses the system names to match the structure of databases in Power BI to assets in Data Catalog.

If you previously integrated Power BI via the lineage harvester, you can copy and paste in this field the JSON code from your Power BI <source ID> configuration file.

Connection definition properties

Property

Description

Mandatory?

found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name>

Important The keys that you specify must be unique.

Tip

You can use wildcards to capture multiple connection string combinations:

Yes

dbname

The true name (display name) of the database collected by the lineage harvester.

schema

The true name (display name) of the schema collected by the lineage harvester.

If the lineage harvester fails to find a specific schema, it uses the schema you specify in this property.

dialect

The dialect of the supported data source in Power BI.

collibraSystemName

The system or server name of a database.

Warning The value of this property must exactly match (including for case-sensitivity) the name of your System asset in Collibra.

Important If you are using a <source ID> configuration file for the purpose of providing the true system name of an ODBC database in Power BI, you are not required to:

Set the useCollibraSystemName property in the lineage harvester configuration file to true.
Specify a Collibra system name in the <source ID> configuration file.

However, if the useCollibraSystemName property is set to true in the lineage harvester configuration file, then you must specify a Collibra system name in the <source ID> configuration file.

Yes (unless you are using the <source ID> file to provide the true system names of ODBC databases in Power BI.)

filters

This section allows you to specify the Power BI workspaces from which you want to ingest metadata.

The filters work as "workspace AND workspace AND capacity AND capacity", meaning that if you specify a capacity, all of the workspaces in that capacity are also ingested.

Warning If you don't want to specify the Power BI workspaces from which to ingest, you must completely remove this filters section.

Tip

You can use wildcards to capture multiple connection string combinations:

domainId

The unique resource ID of the domain (or domains), in Collibra Data Intelligence Platform, in which you want to ingest the Power BI assets.

Tip You can find the domain ID by clicking the domain type. Then look in the URL of your browser to find the ID. The URL looks like https://<yourcollibrainstance>/domain/<domain ID>?<view>.

Yes

description

Any description, as you see fit.

Yes

workspaceNames

The names of Power BI workspaces from which you want to ingest metadata.

workspaceIds

The IDs of Power BI workspaces from which you want to ingest metadata.

Tip We highly recommend that you read through Filtering Power BI workspaces for important information and guidance before configuring your filters.

capacityNames

The names of capacities on which you want to filter.

capacityIds

The IDs of capacities on which you want to filter.

Warning Any letters in a capacity ID must be in upper case.

excludeWorkspaceNames

The names of Power BI workspaces that you want to exclude from the ingestion job.

This is useful if you want to exclude, for example, dedicated development and testing workspaces.

For complete details on the advantages, limitations and configuration considerations of this property, see Filtering Power BI workspaces.

excludeWorkspaceIds

The IDs of Power BI workspaces that you want to exclude from the ingestion job.

This is useful if you want to exclude, for example, dedicated development and testing workspaces.

For complete details on the advantages, limitations and configuration considerations of this property, see Filtering Power BI workspaces.

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Metadata is harvested and uploaded in a ZIP file to a Collibra Data Lineage service instance, for processing.

Use this optional property to specify whether or not the raw metadata should be deleted after it has been processed.

If you select this option, the raw metadata is deleted after processing. If you don't select this option, it is stored in an Amazon S3 bucket.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Use HTTP/1.1 protocol

Option to use HTTP/1.1 streams, in case file-size limitations are resulting in timeout errors when using the default HTTP/2 streams.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Enable lineage for DAX queries

Note This feature is not available on Collibra Cloud for Government.

Option to enable DAX analysis via Collibra AI. This feature:

Creates column-level lineage that includes your calculated columns and measures in Power BI.
Enables stitching between calculated columns in the technical lineage and the corresponding Power BI Column assets in Data Catalog.

Select this option to enable DAX analysis.

Clear the checkbox to disable DAX analysis.

For complete information on this feature, go to DAX analysis via Collibra AI.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SAP HANA

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Note If you are migrating an SAP HANA data source from the lineage harvester, ensure that you run the ignore-source command with the source ID from the lineage harvester configuration file. When you synchronize this capability, an error occurs if the source ID from the lineage harvester exists even if you use the same source ID for this field. For more information, go to Migrate the technical lineage of a data source.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.
Calculated Views	This query retrieves calculated views.
Dependencies of Calculated Views	This query retrieves dependencies of calculated views.
Cross-references of Calculated Views	Cross-references of Calculated Views

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

SQL Active

An option to determine whether to include or remove the technical lineage of the data source with the SQL based input.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

If you have a capability with this option selected and the synchronization of the capability fails with the Missing required parameter hanaUseCloudScanner error message, go to In Edge Harvester, previously configured/working SAP HANA Classic capabilities fail to submit to Edge in Collibra Support Portal for a solution.

Calculated Views Active

An option to determine whether to include or remove the technical lineage from calculated views in an SAP HANA Classic on-premises data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Use Hana Cloud for Calculated Views

An option to determine whether to include or remove the technical lineage from calculated views in an SAP HANA Cloud/Advanced data source.

To include technical lineage from the SAP HANA Cloud/Advanced data source, you must select this option and the Calculated Views Active option.

Note Do not select this checkbox if:

You are not getting technical lineage from Calculated views.
You want to exclude the technical lineage of this data source.

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.
Calculated Views	This query retrieves calculated views.
Dependencies of Calculated Views	This query retrieves dependencies of calculated views.
Cross-references of Calculated Views	Cross-references of Calculated Views

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

SQL Active

An option to determine whether to include or remove the technical lineage of the data source with the SQL based input.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Calculated Views Active

An option to determine whether to include or remove the technical lineage from calculated views in an SAP HANA Classic on-premises data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Use Hana Cloud for Calculated Views

An option to determine whether to include or remove the technical lineage from calculated views in an SAP HANA Cloud/Advanced data source.

To include technical lineage from the SAP HANA Cloud/Advanced data source, you must select this option and the Calculated Views Active option.

Note Do not select this checkbox if:

You are not getting technical lineage from Calculated views.
You want to exclude the technical lineage of this data source.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Snowflake

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Ingestion Method

The Snowflake ingestion methods that Collibra Data Lineage uses to ingest metadata from Snowflake data sources. Select one of the following values:

SQL: The SQL Snowflake ingestion mode. Collibra Data Lineage creates a column-level technical lineage based on SQL statements.

SQL-API: The SQL-API Snowflake ingestion mode. Collibra Data Lineage creates a column-level technical lineage based on Snowflake schemas and the access history.

For more information, go to Technical lineage for Snowflake ingestion methods.

Yes

Days

The number of days of the user access history that Collibra Data Lineage collects and processes. For example, if you set the value to 20, Collibra Data Lineage collects the last 20 days of user access history.

You can use this field to limit data retrieval from the ACCESS_HISTORY table. This field only takes effect when you use the SQL-API Snowflake ingestion mode.

Specify a value in the range of 1 - 366. If you do not enter a value, all user access history is collected by default.

Note A higher value of this field results in Collibra Data Lineage retrieving more data from Snowflake. This might cause a Usage of EmptyDir volume "output" exceeds the limit "15Gi" error when Collibra Data Lineage analyzes the metadata to create the technical lineage.

Extra Database Definitions

The name of the database from which Collibra Data Lineage collects metadata, but the database is excluded from the technical lineage that is created. This field is useful for stitching across databases. You can specify a cross-referenced database to ensure correct lineage across all databases that Collibra Data Lineage processes to create the technical lineage.

Tip You can add extra database definitions by clicking Add property.

Schema Names

The schema name of your data source. This field takes effect only when you use the SQL-API Snowflake ingestion mode. You can use this field as a filter to include lineage for objects only in the specified schema.

Ensure that the schema name you specify matches the Schema asset name that you created when you registered the data source in Data Catalog.

Tip You can add extra schema names by clicking Add property.

Source Configuration

The source configuration for the data source. Specify the following property in JSON format and enter the content in this field. This field applies only when you select the SQL-API Snowflake ingestion mode.

Property

Description

Required?

displaySampleQueries

Indicates whether to display transformations with a question mark (?) or with actual values from queries in the Source code pane in the technical lineage graph. For example, you can choose to display WHERE amount < 100 or WHERE amount < ?.

Specify one of the following values:

true: Actual values from queries are displayed.
false: A question mark (?) is displayed. This is the default value.

analyzeTemporaryTables

Indicates whether to parse the CREATE TEMPORARY TABLE statement in the ingested queries. Specify one of the following values:

true

Collibra Data Lineage examines the queries and parses the CREATE TEMPORARY TABLE statement when the following conditions are met:

The query starts with the CREATE TEMPORARY TABLE statement.
Collibra Data Lineage did not encounter the CREATE TEMPORARY TABLE statement before this query.

false

Collibra Data Lineage does not examine or parse the CREATE TEMPORARY TABLE statement in the ingested queries. This is the default value.

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

If you select the SQL Snowflake ingestion mode, the following queries apply:

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Procedures	This query retrieves the stored procedures.
Views	This query retrieves the view definitions.

If you select the SQL-API Snowflake ingestion mode, the following queries apply:

Query

Description

Object Dependencies

This query retrieves view definitions.

Columns Joined

This query retrieves table and column definition information.

If you have missing upstream lineage information, while creating technical lineage for Snowflake with the SQL-API ingestion mode, you can use this query as a workaround to fix the issue. For more information, go to the following article in the Collibra Support Portal: Snowflake SQL-API lineage missing due to frequent table replacement.

Access History

This query retrieves lineage and transformation details.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Ingestion Method

The Snowflake ingestion methods that Collibra Data Lineage uses to ingest metadata from Snowflake data sources. Select one of the following values:

SQL: The SQL Snowflake ingestion mode. Collibra Data Lineage creates a column-level technical lineage based on SQL statements.

SQL-API: The SQL-API Snowflake ingestion mode. Collibra Data Lineage creates a column-level technical lineage based on Snowflake schemas and the access history.

For more information, go to Technical lineage for Snowflake ingestion methods.

Yes

Days

You can use this field to limit data retrieval from the ACCESS_HISTORY table. This field only takes effect when you use the SQL-API Snowflake ingestion mode.

Specify a value in the range of 1 - 366. If you do not enter a value, all user access history is collected by default.

Extra Database Definitions

Tip You can add extra database definitions by clicking Add property.

Schema Names

Ensure that the schema name you specify matches the Schema asset name that you created when you registered the data source in Data Catalog.

Tip You can add extra schema names by clicking Add property.

Source Configuration

Property

Description

Required?

displaySampleQueries

Specify one of the following values:

true: Actual values from queries are displayed.
false: A question mark (?) is displayed. This is the default value.

analyzeTemporaryTables

Indicates whether to parse the CREATE TEMPORARY TABLE statement in the ingested queries. Specify one of the following values:

true

Collibra Data Lineage examines the queries and parses the CREATE TEMPORARY TABLE statement when the following conditions are met:

The query starts with the CREATE TEMPORARY TABLE statement.
Collibra Data Lineage did not encounter the CREATE TEMPORARY TABLE statement before this query.

false

Collibra Data Lineage does not examine or parse the CREATE TEMPORARY TABLE statement in the ingested queries. This is the default value.

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

If you select the SQL Snowflake ingestion mode, the following queries apply:

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Procedures	This query retrieves the stored procedures.
Views	This query retrieves the view definitions.

If you select the SQL-API Snowflake ingestion mode, the following queries apply:

Query

Description

Object Dependencies

This query retrieves view definitions.

Columns Joined

This query retrieves table and column definition information.

Access History

This query retrieves lineage and transformation details.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Spark SQL

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Select the Technical lineage capability template for your data source to create a technical lineage for the JDBC data source.

Important Technical lineage via Edge is only available in private beta. Please create a support ticket to get access.

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Spark SQL) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Object names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Spark SQL) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Object names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SQL Server

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Database Links	This query retrieves links to other databases.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Database Links	This query retrieves links to other databases.
Synonyms	This query retrieves the alternative names for the database objects.
Views	This query retrieves the view definitions.
Other Queries	This query retrieves other data that technical lineage needs, for example stored procedures, functions, and packages.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions, where you specify relevant translations for each data source. Specify the following properties in JSON format and enter the content in this field.

If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the <source ID>.conf file in this field.

Connection definition properties

Property	Description	Required?
DataSources	The parent element that contains the connection definitions of your data sources in SQL Server Integration Services. If you specify the properties in this section and also the ConnStringRegExTranslation property for a data source, the connection definitions in the ConnStringRegExTranslation property takes precedence.	No
DataSourceName	The name of your data source.	No
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.	No
collibraSystemName	The system or server name of the data source. Use this property with the `useCollibraSystemName` property in the lineage harvester configuration file to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you create when you prepare the physical data layer in Data Catalog. If you don't prepare the physical data layer, Collibra Data Lineage cannot stitch the data objects in your technical lineage to the assets in Data Catalog.	No
ConnStringRegExTranslation	The parent element that opens the connection definitions. If you specify this property and also the properties in the DataSources section for a data source, the connection definitions in this property takes precedence.	No
<regular expression>	A regular expression that must match one or more connection strings. Note Important considerations: By default, the regular expression is not case sensitive. As a consequence, a regular expression can match with connection strings containing uppercase characters or lowercase characters. The connection string is part of the SSIS connection manager. SSIS connection managers are included in an SSIS package files (DTSX) or in connection manager files (CONMGR). Example Regular expression: `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306.` Explanation: The first section, up to ., is a literal, but not case-sensitive, match of the characters. The dot (.) can match any single character. The asterisk (*) means zero or more of the previous, in this case any character. Match: Any connection string that starts with `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306`. Example: `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306;Persist Security Info=True;Auto Translate=False;`.	No
dbname	The name of your database, to which the data source connection refers.	No
schema	The name of your schema, to which the regular expression refers.	No
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.	No
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source.	No

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SQL Server Integration Services (SSIS)

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Yes

Mask

The pattern of the file names in the directory. By default, the value is *.

Source Configuration

The connection definitions, where you specify relevant translations for each data source. Specify the following properties in JSON format and enter the content in this field.

If you previously created a technical lineage for this data source with connection definitions by using the lineage harvester, you can enter the content from the <source ID>.conf file in this field.

Connection definition properties

Property	Description
ConnStringRegExTranslation	The parent element that opens the connection definitions.
<regular expression>	A regular expression that must match one or more connection strings. Note Important considerations: By default, the regular expression is not case sensitive. As a consequence, a regular expression can match with connection strings containing uppercase characters or lowercase characters. The connection string is part of the SSIS connection manager. SSIS connection managers are included in an SSIS package files (DTSX) or in connection manager files (CONMGR). Example Regular expression: `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306.` Explanation: The first section, up to ., is a literal, but not case-sensitive, match of the characters. The dot (.) can match any single character. The asterisk (*) means zero or more of the previous, in this case any character. Match: Any connection string that starts with `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306`. Example: `Server=sb-dhub;User ID=SYB_USER2;Initial Catalog=STAGEDB;Port=6306;Persist Security Info=True;Auto Translate=False;`.
dbname	The name of your database, to which the data source connection refers.
schema	The name of your schema, to which the regular expression refers.
dialect	The dialect of the referenced database. See the list of allowed values. You can enter one of the following values: `azure`, for an Azure SQL Server data source. `bigquery`, for a Google BigQuery data source. `db2`, for an IBM DB2 data source. `hana`, for an SAP HANA data source. `hana-cviews`, for getting lineage from calculated views in an SAP HANA Classic on-premises data source. `hana-cviews-v2`, for getting lineage from calculated views in an SAP HANA Cloud/Advanced data source. Important To get technical lineage including calculated views, you must harvest SAP HANA by adding two Technical Lineage for SqlDirectory capabilities with the Shared Storage connections. In one capability, specify the `hana` dialect, and in the other, specify the `hana-cviews` or `hana-cviews-v2` dialect. `hive`, for a HiveQL data source. `greenplum`, for a Greenplum data source. `mssql`, for a Microsoft SQL Server data source. `mysql`, for a MySQL data source. `netezza`, for a Netezza data source. `oracle`, for an Oracle data source. `postgres`, for a PostgreSQL data source. `redshift`, for an Amazon Redshift data source. `snowflake`, for a Snowflake data source. `spark`, for a Spark SQL data source. `sybase`, for a Sybase data source. `teradata`, for a Teradata data source.
collibraSystemName	The system or server name of the data source. Specify this property when you set the value of the Collibra system name setting to True to override the default Collibra System asset name for this data source. Specify this property with the same name as the name of the System asset that you created when you registered the data source. This property is optional.

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Sybase

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database Name

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves all columns, tables, schemas, databases or projects in the form: database or project > schema > table > column.
Views	This query retrieves the view definitions.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical lineage for Tableau

Yes

Main Properties

The required information for creating a technical lineage.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Tableau server or Tableau online account. Ingesting the same Tableau server or Tableau online account under different source IDs will fail.
Any single Tableau server or Tableau online account can be ingested only once. If you create more than one connection for the same Tableau server or Tableau online account, integration will fail. If you want to ingest from multiple unique Tableau server or Tableau online accounts, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Tableau connection

The Tableau connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Tableau.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Tableau assets.

Yes

REST only

Indication whether or not you want to use both the Tableau REST API and Tableau Metadata API to harvest Tableau metadata.

Cleared (default): The lineage harvester will use the REST API and Metadata API to harvest Tableau metadata.
Selected: The lineage harvester will only use the REST API to harvest Tableau metadata.

Note This filed must be cleared, to:

Enable technical lineage and the automatic stitching of Column assets to Tableau Data Attribute assets.
Harvest owner information for Tableau projects, workbooks and data models.

Exclude images

Indication whether or not you want to excluding the downloading of images.

Cleared: Images are downloaded.
Selected (default): Images are not downloaded.

Note The maximum number of images that can be uploaded to Collibra per day is determined by the configuration of the file upload service, in Collibra Console. For complete details, see the Upload configuration settings in DGC service configuration: options.

Site ID

The site IDs of the Tableau sites that you want to include in the ingestion process.

To ingest from multiple Tableau sites, enter each site ID in a separate Site ID field.

To ingest the default Tableau site, enter "Default" or leave the field empty. This field is not case sensitive.

Warning If you enter "Default", you must include the double quotation marks. The site IDs of any other Tableau sites must not be enclosed in double quotation marks. If the formatting of the site IDs does not conform to this detail, ingestion will fail.

Example

Tip Ensure that you specify the correct value. The correct value is the URL of the site to which you want to sign in. When you manually sign in to Tableau Server or Tableau Online, the site ID is the value that appears after /site/ in the browser address bar. In the following example URLs, the site ID is MarketingTeam:

Tableau Server: http://MyServer/#/site/MarketingTeam/projects
Tableau Online: https://10ay.online.tableau.com/#/site/MarketingTeam/workbooks

On Tableau Server, however, the URL of the default site does not specify the site. For example, the URL for a view named Profits, on a site named Sales, is http://localhost/#/site/sales/views/profits. The URL for this same view on the default site is http://localhost/#/views/profits. The site name Sales does not figure in the URL.

Yes

Site Name

The site name, or names, of the Tableau sites you specified in the Site ID field.

If you don't provide a site ID in the Site ID field, in which case the default Tableau site is ingested, leave this field blank.

Concurrency level

This field is intended to help if you are experiencing HTTP 401 Unauthorized errors due to too many concurrent HTTP calls, using the same token. It allows you to specify the internal sizing, meaning the amount of tasks that can be executed at the same time.

The default value is 10, meaning as many as 10 HTTP requests can take place in parallel. Consider reducing the value if you are experiencing HTTP 401 Unauthorized errors. Setting the value to 1 effectively disables the concurrency level, so that HTTP requests will be run in a synchronous manner, instead of in parallel.

Source configuration

This field allows you to provide JSON code for system mapping, database mapping, domain mapping and filtering.

Tip If you previously integrated Tableau via the lineage harvester, you can copy and paste in this field the JSON code from your Tableau <source ID> configuration file. For more information, go to Tableau hostname, schema, and system name mapping.

Connection definition properties

Property	Description
collibraSystemNames	This section contains the system information for different Tableau data sources. Depending on the kind of data source or connection, you have to specify how to connect to this data source. Tip For more information, see the Tableau documentation. We also recommend to check the list of supported connectors in Tableau.
files	This section contains connection information to one or more files in Tableau. Tip If you do not have files in Tableau, you can remove this section.
filePath	The full path to the file. For example, the path to a JSON file.
collibraSystemName	The system name of the file.
connectors	This section contains connection information to one or more connectors in Tableau. Tip If you do not have connectors in Tableau, you can remove this section. The values that you specify for this property are not case-sensitive.
connectorUrl	The URL of the connector. For example, the URL to Google Analytics.
collibraSystemName	The system name of the connector.
cloudFiles	This section contains connection information to one or more cloud files in Tableau's input data. Tip If you do not have cloud files in Tableau, you can remove this section.
name	The name of the file. For example, the name of a Zendesk file.
collibraSystemName	The system name of the cloud file.
hostnameMapping	This section allows you to map Tableau technical database, server and schema names to the respective real names, to preserve stitching. Warning `hostnameMapping` replaces the following deprecated properties, which have been removed from this topic: The `databaseMapping` property. The `databases` sub-section of the `collibraSystemNames` section. `hostnameMapping` must not be used in combination with either of these properties. If you use the `hostnameMapping` section, you can still use the `collibraSystemName` property in conjunction with the `files`, `connectors` or `cloudfiles` sub-sections. Example configuration "hostnameMapping": { "found_dbname=databasename1;found_hostname=*;found_schema=test": { "dbname": "mssql-database-name", "schema": "mssql-schema-name", "dialect": "mssql", "collibraSystemName": "mssql-system-name" } } For more example configurations, go to Tableau hostname, schema, and system name mapping.
found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name>	The database information of supported data sources in Tableau that is typically collected by the lineage harvester. It allows you to specify the name of the database (found_dbname), on which server a database is running (found_hostname), and optionally, the name of the schema (found_schema).
dbname	The name of the database of a supported data source in Tableau.
schema	The name of the default schema of a supported data source in Tableau. If the lineage harvester fails to find a specific schema, it uses the default schema.
dialect	The dialect of the supported data source in Tableau. You don't have to specify a dialect; it will automatically be detected. If, however, you are using a dialect that is not supported, you can use this property to specify a supported dialect that is a close comparison. That way, most of your queries will be detected and processed. Show me a list of dialects of supported data sources in Tableau. redshift, for an Amazon Redshift data source. azure, for an Azure SQL Server data source. bigquery, for a Google BigQuery data source. greenplum, for a Greenplum data source. hive, for a HiveQL data source. oracle, for an Oracle data source. postgres, for a PostgreSQL data source. mssql, for a Microsoft SQL Server data source. mysql, for a MySQL data source. netezza, for a Netezza data source. hana, for a SAP HANA data source. spark, for a Spark SQL data source. sybase, for a Sybase data source. teradata, for a Teradata data source.
filters	This section defines: From which Tableau sites and projects you want to harvest metadata. Into which domains in Collibra you want to ingest the corresponding assets. Filtering is transitive, which means that all resources in a specified project, such as Tableau workbooks and all sub-projects, are ingested. Tableau assets that are not mapped to the specified domains, for example the Tableau Server assets and the parent projects (if you specify their sub-projects), are ingested in the default domain. Important Filtering does not affect the amount of raw metadata that is harvested from Tableau and sent to the Collibra Data Lineage service instance. Rather, it determines which metadata is ingested as assets in Data Catalog. The `domainMapping` and `filters` sections are mutually exclusive. Do not include both `domainMapping` and `filters` sections in your JSON file. Tip If you want to ingest all of the projects in a Tableau site into multiple domains in Collibra, use the `domainMapping` section. If you want to ingest all of the projects in a Tableau site into the default domain, use only the `domainID` property in the lineage harvester configuration file. The `domainID` property represents the default domain. If you want to ingest all of the projects in a Tableau site into a single domain in Collibra, use site filtering. If you want to ingest metadata from only some of the projects in a Tableau site, use project filtering. You can use site filtering and project filtering together: If filtering on the same site, this "filtering" is actually domain mapping, because nothing is filtered out. The contents of the projects are ingested in the specified domains, and the rest of the contents of the site are ingested in a different, specified domain. If you are site filtering on a specific site and project filtering a different site, then site filtering is again a form of domain mapping, and the filtered projects are ingested in their specified domains. If your lineage harvester configuration file includes sites that are not mentioned in the `filters` section of your <source ID> configuration file, those sites are ingested in the default domain.
sites	The Tableau sites to be ingested and the domain in which you want to ingest metadata from the Tableau sites. Tip If you have only one Tableau site, do not include a `sites` section in your <source ID> file. Instead, use a `projects` section, to filter on Tableau projects. Include a `sites` section only if all of the following are true: You have more than one Tableau site. You want to ingest all of the metadata from only one Tableau site into a single domain in Collibra. The domain into which you want to ingest is not the default domain, meaning the domain specified in the `domainId` property in your lineage harvester configuration file.
site_name: domain_id	`site_name` The name of the site to be ingested. The site name is case-sensitive. `domain_id` The unique reference ID of the domain in Collibra in which you want to ingest metadata. The domain ID is case-sensitive. To ingest all metadata from a Tableau site in the specified domain, specify the site name and a separate domain ID for each site that you list on the `siteIds` property in the lineage harvester configuration file for Tableau. If the `site_name` or `domain_id` property is not specified for a site, the metadata from the site is ingested in the default domain. How do I find a domain reference ID? Open the relevant domain in Collibra. The URL looks like: https://<yourcollibrainstance>/domain/22258f64-40b6-4b16-9c08-c95f8ec0da26?view=00000000-0000-0000-0000-000000040001. In this example, the reference ID is in bold. Show me the example { "filters":{ "sites":{ "Training":"ca60b822-781b-4b3a-b44d-f65bd107ff92" }, "projects":{ "Testing > Databricks":"e8f4e4a8-4062-4a33-9b44-3ce3e18e4e22", "Product Demo > Customer Insights":"a305e6f7-7a49-49aa-aa85-41b1e689121b" } } }
projects	The Tableau projects to be ingested and the domain in which you want to ingest metadata from the Tableau projects or sub-projects. Tip Project filtering is not relevant for those who have an Explorer role in Tableau, because Explorers need to configure permissions for each data object in Tableau that they want to ingest. As the Administrator role has access to all data objects, project filtering allows Administrators to specify which projects to ingest.
site_name > project_name : domain_id	The `site_name` should be the Tableau site name. The `project_name` should be the Tableau project name. The `domain_id` should be the unique reference ID of the domain in Collibra in which you want to ingest metadata. When you specify the site and project names, the following rules apply: Add spaces before and after >. The spaces are separators between the site and project. Specify the full exact site and project names. The values are case-sensitive. When you specify a Tableau project, all assets in the project are ingested in the specified domain. If you want to ingest assets from different Tableau projects in one domain, you can specify the same value for `domain id` for different projects. Example `"Collibra_tab_partner_site > JB_Test_2812": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
site_name > project_name > sub-project_name : domain_id	The `site_name` should be the Tableau site name. The `project_name` should be the Tableau project name. Optionally, use `sub-project_name` to specify the Tableau sub-project name. The `domain_id` property should be the unique reference ID of the domain in Collibra in which you want to ingest metadata. When you specify the site, project and sub-project names, the following rules apply: Add spaces before and after >. The spaces are separators between the site and project. Specify the full exact site and project names. The values are case-sensitive. Example `"Collibra_tab_partner_site > JB_Test_2812 > ProjectJJ2": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
domainMapping	This section defines in which domains in Collibra you want to ingest assets from your Tableau sites and Tableau projects. Domain mapping is transitive, meaning that all resources, such as Tableau workbooks and data attributes in a parent Tableau site, project or sub-project, are ingested in the same domain as the parent. Important The `domainMapping` and `filters` sections are mutually exclusive. Do not include both `domainMapping` and `filters` sections in your JSON file. Tip If you want to ingest all of the projects in a Tableau site into multiple domains in Collibra, use this `domainMapping` section. If you want to ingest all of the projects in a Tableau site into the default domain, use only the `domainID` property in the lineage harvester configuration file. The `domainID` property represents the default domain. Note Tableau assets that are not mapped to specific domains via this `domainMapping` section, for example Tableau Server assets, are ingested in that default domain. If you want to ingest all of the projects in a Tableau site into a single domain in Collibra, use site filtering. If you want to ingest metadata from only some of the projects in a Tableau site, use project filtering. Show me an example Let's say that you have a Tableau site named "Site-1". You want to ingest all Tableau projects in "Site-1" in a domain named "Domain-1" in Collibra, with the exception of one Tableau project named "Project-Default", which you want to ingest in "Domain-2". You should configure the `domainMapping` section as follows. "domainMapping": { "<Site-1>": "reference-id-of-Domain-1", "<Site-1> > <Project-Default>": "reference-id-of-Domain-2" } If you want to specify a domain for a sub-project of "Project-Default", use the `<site name> > <project name> > <sub-project name>` property, as described below. Tip For the properties in this `domainMapping` section, ensure that you maintain the spaces before and after "`>`", for example `"Site-1 > Project-Default"`. The spaces serve as a separator between the site and the projects.
site name	The Tableau site name, followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau site. Important In the configuration file, use the actual site name, along with the domain reference ID, for example: `"Collibra_tab_partner_site": "afc8cfb0-91f1-4075-a3e5-7ce6d1f9bcc9"`
site name > project name	The Tableau project name, preceded by the name of the Tableau site to which it belongs, and followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau project. Important In the configuration file, use the actual site and project names, along with the domain reference ID, for example: `"Collibra_tab_partner_site > JB_Test_2812": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
site name > project name > sub-project name	The Tableau sub-project name, preceded by the name of the Tableau site and project to which it belongs, and followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau sub-project. Important In the configuration file, use the actual site, project and sub-project names, along with the domain reference ID, for example: `"Collibra_tab_partner_site > JB_Test_2812 > ProjectJJ2": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`

Example

Custom Properties

This section identifies to which Collibra Data Lineage service instance you want to upload the Tableau metadata.

Warning This applies only for Collibra Cloud for Government customers.

techlinKey

The unique API key to connect to the Collibra Data Lineage service instance.

A unique user key is needed for each Collibra environment. If you're not sure what your user key is, please contact your Collibra Customer Success Manager.

Warning This applies only for Collibra Cloud for Government customers.

techlinHost

The URL of he Collibra Data Lineage service instance to which you want to upload Tableau metadata, for example "techlin-gcp-eu.collibra.com".

Warning This applies only for Collibra Cloud for Government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Paging

This section allows you to customize the Tableau API pagination settings.

The default values are sufficient in most cases; however, you can decrease them to help mitigate node limit errors, or increase them to speed up API calls.

If the integration fails because of timeout errors due to page sizing limits, Collibra Data Lineage automatically adjusts the limits and retries. For example, if failure occurs with worksheetsPageSize set to 100, the value is automatically reduced to 50 and another integration attempt is automatically started. If it fails again, the value is again halved. If integration is still unsuccessful with an adjusted value of 1, an error is thrown and no further attempts are started. If integration is eventually successful, the page size value is restored to its original value, in this example 100, for the next synchronization.

The complete list of pagination settings, descriptions and default values

"paging": {
	"databasesPageSize": 100,
	"tablesPageSize": 100,
	"tablesColumnsPageSize": 100,
	"tableColumnsPageSize": 1000,
	"datasourcesPageSize": 50,
	"datasourcesFieldsPageSize": 50,
	"datasourceFieldsPageSize": 100,
	"worksheetsPageSize": 100,
	"worksheetsFieldsPageSize": 100,
	"worksheetFieldsPageSize": 1000,
	"usersPageSize": 100,
	"dashboardsPageSize": 100,
	"columnsLimit": 20,
	"fieldsLimit": 20
	}

Settings per metadata type and descriptions

Metadata type	Setting and description
Dashboard	`dashboardsPageSize`: The number of dashboards per page.
Worksheet	`worksheetsPageSize`: The number of worksheets per page. `worksheetsFieldsPageSize`: The number of worksheet fields per page.
Database	`databasesPageSize`: The number of databases per page.
Table	`tablesPageSize`: The number of tables per page. `tablesColumnsPageSize`: The number of table columns per page.
Table columns	`tableColumnsPageSize`: The number of table columns per page.
Users	`usersPageSize`: The number of users per page.
Data source	`datasourcesPageSize`: The number of data sources per page. `datasourcesFieldsPageSize`: The number of data source fields per page. `columnsLimit`: The number of data source field columns per page. `fieldsLimit` : The number of referenced data source fields per page.
Data source field	`datasourceFieldsPageSize`: The number of data source fields per page. `columnsLimit`: The number of data source field columns per page. `fieldsLimit` : The number of referenced data source fields per page.

Logging

This section contains the properties for debug logging. This setting is not valid for this integration.

Debug

This setting is not valid for this integration. It should be set to false.

Field Description Required

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Source ID

The name of the data source. You can give this any name, as long as it is unique.

Warning

You can only specify one source ID per Tableau server or Tableau online account. Ingesting the same Tableau server or Tableau online account under different source IDs will fail.
Any single Tableau server or Tableau online account can be ingested only once. If you create more than one connection for the same Tableau server or Tableau online account, integration will fail. If you want to ingest from multiple unique Tableau server or Tableau online accounts, you have to create a new Edge connection for each one, configure a new capability template for each one, and each must have a unique source ID.

Warning If you are switching between the lineage harvester and Edge, the value in this field must exactly match the value of the id property in your lineage harvester configuration file.

Yes

Tableau connection

The Tableau connection that you created for ingestion in Data Catalog.

Tip Select the name that you provided in the Name field when you created a connection to Tableau.

Yes

Domain ID

The unique reference ID of the domain in Collibra Data Intelligence Platform in which you want to ingest the Tableau assets.

Yes

REST only

Indication whether or not you want to use both the Tableau REST API and Tableau Metadata API to harvest Tableau metadata.

Cleared (default): The lineage harvester will use the REST API and Metadata API to harvest Tableau metadata.
Selected: The lineage harvester will only use the REST API to harvest Tableau metadata.

Note This filed must be cleared, to:

Enable technical lineage and the automatic stitching of Column assets to Tableau Data Attribute assets.
Harvest owner information for Tableau projects, workbooks and data models.

Exclude images

Indication whether or not you want to excluding the downloading of images.

Cleared: Images are downloaded.
Selected (default): Images are not downloaded.

Site ID

The site IDs of the Tableau sites that you want to include in the ingestion process.

To ingest from multiple Tableau sites, enter each site ID in a separate Site ID field.

To ingest the default Tableau site, enter "Default" or leave the field empty. This field is not case sensitive.

Example

Tableau Server: http://MyServer/#/site/MarketingTeam/projects
Tableau Online: https://10ay.online.tableau.com/#/site/MarketingTeam/workbooks

Yes

Site Name

The site name, or names, of the Tableau sites you specified in the Site ID field.

If you don't provide a site ID in the Site ID field, in which case the default Tableau site is ingested, leave this field blank.

Concurrency level

Source configuration

This field allows you to provide JSON code for database mapping, domain mapping and filtering.

If you previously integrated Tableau via the lineage harvester, you can copy and paste in this field the JSON code from your Tableau <source ID> configuration file.

Connection definition properties

Property	Description
collibraSystemNames	This section contains the system information for different Tableau data sources. Depending on the kind of data source or connection, you have to specify how to connect to this data source. Tip For more information, see the Tableau documentation. We also recommend to check the list of supported connectors in Tableau.
files	This section contains connection information to one or more files in Tableau. Tip If you do not have files in Tableau, you can remove this section.
filePath	The full path to the file. For example, the path to a JSON file.
collibraSystemName	The system name of the file.
connectors	This section contains connection information to one or more connectors in Tableau. Tip If you do not have connectors in Tableau, you can remove this section. The values that you specify for this property are not case-sensitive.
connectorUrl	The URL of the connector. For example, the URL to Google Analytics.
collibraSystemName	The system name of the connector.
cloudFiles	This section contains connection information to one or more cloud files in Tableau's input data. Tip If you do not have cloud files in Tableau, you can remove this section.
name	The name of the file. For example, the name of a Zendesk file.
collibraSystemName	The system name of the cloud file.
hostnameMapping	This section allows you to map Tableau technical database, server and schema names to the respective real names, to preserve stitching. Warning `hostnameMapping` replaces the following deprecated properties, which have been removed from this topic: The `databaseMapping` property. The `databases` sub-section of the `collibraSystemNames` section. `hostnameMapping` must not be used in combination with either of these properties. If you use the `hostnameMapping` section, you can still use the `collibraSystemName` property in conjunction with the `files`, `connectors` or `cloudfiles` sub-sections. Example configuration "hostnameMapping": { "found_dbname=databasename1;found_hostname=*;found_schema=test": { "dbname": "mssql-database-name", "schema": "mssql-schema-name", "dialect": "mssql", "collibraSystemName": "mssql-system-name" } } For more example configurations, go to Tableau hostname, schema, and system name mapping.
found_dbname=<database name>;found_hostname=<server name>;found_schema=<schema name>	The database information of supported data sources in Tableau that is typically collected by the lineage harvester. It allows you to specify the name of the database (found_dbname), on which server a database is running (found_hostname), and optionally, the name of the schema (found_schema).
dbname	The name of the database of a supported data source in Tableau.
schema	The name of the default schema of a supported data source in Tableau. If the lineage harvester fails to find a specific schema, it uses the default schema.
dialect	The dialect of the supported data source in Tableau. You don't have to specify a dialect; it will automatically be detected. If, however, you are using a dialect that is not supported, you can use this property to specify a supported dialect that is a close comparison. That way, most of your queries will be detected and processed. Show me a list of dialects of supported data sources in Tableau. redshift, for an Amazon Redshift data source. azure, for an Azure SQL Server data source. bigquery, for a Google BigQuery data source. greenplum, for a Greenplum data source. hive, for a HiveQL data source. oracle, for an Oracle data source. postgres, for a PostgreSQL data source. mssql, for a Microsoft SQL Server data source. mysql, for a MySQL data source. netezza, for a Netezza data source. hana, for a SAP HANA data source. spark, for a Spark SQL data source. sybase, for a Sybase data source. teradata, for a Teradata data source.
filters	This section defines: From which Tableau sites and projects you want to harvest metadata. Into which domains in Collibra you want to ingest the corresponding assets. Filtering is transitive, which means that all resources in a specified project, such as Tableau workbooks and all sub-projects, are ingested. Tableau assets that are not mapped to the specified domains, for example the Tableau Server assets and the parent projects (if you specify their sub-projects), are ingested in the default domain. Important Filtering does not affect the amount of raw metadata that is harvested from Tableau and sent to the Collibra Data Lineage service instance. Rather, it determines which metadata is ingested as assets in Data Catalog. The `domainMapping` and `filters` sections are mutually exclusive. Do not include both `domainMapping` and `filters` sections in your JSON file. Tip If you want to ingest all of the projects in a Tableau site into multiple domains in Collibra, use the `domainMapping` section. If you want to ingest all of the projects in a Tableau site into the default domain, use only the `domainID` property in the lineage harvester configuration file. The `domainID` property represents the default domain. If you want to ingest all of the projects in a Tableau site into a single domain in Collibra, use site filtering. If you want to ingest metadata from only some of the projects in a Tableau site, use project filtering. You can use site filtering and project filtering together: If filtering on the same site, this "filtering" is actually domain mapping, because nothing is filtered out. The contents of the projects are ingested in the specified domains, and the rest of the contents of the site are ingested in a different, specified domain. If you are site filtering on a specific site and project filtering a different site, then site filtering is again a form of domain mapping, and the filtered projects are ingested in their specified domains. If your lineage harvester configuration file includes sites that are not mentioned in the `filters` section of your <source ID> configuration file, those sites are ingested in the default domain.
sites	The Tableau sites to be ingested and the domain in which you want to ingest metadata from the Tableau sites. Tip If you have only one Tableau site, do not include a `sites` section in your <source ID> file. Instead, use a `projects` section, to filter on Tableau projects. Include a `sites` section only if all of the following are true: You have more than one Tableau site. You want to ingest all of the metadata from only one Tableau site into a single domain in Collibra. The domain into which you want to ingest is not the default domain, meaning the domain specified in the `domainId` property in your lineage harvester configuration file.
site_name: domain_id	`site_name` The name of the site to be ingested. The site name is case-sensitive. `domain_id` The unique reference ID of the domain in Collibra in which you want to ingest metadata. The domain ID is case-sensitive. To ingest all metadata from a Tableau site in the specified domain, specify the site name and a separate domain ID for each site that you list on the `siteIds` property in the lineage harvester configuration file for Tableau. If the `site_name` or `domain_id` property is not specified for a site, the metadata from the site is ingested in the default domain. How do I find a domain reference ID? Open the relevant domain in Collibra. The URL looks like: https://<yourcollibrainstance>/domain/22258f64-40b6-4b16-9c08-c95f8ec0da26?view=00000000-0000-0000-0000-000000040001. In this example, the reference ID is in bold. Show me the example { "filters":{ "sites":{ "Training":"ca60b822-781b-4b3a-b44d-f65bd107ff92" }, "projects":{ "Testing > Databricks":"e8f4e4a8-4062-4a33-9b44-3ce3e18e4e22", "Product Demo > Customer Insights":"a305e6f7-7a49-49aa-aa85-41b1e689121b" } } }
projects	The Tableau projects to be ingested and the domain in which you want to ingest metadata from the Tableau projects or sub-projects. Tip Project filtering is not relevant for those who have an Explorer role in Tableau, because Explorers need to configure permissions for each data object in Tableau that they want to ingest. As the Administrator role has access to all data objects, project filtering allows Administrators to specify which projects to ingest.
site_name > project_name : domain_id	The `site_name` should be the Tableau site name. The `project_name` should be the Tableau project name. The `domain_id` should be the unique reference ID of the domain in Collibra in which you want to ingest metadata. When you specify the site and project names, the following rules apply: Add spaces before and after >. The spaces are separators between the site and project. Specify the full exact site and project names. The values are case-sensitive. When you specify a Tableau project, all assets in the project are ingested in the specified domain. If you want to ingest assets from different Tableau projects in one domain, you can specify the same value for `domain id` for different projects. Example `"Collibra_tab_partner_site > JB_Test_2812": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
site_name > project_name > sub-project_name : domain_id	The `site_name` should be the Tableau site name. The `project_name` should be the Tableau project name. Optionally, use `sub-project_name` to specify the Tableau sub-project name. The `domain_id` property should be the unique reference ID of the domain in Collibra in which you want to ingest metadata. When you specify the site, project and sub-project names, the following rules apply: Add spaces before and after >. The spaces are separators between the site and project. Specify the full exact site and project names. The values are case-sensitive. Example `"Collibra_tab_partner_site > JB_Test_2812 > ProjectJJ2": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
domainMapping	This section defines in which domains in Collibra you want to ingest assets from your Tableau sites and Tableau projects. Domain mapping is transitive, meaning that all resources, such as Tableau workbooks and data attributes in a parent Tableau site, project or sub-project, are ingested in the same domain as the parent. Important The `domainMapping` and `filters` sections are mutually exclusive. Do not include both `domainMapping` and `filters` sections in your JSON file. Tip If you want to ingest all of the projects in a Tableau site into multiple domains in Collibra, use this `domainMapping` section. If you want to ingest all of the projects in a Tableau site into the default domain, use only the `domainID` property in the lineage harvester configuration file. The `domainID` property represents the default domain. Note Tableau assets that are not mapped to specific domains via this `domainMapping` section, for example Tableau Server assets, are ingested in that default domain. If you want to ingest all of the projects in a Tableau site into a single domain in Collibra, use site filtering. If you want to ingest metadata from only some of the projects in a Tableau site, use project filtering. Show me an example Let's say that you have a Tableau site named "Site-1". You want to ingest all Tableau projects in "Site-1" in a domain named "Domain-1" in Collibra, with the exception of one Tableau project named "Project-Default", which you want to ingest in "Domain-2". You should configure the `domainMapping` section as follows. "domainMapping": { "<Site-1>": "reference-id-of-Domain-1", "<Site-1> > <Project-Default>": "reference-id-of-Domain-2" } If you want to specify a domain for a sub-project of "Project-Default", use the `<site name> > <project name> > <sub-project name>` property, as described below. Tip For the properties in this `domainMapping` section, ensure that you maintain the spaces before and after "`>`", for example `"Site-1 > Project-Default"`. The spaces serve as a separator between the site and the projects.
site name	The Tableau site name, followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau site. Important In the configuration file, use the actual site name, along with the domain reference ID, for example: `"Collibra_tab_partner_site": "afc8cfb0-91f1-4075-a3e5-7ce6d1f9bcc9"`
site name > project name	The Tableau project name, preceded by the name of the Tableau site to which it belongs, and followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau project. Important In the configuration file, use the actual site and project names, along with the domain reference ID, for example: `"Collibra_tab_partner_site > JB_Test_2812": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`
site name > project name > sub-project name	The Tableau sub-project name, preceded by the name of the Tableau site and project to which it belongs, and followed by the unique reference ID of the domain in Collibra in which you want to ingest resources from the Tableau sub-project. Important In the configuration file, use the actual site, project and sub-project names, along with the domain reference ID, for example: `"Collibra_tab_partner_site > JB_Test_2812 > ProjectJJ2": "d224a1a5-43b4-43b2-8df0-ddf8f2726b82"`

Example

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Paging

This sectionproperty allows you to customize the Tableau API pagination settings.

The default values are sufficient in most cases; however, you can decrease them to help mitigate node limit errors, or increase them to speed up API calls.

The complete list of pagination settings, descriptions and default values

"paging": {
	"databasesPageSize": 100,
	"tablesPageSize": 100,
	"tablesColumnsPageSize": 100,
	"tableColumnsPageSize": 1000,
	"datasourcesPageSize": 50,
	"datasourcesFieldsPageSize": 50,
	"datasourceFieldsPageSize": 100,
	"worksheetsPageSize": 100,
	"worksheetsFieldsPageSize": 100,
	"worksheetFieldsPageSize": 1000,
	"usersPageSize": 100,
	"dashboardsPageSize": 100,
	"columnsLimit": 20,
	"fieldsLimit": 20
	}

Settings per metadata type and descriptions

Metadata type	Setting and description
Dashboard	`dashboardsPageSize`: The number of dashboards per page.
Worksheet	`worksheetsPageSize`: The number of worksheets per page. `worksheetsFieldsPageSize`: The number of worksheet fields per page.
Database	`databasesPageSize`: The number of databases per page.
Table	`tablesPageSize`: The number of tables per page. `tablesColumnsPageSize`: The number of table columns per page.
Table columns	`tableColumnsPageSize`: The number of table columns per page.
Users	`usersPageSize`: The number of users per page.
Data source	`datasourcesPageSize`: The number of data sources per page. `datasourcesFieldsPageSize`: The number of data source fields per page. `columnsLimit`: The number of data source field columns per page. `fieldsLimit` : The number of referenced data source fields per page.
Data source field	`datasourceFieldsPageSize`: The number of data source fields per page. `columnsLimit`: The number of data source field columns per page. `fieldsLimit` : The number of referenced data source fields per page.

Debug

This setting is not valid for this integration. It should be set to false.

Log level

This setting is not valid for this integration. It should be set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for Teradata

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Teradata) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Object Names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Logging

This section contains general information about logging.

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

JDBC Connection

The JDBC connection that you created for Catalog JDBC ingestion.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

External Database Name

CData, which CDATA drivers returned as a placeholder. Use this value if you did not create a custom database name by using the CustomizedDefaultCatalogName property when you registered your data source.
The custom database name that you specified for the CustomizedDefaultCatalogName property when you registered your data source.

Database Name

The name of the database or schema (these terms are synonymous for Teradata) from which you want to harvest metadata.

Yes

Queries

Note

If you change queries, you can only use supported SQL syntax.
Collibra Support does not provide support for customized SQL files.

Query	Description
Columns	This query retrieves the columns, tables, schemas, databases or projects fields in the form: database or project > schema > table > column.
Object Names	This query retrieves a list of object names from which technical lineage can be created. The objects can include stored procedures, views, macros, and so on.

Yes

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Debug

Select one of the following values:

True: Enables logging of the JDBC job.
False: Disables logging of the JDBC job. This is the default value.

Log level

An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.

Field Description Required?

Capability

This section contains general information about the capability.

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Main Properties

This section contains the information for creating a technical lineage.

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Advanced Properties

This section contains the advanced properties for creating a technical lineage.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field Description Required?

Name

The name of the Edge capability.

Yes

Description

The description of the Edge capability.

Yes

Capability template

The capability template. The value that you select in this field determines which sections appear on the page.

Select the following Edge capability:

Technical Lineage for SqlDirectory

Yes

Source ID

The name of the data source. Specify a name that is unique.

Yes

Shared Storage Connection

The Shared Storage connection that you created.

Mask

The pattern of the file names in the directory. By default, the value is *.

Yes

Dialect

The dialect of the database.

Yes

Collibra System Name

The system or server name of the data source. This field is also the full name of your System asset in Data Catalog.

The value of this field must be the same as the full name of the System asset that you created when you registered the data source.

Yes

Database

The name of your database, which is also the name of your Database asset in Data Catalog.

Yes

Schema

The name of the default schema, if not specified in the data source itself. This corresponds to the name of your Schema asset.

Yes

Database-System mapping

Property

This section contains the custom parameters you can specify to create technical lineage. Click Add property to add a property.

You can use this field to set the HTTP timeout duration by adding the httpTimeout property:

httpTimeout

Name	Description	Type	Encryption	Example value
httpTimeout	Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	Text	Not encrypted (plain text)	15

Type	Value Type	Name	Example value
Text	Plaintext	httpTimeout Sets the HTTP timeout duration, in seconds. You can enter a value in the range of 1 to 3599. The default value is 15.	15

Warning If you are a Collibra Cloud for Government customer, this field is required to connect to a Collibra Data Lineage service instance:

Properties for Collibra Cloud for Government customers

Name Description Type Encryption Example value

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Text

Not encrypted (plain text)

https://techlin-gcp-dev.collibra.com

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

Text

To be encrypted by Edge management server

Value will be stored on Edge

Type Value Type Name Value

Text

Plaintext

techlinHost

This is the URL of the Collibra Data Lineage service instance to which you want to upload metadata, for example techlin-gcp-eu.collibra.com.

Example: https://techlin-gcp-dev.collibra.com

Text

Secret

techlinKey

This is the unique API key to connect to a Collibra Data Lineage service instance.

Specify a unique user key for each Collibra environment. If you're not sure what your user key is, contact your Collibra Customer Success Manager.

<your-techlin-key>

Yes for US government customers.

Dependent On Sources

To use this option, enter the source ID of the independent source.

For complete information, go to Sharing database models across data sources.

Delete Raw Metadata After Processing

Select this option to indicate that the raw source metadata is deleted after processing.

Clear the checkbox to keep the raw source metadata after processing. In this case, it is stored in the Collibra infrastructure.

Analyze Only (Deprecated)

Important This option is deprecated and will be removed in a future version of Collibra. We recommend that you no longer use this option and, instead, start using the Processing Level setting, below.

The "Analyze" option in the Processing Level setting is the equivalent of selecting the Analyze Only option.
The "Sync" option in the Processing Level setting is the equivalent of clearing the Analyze Only option.

This option determines whether or not synchronization with Data Catalog begins, or not, after metadata is successfully analyzed:

If the Analyze Only option is selected, synchronization does not start after analysis. This allows you to synchronize all data sources in a single job.
If the Analyze Only option is cleared, synchronization starts – or is queued, if another synchronization job is running -- immediately after analysis.

Selecting this option is equivalent to entering the load-sources and analyze command with a source specified when you use the lineage harvester.

This option is not enabled by default.

Processing Level

Important This setting replaces the deprecated Analyze Only option, which will be removed in a future version of Collibra.

Select one of the following values:

Value

Description

Load

Analyze

If you synchronize multiple data sources, this option allows you to synchronize all of them in a single job.

Sync

Load, analyze, and synchronize metadata from all data sources. Synchronization starts – or is queued, if another synchronization job is running – immediately after analysis.

Yes

Active

The option determines whether to include or remove the technical lineage of the data source.

Select this option to include the technical lineage of this data source.

Clear the checkbox to exclude the technical lineage of this data source.

Yes

Field	Description	Required
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Collibra Protect for AWS Lake Formation`	Yes
AWS Lake Formation Connection	The AWS Lake Formation connection to connect to AWS Lake Formation.	Yes

Field	Description	Required
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Collibra Protect for BigQuery`	Yes
GCP Connection	The GCP connection to connect to Google Cloud Platform.	Yes

Field	Description	Required
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Collibra Protect for Databricks`	Yes
JDBC Connection	The JDBC connection to connect to Databricks.	Yes

Field	Description	Required
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `Collibra Protect for Snowflake`	Yes
JDBC Connection	The JDBC connection to connect to Snowflake.	Yes
Snowflake role testing	An option that determines how Snowflake checks roles (that is, Protect groups) for applying data protection standards and data access rules. This is to accommodate Snowflake users who have multiple roles. This field contains the following options: CURRENT_ROLE: Checks only the primary role assigned to the Snowflake user. This is the default option. IS_ROLE_IN_SESSION: Checks all the roles assigned to the Snowflake user, including secondary roles, within the active session.	Yes

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
Capability template	The capability template. The value that you select in this field determines which sections appear on the page. Select the following Edge capability: `SAP Datasphere synchronization`	Yes
Connection
SAP Connection	The SAP Datasphere Catalog connection to be used. Tip If you already integrated SAP Analytics Cloud, you can also reuse its SAP Datasphere Catalog connection.	Yes
Configuration	This section contains information on how to connect to SAP Datasphere.
Save input metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. If this option is selected, you can download the files from the Synchronization Result dialog box once the synchronization activity is completed.	No
Advanced Configuration Logging configuration Memory JVM arguments	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This setting is not valid for this integration. It should be set to false.	No
Log level	This setting is not valid for this integration. It should be set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
GCP service account	This section contains information on how to connect to Google Cloud Storage.
GCP Connection	The GCP connection to be used.	Yes
Configuration	This section contains information on the configuration of the crawlers.
Project IDs (Deprecated)	Add a comma-separated list of the Project IDs where Dataplex is enabled. This field is deprecated in the latest user interface and replaced by the Project IDs field on the Synchronize Metadata page. You can add the Project IDs when you synchronize Google Dataplex. The following rules apply when you add Project IDs: If you enter a value in this field and do not add Project IDs on the Synchronize Metadata page, the capability will search in these projects in this field when you synchronize the capability. If you leave this field empty and do not add project IDs on the Synchronize Metadata page, the capability will search in the projects that you entered in the GCP Service Account field in the GCP connection. Do not enter a value in this field and also add Project IDs on the Synchronize Metadata page; otherwise, the synchronization will end with an error when you synchronize the capability.	No
Save input metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. The Collibra Support team can provide the location of the saved ZIP files after the synchronization. This checkbox is not selected by default.	No
(deprecated) Filters and Domain Mapping	Important This field is deprecated in the latest UI. You can now define the mappings in the integration configuration. If you have existing mappings here, they will continue to work. However, we advise you to move them to the integration configuration. Text in JSON format to include or exclude lakes and zones, and to configure domain mappings. The text must be in JSON format and can contain an include and an exclude block. In the include block, You can specify the domain in which specific lakes or zones must be ingested. The format is: `“project ID> lake ID> zone ID”: “domain ID”`. For example, `"integrations-automated-uer > testlake> testzone": "c8fe882a-a12e-4284-b655-7ac2a4fb08cb`. You can also specify the domain in which specific tables and columns must be ingested. The format is `"project ID> lake ID > zone ID > table ID":"domain ID"` In the exclude block, you can specify the lakes or zones that you don't want to ingest. For example, `"* > test"`. The exclude block has priority over the include block. If the include block is not present, we ingest all assets into the same domain as the System asset. If there is no explicit domain mapping for a zone, we use the domain specified for the Lake. You can use the keyword `default` as a domain ID. In that case, the lake or zone will be ingested in the same domain as the System asset. A match with a lake has priority over a match with a zone. The integration fails before the synchronization starts, if one or more domain IDs specified in the include block don't exist. The integration fails before the synchronization starts if a domain ID is left empty in the include block. You can use the ? and * wildcards in the zone and lake names. If a lake or zone matches multiple lines, the most detailed match is taken into account. If you registered the BigQuery data source via the BigQuery JDBC connector, and then integrate Google Dataplex, assets will be ingested in the same domains that were registered during JDBC ingestion. Specifically, Project assets are registered in the Database domains, and Zone assets are registered in the Schema domains. The mapping created by JDBC ingestion takes priority over the configurations in this field. In this way, no duplicated tables or columns are created. For more information, go to Ways to work with Google Cloud Platform (GCP). Examples Example 1 Ingest assets from Projects and Lakes to separate domains { "include": { "integrations-automated-uer > testlake": "20000000-0000-0000-0000-000000000000", "integrations-automated-uer > testlake > us-east-1":"8c09099c-2e89-4d06-a880-464172b7767e", "integrations-automated-uer > ay-west-1": "e274aed6-fdd7-40bb-80d0-2ae27848855d", "integrations-automated-uer > vb-mapping-test-1": "default" }, "exclude": [ integrations-automated-uer > testlake > testzone" ] } In this example: Assets from Project ID "integrations-automated-uer" and Lake ID "testlake" will be ingested into the Collibra domain with ID "20000000-0000-0000-0000-000000000000". However, all assets from the "us-east-1" zone will be ingested into the domain with ID "8c09099c-2e89-4d06-a880-464172b7767e". All assets from Project ID "integrations-automated-uer" and Lake ID "ay-west-1" will be ingested into the domain with ID "e274aed6-fdd7-40bb-80d0-2ae27848855d". All assets from Project ID "integrations-automated-uer" and Lake ID "vb-mapping-test-1" will be ingested in the same domain as the System asset. All assets from Zone ID "testzone” in Project ID "integrations-automated-uer" and Lake ID "testlake" will be excluded. Example 2 Ingest assets from Projects, Lakes, and Zones to separate domains { "include": { "project": "project-domain-id", "project > datalake ": "lake-domain-id", "project > datalake > zone1": "domain-id-1", "project > datalake > zone2": "domain-id-2", "project > datalake > zone1 > *": "table-column-domain-id" }, "exclude": [ "project > datalake > zone3" ] } In this example: The GCP Project, Dataplex Lake, Zone 1, and Zone 2 will be ingested in separate domains from each other. Zone 1 will be ingested in the `zone-domain-id-1` domain, and the table and columns from Zone 1 will be ingested in the `table-column-domain-id` domain. If you use this strategy, ensure to exclude any lakes and zones that you don't want to ingest. Example 3 and 4 Ingest assets from Tables to a separate domain If your Dataplex has the following structure, JSON example 3 and 4 show different ways of filtering: Dataplex structure project-1 -lake-1 zone-1 table-1 table-2 zone-2 table-3 -lake-2 zone-3 table-4 zone-4 table-5 project-2 -lake-3 table-7 Example 3 This JSON example ensures that only the hierarchy of `table 1` will be ingested in the`table-column-domain-id` domain, and no exclude block is needed: { "include": { "project-1 > lake-1 > zone1 > table-1": "table-column-domain-id" }, } Assets in Data Catalog resulting from example 3 This structure shows the assets in Data Catalog resulting from the configuration in Example 3: project-1 : system-domain(default) -lake-1 : system-domain(default) zone-1 : system-domain(default) table-1 : table-column-domain Example 4 This JSON example ensures that the `table 1` Table is ingested in a different domain from the other assets in the Project: { "include": { "project-1 > "project-domain-id", "project-1 > lake-1 > zone1 > table-1": "table-column-domain-id" }, } Assets in Data Catalog resulting from example 4 This structure shows the assets in Data Catalog resulting from the configuration in Example 4: project-1 : project-domain -lake-1 : project-domain zone-1 : project-domain table-1 : table-column-domain table-2 : project-domain zone-2 : project-domain table-3 : project-domain -lake-2 : project-domain zone-3 : project-domain table-4 : project-domain zone-4 : project-domain table-5 : project-domain Example 5 Integrate Google Dataplex with Google BigQuery { "include": { "integrations-automated-uer > testBigQuerylake > us-east-1":"10098d38-3e53-463d-8e4b-0b93d308a8ea" } } In this example, all assets in the us-east-1 Dataplex Zone, including the Schema, Table, and Column assets, will be ingested in the `10098d38-3e53-463d-8e4b-0b93d308a8ea` domain.	No
Extensible Properties Mapping	Via the Extensible Properties Mapping field, you can integrate additional properties from Dataplex: Table creation date, Table modified date, System (showing where the table comes from), and type (the Zone type). Important If you use this feature, make sure to set up all required characteristic assignments for the asset types. You do this by adding the mapping between the fields for the objects in Dataplex and the Collibra attribute IDs to ingest the data in, using a JSON string. The text must be in JSON format and can contain a Zones and Tables block. In each block, you specify the property name and the attribute ID to which you want to map the value in the property. The format is: `"[property name]": "[attribute resource ID]"`. For example, `"system": "19a27fda-8c50-48a8-87b3-f275ad450fe5"`. Example { "tables": { "system": "19a27fda-8c50-48a8-87b3-f275ad450fe5", "create_time": "00c57a11-37ca-4259-9c38-0ac5e522e9e8", "update_time": "a415c2a6-8289-4a4d-8d49-3685712d7622", }, "zones": { "zone_type": "c217db55-b5d6-4430-ad80-8534e691e54a" } }	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from the Google Dataplex. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the Google Dataplex. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Field	Description	Required
Capability	This section contains general information about the capability.
Name	The name of the Edge capability.	Yes
Description	The description of the Edge capability.	No
GCP service account	This section contains information on how to connect to Google Cloud Storage.
GCP Connection	The GCP connection to be used.	Yes
Configuration	This section contains information on the configuration of the crawlers.
Project IDs (Deprecated)	Add a comma-separated list of the Project IDs where Dataplex is enabled. This field is deprecated in the latest user interface and replaced by the Project IDs field on the Synchronize Metadata page. You can add the Project IDs when you synchronize Google Dataplex Catalog. The following rules apply when you add Project IDs: If you enter a value in this field and do not add Project IDs on the Synchronize Metadata page, the capability will search in these projects in this field when you synchronize the capability. If you leave this field empty and do not add project IDs on the Synchronize Metadata page, the capability will search in the projects that you entered in the GCP Service Account field in the GCP connection. Do not enter a value in this field and also add Project IDs on the Synchronize Metadata page; otherwise, the synchronization will end with an error when you synchronize the capability.	No
Save input metadata	Select the checkbox if you want to save the input metadata extracted from the data source in ZIP files. The files can be useful for troubleshooting. Select this option only on request of Collibra Support. The Collibra Support team can provide the location of the saved ZIP files after the synchronization. This checkbox is not selected by default.	No
(Deprecated) Filters and Domain Mapping	Important This field is deprecated. Define any mappings in the integration configuration.	No
Extensible Properties Mapping	Via the Extensible Properties Mapping field, you can integrate additional properties from Dataplex: Table creation date, Table modified date, System (showing where the table comes from), and type (the Zone type). Important If you use this feature, make sure to set up all required characteristic assignments for the asset types. You do this by adding the mapping between the fields for the objects in Dataplex and the Collibra attribute IDs to ingest the data in, using a JSON string. The text must be in JSON format and can contain a Zones and Tables block. In each block, you specify the property name and the attribute ID to which you want to map the value in the property. The format is: `"[property name]": "[attribute resource ID]"`. For example, `"system": "19a27fda-8c50-48a8-87b3-f275ad450fe5"`. Example { "tables": { "system": "19a27fda-8c50-48a8-87b3-f275ad450fe5", "create_time": "00c57a11-37ca-4259-9c38-0ac5e522e9e8", "update_time": "a415c2a6-8289-4a4d-8d49-3685712d7622", }, "zones": { "zone_type": "c217db55-b5d6-4430-ad80-8534e691e54a" } }	No
Advanced Configuration	These configuration options help when investigating issues with the capability. Important Only complete the fields Save Input Metadata, Logging configuration, Memory (MiB), and JVM arguments on request of or together with Collibra Support.	No
Debug	This field is ignored when you integrate metadata from the Google Dataplex Catalog. An option to automatically send Edge infrastructure log files to Collibra Data Intelligence Platform. By default, this option is set to false. Note We highly recommend to only send Edge infrastructure log files to Collibra Data Intelligence Platform when you have issues with Edge. If you set it to true, it will automatically revert to false after 24h.	No
Log level	This field is ignored when you integrate metadata from the Google Dataplex Catalog. An option to determine the verbosity level of Catalog connector log files. By default, this option is set to No logging.	No

Click Create.
The capability is added to the Edge site.
The fields become read-only.

More information

ADLS integration

Catalog Data Classification

Catalog JDBC ingestion

JDBC Profiling

Catalog JDBC Sampling

S3 synchronization

GCS synchronization

Databricks Unity Catalog integration

DQ Connector

Technical lineage via Edge

Protect for AWS Lake Formation

Protect for BigQuery

Protect for Databricks

Protect for Snowflake

Azure ML for Azure ML

AWS SageMaker AI for AWS SageMaker AI

AWS Bedrock AI for AWS Bedrock AI

SAP AI Core for SAP AI Core

SAP Datasphere integration

Google Dataplex integration

Google Dataplex Catalog integration

Add an Edge capability to an Edge site

Prerequisites

Steps

How to find the folder path

More information