Release 2024.11

Lineage Harvester (CLI and Edge)

• Collibra Data Lineage now harvests lineage from all projects specified by the "Project ID" field on the synchronization page.
• You can now use a custom Catalog to ingest metadata from Databricks Unity Catalog and create technical lineage.
• When integrating Power BI, you can now benefit from new filtering options, which we refer to as “filtering v2”. In addition to filtering on capacities and workspaces, v2 allows you to also filter on dashboards and reports, including reports that are published in Power BI apps. For complete information, go to Prepare a <source ID> configuration file and Power BI workspaces.
• When integrating Looker, Collibra Data Lineage can now access different API endpoints, depending on your Looker user permissions. Therefore, your Looker user no longer requires an Admin role. You can now configure a custom role with a specific set of permissions. For complete details, go to Set up Looker.
• We updated the Snowflake JDBC driver that is bundled with the CLI lineage harvester to version 3.23.1.

Security

• We improved the security of Edge and Protect capabilities via Edge.

Metadata integrations

• The Azure AI Foundry integration via Edge is now available in public beta. When you integrate Azure AI Foundry, you integrate the metadata of AI models and AI agents from Azure AI Foundry to the Collibra Platform. The resulting assets represent the Azure AI Foundry model and agent.
  For information, go to About the Azure AI Foundry integration via Edge.
• With the release of SAP version 2025.5, when integrating SAP Datasphere Catalog, the user who owns the system in SAP Datasphere Catalog must have access to all of the spaces that they want to synchronize in SAP Datasphere Catalog. To own a system:
  • You need the “Catalog Admin” permission in SAP Datasphere Catalog.
  • You can’t be the System Owner.

Lineage Harvester (CLI and Edge)

• Collibra Data Lineage now ingests lineage from file assets for Google Dataplex.
• Collibra Data Lineage now supports OpenLineage, AWS Glue (via OpenLineage), and Apache Airflow (via OpenLineage) in public beta. This functionality is available on Edge.
  For information, go to Create a technical lineage via Edge.
• When creating technical lineage for Google Dataplex, column-level lineage support is generally available.
• When integrating MicroStrategy, Collibra Data Lineage now correctly processes data source mapping configurations. You no longer encounter "java.lang.CharSequence.length()" errors.

Metadata integrations

• When ingesting SAP Datasphere metadata as part of the SAP Datasphere Catalog integration, we now ingest the URLs for the following asset types. The URLs point to the instances of the assets the data builder.
  • SAP Datasphere Analytic Model
  • SAP Datasphere Intelligent Lookup
  • Table
  • Database View
• We improved the clarity of Azure ML capability logs.

Lineage Harvester (CLI and Edge)

• Collibra Data Lineage now includes transformation source code in the technical lineage viewer for Databricks Unity Catalog.
• Collibra Data Lineage now ingests table-level lineage for Databricks Unity Catalog.

Lineage Harvester (CLI and Edge)

• When ingesting Azure Synapse Analytics data sources, you can now choose between two Edge capabilities, depending on whether you want to ingest metadata from one database or multiple databases.
  • Use the existing “Technical Lineage for Azure” capability to harvest metadata from a single database (dedicated SQL pool).
  • Use the new “Technical Lineage for Azure (multi-DB)” capability to harvest metadata from multiple databases (serverless SQL pool).
• When harvesting Azure synapse data sources, Collibra Data Lineage now supports multiple databases in one capability.
• When ingesting Azure Data Factory data sources, Collibra Data Lineage now supports pipeline run data.

Security

• We've improved the security of capabilities via Edge.

Metadata integrations

• For schemas integrated using an earlier version of the Databricks Unity Catalog integration, domains with spaces in their names will remain unchanged. Duplicate empty domains without spaces will no longer be created in future integrations.
• When you synchronize with Azure ML, access tokens are now refreshed to reduce failures if the synchronization takes longer than an hour to complete.

Lineage Harvester (CLI and Edge)

• Collibra Data Lineage now supports the OAuth authentication type, via the Technical Lineage Edge capabilities, to authenticate connections to the Collibra Data Lineage service instances.

Lineage Harvester (CLI and Edge)

• When ingesting dbt Cloud data sources, Collibra Data Lineage now supports aliases.
• When ingesting data source types that allow for multiple databases (meaning HiveQL, Microsoft SQL Server, MySQL, Netezza, Spark SQL, Sybase and Teradata), you can now use the optional “Database name JSON” field in your technical lineage Edge capabilities, to provide multiple database names in a list format.
• Collibra Data Lineage now supports the OAuth authentication type, via the Technical Lineage Edge capabilities, to authenticate connections to the Collibra Data Lineage service instances.

Metadata integrations

• When integrating SAP Datasphere, Collibra now creates relations between assets, even if the corresponding assets in SAP Datasphere exist in different Datasphere Spaces. If you configure a filter, the relevant Datasphere Spaces must pass the filter.
• Collibra Data Lineage now includes transformation source code in the technical lineage viewer for Databricks.
• You can now select "us" and "eu" from the "Location" field when you configure the synchronization of the Dataplex Catalog ingestion.

Protect

• Protect for BigQuery now sets retry parameters for all Google APIs. This ensures that operations such as setting or clearing IAM members on policy tags are retried if a service is temporarily unavailable.

Security

• We have improved the security of capabilities via Edge.

Metadata integrations

• Google Dataplex Catalog integration now ingests columns for views.
• The ADLS with Purview integration now synchronizes correctly when the "Max Schema Level" field in the capability is set to a value other than 1.
• Collibra Data Lineage now fetches information from the Dataplex API differently to prevent failures.

Collibra Data Lineage Service

• We fixed an issue that caused delays in generating technical lineage.
• We optimized memory usage to prevent technical lineage synchronization failures when computing and generating large lineage graphs.
• When ingesting Teradata data sources:
  • Collibra Data Lineage can now parse long strings of queries more efficiently.
  • Collibra Data Lineage now raises an analyze error when there is ambiguity about a column in auto-create mode.
• When integrating Power BI:
  • If a harvested dataflow is missing the content property, Collibra Data Lineage now produces an analyze error. Previously, the integration failed.
  • Collibra Data Lineage no longer raises duplicate analyze errors.
• When integrating Tableau, Collibra Data Lineage now correctly processes multiple sites.
• When ingesting SAP HANA data sources with calculation views, Collibra Data Lineage now creates lineage between a CUBE in one schema as its synonym in another schema.
• When integrating DataStage, Collibra Data Lineage now parses SQL SELECT statements to get the list of columns when Runtime Column Propagation (RCP) is enabled.
• When integrating PowerCenter, Collibra Data Lineage now extracts expressions for the transformations of Aggregator type.
• When integrating Informatica Intelligent Cloud Services (IICS), Collibra Data Lineage now has improved SQL statement analysis.

Protect

• Protect capabilities are now more resilient to network instabilities when communicating with the Collibra Platform.

Lineage Harvester (CLI and Edge)

• When ingesting JDBC data sources, you can now use database-system mapping to:
  • Map databases to their correct systems, to obtain stitching.
  • Resolve missing stitching, which occurs if Collibra Data Lineage associates multiple databases with the default system name that you provide in the Collibra System Name field.
• When ingesting Oracle data sources via shared storage connection (if via Edge) or folder connection (if via CLI harvester), you can now use database link mapping to configure, per data source, the database and schema to which a DBLink points.

Collibra Data Lineage Service

• You can now use the new Technical Lineage Admin Edge connection and capability to:
  • List all of the data sources that will be used to create a technical lineage.
  • Ignore, or exclude, data sources from those that will be used to create a technical lineage.
  • Analyze specified batches (ZIP files) of metadata on the Collibra Data Lineage service instance.
  • Trigger the synchronization of analyzed metadata on the Collibra Data Lineage service instance, for all data sources, in a single go.

  Previously, these options were available only via the CLI harvester. For complete information, go to Technical lineage admin options.

• When sharing database models across data sources, Collibra Data Lineage now supports configurations wherein multiple Oracle databases are specified as dependent sources.
• When integrating Power BI with DAX analysis enabled, Collibra Data Lineage now raises an analyze error in the case of failed DAX analysis.
• When integrating Power BI, Collibra Data Lineage now correctly applies capacity filtering.
• When integrating Informatica Intelligent Cloud Services (IICS), Collibra Data Lineage now also analyzes Pre SQL and Post SQL attributes if they are present under the advancedProperties key of the transformation definition.

Infrastructure

• Edge sites installed on bundled k3s from the Edge 2024.10.112 version or newer are installed with k8s 1.31. No action is required.

Security

• We improved the security of lineage capabilities via Edge.

Metadata integrations

• The technical lineage capability synchronization no longer fails due to large metadata files sent to DIP.

Protect

• Protect for BigQuery now properly handles project IDs and dataset names that include dashes.

Lineage Harvester (CLI and Edge)

• When integrating SSRS, Collibra Data Lineage now skips reports for which the configured user does not have sufficient access rights and creates an analyse error.
• When harvesting Oracle data sources, the memory footprint is now reduced, allowing the lineage harvester to successfully harvest metadata from much larger data sources.

Collibra Data Lineage Service

• When integrating MicroStrategy:
• Collibra Data Lineage now creates relations of the type “Data Asset is source for / source BI Report” between MicroStrategy Dossier assets and MicroStrategy Data Model assets.
• If the MicroStrategy API response doesn’t include the required “lookupTable” parameter, Collibra Data Lineage now raises an analyse error and looks to the “attributeLookupTable” parameter as an alternative to improve the lineage.
• When integrating Power BI, Collibra Data Lineage no longer creates duplicate tables with different system names when “useCollibraSystemName” is set to “true”.
• When integrating Informatica PowerCenter, Collibra Data Lineage now uses the configured dialect for the SQL analysis.
• When ingesting Oracle data sources, Collibra Data Lineage now supports limited lineage for arrays in PL/SQL.
• When ingesting Spark data sources, Collibra Data Lineage now supports microseconds as an interval unit of time.

Protect

• Protect for BigQuery now properly handles unprotected tables that are renamed during synchronization. "Table not found" errors are no longer reported, ensuring synchronization continues without interruption.
• Protect capabilities are now more resilient to network instabilities when communicating with the Collibra Platform.

Collibra Data Lineage Service

• When integrating Power BI, Collibra Data Lineage now handles unnamed dataflows.

Collibra Data Lineage Service

• When ingesting Snowflake data sources, Collibra Data Lineage now analyses view definitions first so that the correct DDL is available for analysing DML statements.
• We fixed an issue with relations for data objects in technical lineage. Collibra Data Lineage now filters relations found in Collibra using the resource IDs of asset types instead of asset type names.
• When ingesting Informatica Intelligent Cloud Services (IICS) sources, the following enhancements are made:
  • Collibra Data Lineage now attempts a case-insensitive match between adapter info and transformations when there is no case-sensitive match.
  • Collibra Data Lineage now raises an analyze error for taskflows without any mapping tasks and for unsupported task types.
• When ingesting Big Query data sources, Collibra Data Lineage now recognizes implicit OFFSET columns from UNNEST WITH OFFSET.
• When integrating Power BI with DAX analysis enabled:
  • Collibra Data Lineage now creates correct table names when single quotes are used in DAX queries.
  • An improvement to the caching mechanism allows for faster processing.
• When integrating SSRS, Collibra Data Lineage now skips reports for which the configured user does not have sufficient access rights and creates an analyse error.
• When ingesting DataStage data sources, the following enhancements are made:
  • Collibra Data Lineage now detects more complex DataStage expressions.
  • Collibra Data Lineage now parses subrecords descriptions and correctly shows values.
• When ingesting Oracle data sources, the Collibra Data Lineage service instances now extract lineage from anonymous blocks with DECLARE statements.
• Collibra Data Lineage now supports column-level lineage for Google Dataplex.
• When creating technical lineage for Dataplex, Collibra Data Lineage now ingests interactive queries by default. You can disable the ingestion of interactive queries when synchronizing the capability.
• We optimized technical lineage graph rendering to prevent failures in certain use cases.

Infrastructure

• The Created Date for Edge images now shows the actual date the image was created.
• We have improved the Edge CLI to handle special characters, such as ---. Previously, if --- was used, the Edge site would fail to install. You must download the latest Edge CLI version to receive this update.

Security

• We improved the security of capabilities via Edge.

Data Catalog

• We fixed an issue that sometimes caused the registration of a JDBC data source to fail.

Metadata integrations

• When you run the Dataplex integration for a data source, the asset name is now taken from the "data_path" field in the Dataplex API. This ensures the name represents the original asset name in GCP.
• OAuth autentication for Databricks Unity Catalog now works as expected with Edge sites installed with a forward proxy.
• You can now integrate AWS Bedrock AI model metrics into Collibra. To do this, add the mapping between the custom metric and the Collibra attribute in the "Custom AI Metrics Mappings" section when you configure the Amazon Bedrock AI (beta) synchronization.

Lineage Harvester (CLI and Edge)

• When integrating PBRS, Collibra Data Lineage now honors the "no_proxy" configuration.

Collibra Data Lineage Service

• When integrating MicroStrategy, Collibra Data Lineage: 
  • Again uses the system name "DEFAULT" for databases when "useCollibraSystemName" is set to "True", but no name is provided for the "collibraSystemName" property in the <source ID> configuration file.
  • Now creates relations of the type “Report groups / is grouped into Report” between MicroStrategy Dossier assets and MicroStrategy Report assets.
• When integrating Power BI, Collibra Data Lineage now raises an analyze error if data is missing in the API response because the semantic model needs to be refreshed in Power BI.

Collibra Data Lineage Service

• When ingesting Sybase data sources, Collibra Data Lineage now:
  • Supports the LIST function.
  • Supports "rule" as a column name.
  • Supports "today(*)" syntax.
• When ingesting Snowflake data sources, Collibra Data Lineage now:
  • Correctly handles CTE name shadowing.
  • Supports INSERT INTO IDENTIFIER() statements.
  • Supports EXECUTE IMMEDIATE statements assigned to a variable.
• When ingesting Oracle data sources, Collibra Data Lineage now resolves the source database link name for a synonym using the information provided in the Database Link Mapping property.
• The analysis of SQL files with many volatile tables no longer results in out-of-memory errors.
• When integrating MicroStrategy, Collibra Data Lineage now uses the system name "DEFAULT" when "useCollibraSystemName" is set to "True" but no name is provided for the "collibraSystemName" property in the <source ID> configuration file.
• When ingesting DataStage data sources, Collibra Data Lineage now supports DSX files using Japanese locale.
• We fixed an issue on the Source tab page where clicking “Export All Transformations” with no sources checked would result in an empty export CSV file.

Collibra Data Lineage Service

• When ingesting Oracle data sources, Collibra Data Lineage now
  • Extracts lineage from anonymous blocks.
  • Fully supports stored procedures, including variable tracking across statements.
• When ingesting Snowflake data sources using SQL-API mode, Collibra Data Lineage now creates lineage between base object and direct object.
• When ingesting DataStage data sources, Collibra Data Lineage now correctly handles the cache during processing, avoiding failures.
• On the technical lineage Source tab page, you can now filter the source codes by ERROR, PARSING_ERROR, and ANALYZE_ERROR.
• When integrating Power BI with filtering, Collibra Data Lineage no longer creates empty capacities for capacities that have been filtered out.
• When ingesting IBM Db2 data sources, Collibra Data Lineage now supports NEXTVAL used as a keyword.

Metadata integrations

• Aspects that refer to columns are already integrated as Column assets in Collibra during a Dataplex Catalog ingestion. Now, you can map other aspects in Google Dataplex with attributes in Collibra. This allows you to show these aspects on asset pages.

Protect

Lineage Harvester (CLI and Edge)

• When harvesting Oracle data sources, Collibra Data Lineage now supports TRIGGERS.
• When creating a custom technical lineage via the CLI lineage harvester or Edge, using the batch definition method, the lineage harvester now validates the syntax in your JSON files.
• The lineage harvester (Edge and CLI) and the technical lineageEdge capabilities are using the DNS names documented in Collibra Data Lineage service instances.
• When harvesting Oracle technical lineage sources, logging can now be enabled in the Edge harvester, to aide troubleshooting. Data object harvesting details are captured in a log file in the metadata batch.
• The Dataplex lineage can now handle GCS file names which contain wildcards.

Collibra Data Lineage Service

• When integrating Power BI with DAX analysis enabled, if the machine learning platform returns “None” as a table name, processing of the table is skipped and an analyze error is produced.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support "identifier()" in UPDATE statements.
• When ingesting Sybase data sources, the Collibra Data Lineage service instances now support getdate(*) syntax.
• When ingesting SAP Analytics Cloud data sources, the Collibra Data Lineage service instances now correctly handle assets that don't have a published parent asset.
• When ingesting SAP Hana Cloud data sources with calculation views, the Collibra Data Lineage service instances now correctly parse the resource URI.
• When ingesting PowerCenter data sources, the Collibra Data Lineage service instances now raise an analyze error when the "dbname" is "FlatFile" but the "DATABASETYPE" is not "FLAT_FILE_JOINED".
• When ingesting Datastage data sources, the Collibra Data Lineage service instances now support parameter resolution of UserVariables that are Datastage expressions.
• Collibra Data Lineage now continues processing the collected mapping resource for BI tools even when errors occur during an import job. Previously, processing would end as soon as an error occurred. Now, processing ends only when the number of failed commands before stopping import job is reached.
• When ingesting dbt data sources, the Collibra Data Lineage service instances now use both "compiled_code" and "compiled_path".

Security

• We improved the security of integrations via Edge.

Metadata integrations

• We improved the scalable ingestion flow to ensure assets receive the correct status when resynchronizing a data source.
• You can now use sampling and profiling for JDBC Databricks data sources that have databases with numeric names.

Collibra Data Lineage Service

• When ingesting PowerCenter data sources, the Collibra Data Lineage service instances now:
  • Correctly handle mapplet connectors.
  • Correctly process service headers.
  • Provide the correct configuration file example when there are UNDEFINED values present.
  • Report on missing definitions of sessions and mappings.
• When ingesting JDBC data sources, the Collibra Data Lineage service instances now handle complex nested queries more efficiently.
• When ingesting Tableau data sources, the Collibra Data Lineage service instances now produce an analyze error if database information is missing in the API response.
• When ingesting Spark data sources, the Collibra Data Lineage service instances now support identifiers that start with a digit in the table definition.
• When ingesting Teradata data sources, the Collibra Data Lineage service instances now correctly handle volatile tables.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support:
  • DELETE statements.
  • PIVOT statements with numbers as pivot values.
• When integrating Power BI, the Collibra Data Lineage service instances now create technical lineage relations between data flows and data models.

Collibra Data Lineage Service

• When ingesting Informatica PowerCenter data sources, Collibra Data Lineage now supports lineage between two tables that are linked by a file.
• When ingesting Teradata data sources, the Collibra Data Lineage service instances now support the ERRLIMIT command.
• When ingesting Oracle data sources, the Collibra Data Lineage service instances now correctly create lineage for CREATE TABLE AS statements in combination with INSERT INTO (the created table) statements.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support:
  • The PIVOT function with “table.column” as the value column in the pivot.
  • "identifier()" in update statements.
• When ingesting Tableau data sources, if a worksheet or dashboard in Tableau has been deleted or can't be found on the Tableau server, the value of the “Visible on server” attribute on the respective asset pages in Collibra is “False”. The URLs to these worksheets and dashboards are no longer harvested and shown on the respective asset pages.
• When creating a custom technical lineage via the CLI lineage harvester or Edge, using the batch definition method, the lineage harvester now validates the syntax in your JSON files.
• When ingesting SAP Hana Cloud data sources with calculation views, the Collibra Data Lineage service instances now correctly parse the resource URI.
• When ingesting Spark data sources, the Collibra Data Lineage service instances now correctly handle the PARTITION condition, regardless of the position in the SQL Statement.
• When ingesting Informatica Information Cloud Services data sources, the Collibra Data Lineage service instances now correctly handle the class rule TmplDatatypeRule.
• When ingesting Azure Data Factory data sources, the Collibra Data Lineage service instances now support the "Copy" activity returning the "additionalColumns" key API response as a dictionary. Previously, the "additionalColumns" key API response could only be accepted as an array.
• When ingesting DataStage data sources, any unresolved parameters (which result in analyze errors) are now identified in the error details on the technical lineage Sources tab page.

Collibra Data Lineage Service

• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support:
  • Indexed values in parameter expressions.
  • EXECUTE statements assigned to variables.
• When ingesting DataStage data sources, the Collibra Data Lineage service instances can now successfully process queries that are defined in the QUERY stage.
• When ingesting dbt data sources, the Collibra Data Lineage service instances now correctly create SQL statements for analysis.
• When ingesting DataStage date sources, Collibra Data Lineage now supports the implicit parameter "DSJobInvocationId" when the property "Allow Multiple Instance" is enabled.
• When ingesting dbt data sources, if "dbt run" or "dbt compile" was not run, an analyze error message and resolution is now shown in the technical lineage Sources tab page.

Infrastructure

• When you install your Edge site via the Helm chart method, your Linux machine must have yq version 4.18.1 or later installed.

Data classification

• Pushdown sampling (source-driven sampling) is now correctly used in the sampling and classification processes for Oracle data sources.

Collibra Protect

• To prevent excessively log entries, Protect for BigQuery no longer logs debug messages.
• BadSqlGrammarException errors reported in Protect for Databricks and Snowflake now include the underlying root cause instead of vague descriptions.

Collibra Data Lineage Service

• When ingesting any of the supported JDBC data sources, the Collibra Data Lineage service instances now skip any SQL statements that contain unresolved parameters.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support BEGIN TRANSACTION statements.
• When integrating Power BI with DAX analysis enabled, Collibra Data Lineage is now more resilient when receiving API errors from the Vertex AI API.
• When ingesting DataStage data sources, any unresolved parameters (which result in analyze errors) are now identified in the error details on the technical lineage Sources tab page.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support the CONNECT BY clause without the START WITH clause.
• When analyzing Snowflake SQL statements as part of a supported BI or ETL tool integration, the Collibra Data Lineage service instances now correctly process stages.
• When ingesting Teradata data sources, the Collibra Data Lineage service instances now correctly handle tables and views that contain constants.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now support SELECT INTO statements into variables without a preceding colon.
• When creating a custom technical lineage, the Collibra Data Lineage service instances now now strictly check the JSON schema, allowing for better detection of errors.

Security

• We improved the security of integrations via Edge.

Data Catalog

• Dataplex technical lineage now also supports Google Cloud Storage (GCS) assets that have a full name which starts with gcs: in addition to gs:.
• The AI model integration for Databricks Unity Catalog now correctly handles cases where a model has an associated run that no longer exists.
• The Databricks Unity Catalog integration now handles errors during synchronization better, ensuring that the synchronization job does not get stuck.
• The Amazon S3 integration now successfully completes when encountering an AWS Glue table with duplicated column names. The synchronization detects duplicated columns and logs a warning about the duplication, advising you to fix the issue on the data source side. The column position will not be included for these columns.
• We updated the error message for AWS SageMaker AI integrations in unsupported regions.
• The Azure ML integration now processes attribute names regardless of their casing. All attribute names are converted to lowercase to ensure they are ingested correctly.
• You can now indicate that you don't want asset statuses to change during a Databricks resynchronization. The Databricks synchronization configuration includes a "Default Asset Status" field. The default value is "Implemented" and is available for all Databricks ingestion types.
  • If "Implemented" is selected, all assets are ingested with "Implemented" status.
  • If "No Status" is selected, all assets are ingested with the first status listed in the Operating Model statuses, and changes will not be overwritten. For example, if you change an asset status from "Candidate" to "Implemented" before the resynchronization, the status remains "Implemented."

Collibra Data Lineage Service

• When ingesting DataStage data sources, the Collibra Data Lineage service instances now also extract SQL statements from the input links.
• dbt Cloud lineage capabilities are now supported on Edge sites with a proxy and certificates.
• When ingesting Snowflake data sources (SQL-API ingestion mode), the Collibra Data Lineage service instances now use improved hashing internally.

Infrastructure

• We improved our support for custom certificates on Edge sites. You can now:
  • Add a custom certificate to an Edge site, in addition to the existing system certificates.
  • Add, merge, or replace custom certificates after an Edge site has been installed.
  • Add custom certificates for Data Sources, in addition to proxy setup.
• When you run the Edge CLI update command, Edge CLI checks whether it is on the latest version. If it isn’t, it updates itself to the newest version available.
• You can now download a new version of the Edge CLI outside the legacy Edge tool. For more information, go to the Edge CLI documentation.
• Edge sites can now be installed via the Helm chart method successfully, regardless of the yq version.
• When you upload a Shared Storage connection folder or file via the Edge CLI, the key length can have up to 300 characters.
• Shared Folder Storage now can use up to 6 GB. The maximum file size has been reduced to 1 GB.

Security

• We improved the security of Edge and SAP Datasphere synchronization.

Data classification

• The Classification and Sample Data capabilities no longer use the same Edge cache. This fixes an issue where the sample data for an asset used for the classification was immediately visible to users with the required permission.

Data Catalog

• The Google Dataplex integration now supports the ingestion of BigQuery Entries and Aspects from the new Dataplex Catalog.
• We improved the scalable ingestion flow to better handle deleted assets that are still present at the source.

Lineage Harvester (CLI and Edge)

• If Tableau integration fails because of timeout errors due to page sizing limitations, Collibra Data Lineage now automatically adjusts the settings and retries.
• The capability for Technical Lineage for OpenLineage is now visible in the “Capability template” drop-down list, preparing for public availability. Note that only participants in the private beta should use this feature.
• When ingesting Oracle data sources, the Collibra Data Lineage service instances now support scoped database link names.

Collibra Data Lineage Service

• When ingesting Spark SQL data sources, the Collibra Data Lineage service instances now support the TBLPROPERTIES function even if the key is not between double quotations.
• When integrating Looker, Collibra Data Lineage now uses the value of the “label_short” attribute of the column in Looker, for the display name of ingested Looker Data Set Column assets in Collibra.
• When ingesting DataStage data sources, the Collibra Data Lineage service instances now correctly handle:
  • Sequence in SQL statements.
  • Uppercase and lowercase letters in SQL statements.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances now correctly handle SQL statements that include a lot of indirect lineage.
• When ingesting Snowflake data sources via the SQL mode, the Collibra Data Lineage service instances now support dynamic tables.
• When ingesting Azure Data Factory data sources, SQL statements of the type “expression” no longer result in analyze errors.
• When ingesting Netezza data sources, the Collibra Data Lineage service instances now support the WITH FINAL clause used with a table function.
• You can again export a technical lineage as a JSON file.
• When analyzing SQL statements from one of the supported BI or ETL tools, the Collibra Data Lineage service instances now support SQL statements for which multiple sources are used to create a column.
• When ingesting custom technical lineage via the batch definition option, Collibra Data Lineage now provides improved error messaging when duplicates are detected.
• When integrating Power BI with DAX analysis enabled, previously unrecoverable HTTP 400 errors that occurred due to size limitations are now skipped and logged as analyse errors.

Collibra Data Lineage Service

• When ingesting Oracle data sources, the Collibra Data Lineage service instances now convert to uppercase any lowercase database names that are provided via the "databaseLinkMapping" option.
• The Collibra Data Lineage service instances now correctly process SQL Common Table Expressions with three or more queries in a set.
• When ingesting DataStage data sources, the Collibra Data Lineage service instances now correctly identify CONTEXT as an SQL clause, instead of as a DataStage expression.
• When ingesting Snowflake data sources via the SQL-API mode, the Collibra Data Lineage service instances now create lineage even if a referenced database has not been harvested.
• When ingesting Snowflake data sources, the Collibra Data Lineage service instances use full names instead of native IDs to parse column lineage. This update allows for faster parsing, smaller batches, and less duplicate data.
• When ingesting Azure Data Factory data sources, the Collibra Data Lineage service instances:
  • Can now process single-space escape characters.
  • Now support the UTCNOW and FormatDateTime functions.
• When ingesting DataStage data sources, variables are now applied at job level for nested jobs. Previously, they were applied at job level only for directly executed jobs.

Infrastructure

• We have improved our support for custom certificates on Edge sites. You can now:
  • Add a custom certificate to an Edge site, in addition to the existing system certificates.
  • Add, merge, or replace custom certificates after an Edge site has been installed.
  • Add custom certificates for Data Sources, in addition to proxy setup.

Security

• We have improved the security of Data profiling.

Collibra Protect

• Protect for Snowflake supports column names containing spaces and special characters.
• You can now choose how Snowflake checks roles (that is, Protect groups) for applying standards and rules, accommodating Snowflake users who have multiple roles. When adding a capability for Protect for Snowflake, the "Edit Capability" dialog box contains a new "Snowflake role testing" field with the following options:
  • "CURRENT_ROLE" (default): Checks only the primary role assigned to the Snowflake user.
  • "IS_ROLE_IN_SESSION": Checks all the roles assigned to the Snowflake user, including secondary roles, within the active session.

Data Catalog

• The Collibra classification and sampling features now support Edge connection names that include the '>' character. However, we still advise against using '>' as it serves as a separator in ingested Catalog asset full names.
• If you request sample data via Edge, we now report sooner that the 'Request sample data' job is completed.
• We no longer mark valid dates and numbers in tables as "Invalid value" when collecting samples or classifying data.
• The attribute "URL" for asset types Table and Database view is automatically synchronized via Edge for AWS Glue data sources only.
• The AWS Bedrock integration now supports the integration of imported AI models in AWS Bedrock. Note that "imported models" is a preview feature in Amazon Bedrock.
• The Azure ML integration now supports the integration of custom AI models in Azure ML.
• You can now download the input metadata from the AWS SageMaker "Synchronization Results" dialog box.

Lineage Harvester (CLI and Edge)

• When synchronizing a technical lineage via Edge, Collibra Data Lineage now returns the correct result message.
• When ingesting Matillion data sources via Edge, the Collibra Data Lineage service instances can now process the "startTimestamp" parameter.

Collibra Data Lineage Service

• When ingesting DataStage date sources, Collibra Data Lineage now retains the system names in the file paths. Previously, system names were removed during ingestion, which meant end-to-end lineage could not be attained.
• When ingesting BigQuery data sources, the Collibra Data Lineage service instances now support STRING FORMAT when the expression is enclosed between parenthesis.
• When ingesting ADF data sources, the Collibra Data Lineage service can now get objects from an array.

Infrastructure

• We have removed ArgoCD from all Edge components. This change improves the security of Edge by reducing future security vulnerabilities associated with ArgoCD.
• You can now create a backup of your Edge site installed via the Helm chart method, and use it to reinstall that Edge site via the Helm chart method.
• We have simplified and improved the Helm chart installation and uninstallation methods. These improved methods reduce steps and simplify the prerequisite resources required for Edge sites installed via the Helm chart method.

Lineage Harvester (CLI and Edge)

• When ingesting Netezza data sources via Edge, Collibra Data Lineage now correctly harvests views when multiple databases are specified in the capability.
• When integrating Tableau Server data sources via Edge, Collibra Data Lineage now successfully harvests the default site, even if it was renamed in Tableau.
• We fixed an issue with the CLI Harvester URI syntax on Windows.

Collibra Data Lineage Service

• When ingesting PostgreSQL data sources, the Collibra Data Lineage service instances now support SQL statements that contain ON CONFLICT clauses.
• When ingesting Oracle data sources, the Collibra Data Lineage service instances now:
  • Correctly handle casing for asset names.
  • Support all keywords in CONNECT BY clauses.
• When ingesting Netezza data sources, the Collibra Data Lineage service instances now support the /=/ operator.
• When ingesting SAP Hana data sources, the Collibra Data Lineage service instances now support the @prompt command.
• When integrating Power BI, the Power Query M function “Cube.AddAndExpandDimensionColumn” now supports custom table names.