Missing stitching and possible solutions

When preparing your lineage harvester configuration file, you can set the useCollibraSystemName property to true, to indicate that you want to use the system or server name of a data source to match to the System asset you created when you prepared the physical data layer. This is useful when you have multiple databases with the same name, in different systems or servers.

By default, the useCollibraSystemName property is set to false. If you want to use it, set it to true.

If you set the useCollibraSystemName property to false, the lineage harvester ignores the collibraSystemName property.

Collibra Data Lineage does not support DAX. Therefore, calculated columns result in missing lineage, as do unsupported Power Query M function.

In this example, notice that the database node, which should be situated to the left of the BI data model node, is missing from the technical lineage graph:

We can identify that these nodes represent, respectively:

• A Power BI data set table named al_test NewTable_jdbc
  
• A Power BI report named al_jdbc_mysql_powerbi_27Mar

Note The same scenario can surface for Tableau if, for example, you do not have sufficient permission, or if you have stored procedures or custom SQL that is not supported by Tableau Catalog.

Examine the BI data set table, to see if you can identify a problem

1. Click the Settings tab.
2. Click Show status.
   The Sources tab shows a list of all data sources on the Collibra Data Lineage service instance.
3. Click one or more checkboxes, to show the transformations and source code fragments for specific data sources, or clear all checkboxes, to show for all data sources.
   

Transformations and source code fragments are shown in the transformations table below. We can quickly identify that there was an analyze error, because the MySQL.Database function is not supported.

To achieve stitching, ensure that your queries only include supported Power Query M functions. We also encourage you to create an Ideation ticket via the Collibra Ideation Portal, if you'd like to request support for a particular function.

First, let's look at the name of an unstitched database table in the technical lineage graph:

MODEL.PRODUCT CATEGORY [ADVENTUREWORKS::database]

We can identify the following:

• The database name: ADVENTUREWORKS
• The schema name: MODEL
• The table name: PRODUCT CATEGORY

Now let's find the table in the Stitching tab:

1. Click the Settings tab.
2. Click Show status.
3. Click the Stitching tab.
   The Stitching tab shows a list of all tables that exist in Data Catalog and on the Collibra Data Lineage service instance.
   

Use the Search field to find the unstitched database table PRODUCT CATEGORY.

In the Found in column, the value "Technical Lineage" confirms what we already know: the table was found only in the technical lineage. An exactly matching asset was not identified in Collibra.

Now try to find a likely match. Look for a table that has the same name and the value "Catalog" in the Found in column.

The table shown in the following image looks like a match. The schema and table names match exactly; only the database names differ.

To achieve stitching:

1. Prepare the database mapping section of your Power BI <source ID> configuration file as follows:
   

   {
   	"found_dbname=adventrueworks;found_hostname=*": {
   		"dbname": "aas-model",
   	}
   }
   

2. Run the lineage harvester again.
   Stitching is achieved
   

In this scenario, lineage is not shown between the BI data model – in other words, the data set that was used to build your report – and your BI report. The most likely explanation for this is that the Microsoft APIs could not send the Power BI dataset metadata to the Collibra Data Lineage service instance because the dataset type is not supported by the Microsoft API.

A common issue is if you have Object Level Security (OLS) applied to one or more columns in the dataset. The Microsoft APIs do not return metadata for any data objects with OLS applied.

As shown in the following example image, if the dataset metadata is not provided by the Microsoft APIs, an analysis error is produced and shown in the Technical lineage Sources tab page, for the unsupported dataset.

First, let's look at the name of name of the unstitched database table in the technical lineage graph:

DBADMIN.TESTING1 [111.93.0.181::database]

We can identify the following:

• The database name: 111.93.0.181
• The schema name: DBADMIN
• The table name: TESTING1

Now let's find the table in the Stitching tab:

1. Click the Settings tab.
2. Click Show status.
3. Click the Stitching tab.
   The Stitching tab shows a list of all tables that exist in Data Catalog and on the Collibra Data Lineage service instance.
   

Use the Search field to find the unstitched table TESTING1.

In the Found in column, the value "Technical Lineage" confirms what we already know: the table was found only in the technical lineage. An exactly matching asset was not identified in Collibra.

Now try to find a likely match. Because DBADMIN.TESTING1 [111.93.0.181::database] was found only in the technical lineage, we know the match we're looking for must have the value "Catalog" in the Found in column.

The table shown in the following image looks like a match. The schema and table names match exactly; only the database names differ.

To achieve stitching:

1. Configure the hostnameMapping section in your Tableau <source ID> configuration file as follows:
   

   "hostnameMapping": {
   	"found_dbname=111.93.0.181;found_hostname=*: {
   		"dbname": "oracle-db",
   		"dialect": "oracle"
   		}
   }
   

2. Run the lineage harvester again.
   Stitching is achieved
   

If you use a Viewer role or an Explorer role in Tableau, without access to external assets, stitching is always missing. To achieve stitching, ensure that your Tableau user has access to the external assets for which you want lineage.

For complete details on the effect of roles and permissions on technical lineage and stitching, including how to provide necessary permissions to get lineage and automatic stitching with an Explorer role,go to Tableau roles and permissions.

Tableau Catalog doesn't support all custom SQL and stored procedures. For complete information, see the Tableau documentation Tableau Catalog support for custom SQL.

If a Tableau data object uses an unsupported custom SQL or stored procedure, the external database node, which should be situated to the left of the BI data model node, is missing in the technical lineage graph, as shown in the following example image.

If all of your assets were successfully ingested, but relations are missing between MicroStrategy Data Entity assets (meaning MicroStrategy attributes and facts) and MicroStrategy Data Attributes (meaning MicroStrategy columns) and there is no lineage – only orphaned boxes and no database nodes – it means the API call to the tables endpoint api/model/tables was unsuccessful.

To resolve this issue:

• Ensure that your MicroStrategy permissions are correctly configured.
• Ensure that your proxy (if applicable) can connect to this endpoint.

For complete information on MicroStrategy REST, see the MicroStrategy documentation.