Integrated Azure Data Lake Storage data

Important 

In Collibra 2024.02, we've launched a new user interface (UI) in beta for Collibra Data Intelligence Platform! You can learn more about this latest UI in the UI overview.

Use the following options to see the documentation in the latest UI or in the previous, classic UI:

After you have synchronized the data, the integration of the Azure Data Lake Storage file system is completed, and the resulting assets are available in the domain that was specified in the crawler.

Warning Do not move the assets to another domain. Doing so may lead to errors during future synchronizations.

Tip ADLS synchronization relies on UUIDs.

Note In case of a partial synchronization caused by a temporary communication issue, the status of the assets that cannot be synchronized is set to Missing from source. Their previous status is restored, if they are found in the source system during the next fully successful synchronization.

By default, the assets are shown in a plain list, but you can enable a multi-path hierarchy to show it in a tree structure. The resulting assets depend on whether you use Microsoft Purview.

Synchronization results without Microsoft Purview

For the best result, use the following relations when you define a multi-path hierarchy:

  • File Storage contains File Container
  • File Container contains File Container
  • Directory contains Directory
  • File Container contains File

Synchronization results with Microsoft Purview

For the best result, use the following relations when you define a multi-path hierarchy:

  • File Storage contains File Container
  • File Container contains File Container
  • Directory contains Directory
  • File Container contains File
  • File contains Table
  • Table contains Column

Synchronized metadata per asset type

This table shows the metadata for each ADLS asset type.

Asset type

Synchronized metadata

Resource ID
ADLS Storage Account File Storage contains / is part of File Container 00000000-0000-0000-0001-002600000000
ADLS Container

Location

00000000-0000-0000-0000-000000000203
File Container contains / is part of File Container 00000000-0000-0000-0001-002600000001
Directory File Container contains / is part of File Container 00000000-0000-0000-0001-002600000001
Directory contains / is part of directory 00000000-0000-0000-0001-002600000003
File File Type 00000000-0000-0000-0001-002500000012
Size 00000000-0000-0000-0001-000500000009
File Container contains / contained in 00000000-0000-0000-0000-000000007060
Table

Description

00000000-0000-0000-0000-000000003114
File contains / is part of Table 00000000-0000-0000-0001-002600000002
Column

Description

00000000-0000-0000-0000-000000003114
Column Position 00000000-0000-0000-0001-000500000020

Technical Data Type

Tip 

For columns that have a structured technical data type, Array or Struct, you can click the button in the Column asset to see the structure of the data in a dialog box. You see the technical data type in the Technical Data Type field in the At a Glance sidebar of the Column asset. For columns that have a structured technical data type, Array or Struct, click the data type name to see the structure of the data in a dialog box.
This is supported for AVRO, CSV, JSON, ORC, PARQUET, PSV, SSV, TSV, TXT, and XML.
In the capability settings, you can define the maximum level you want to see in the structure.

00000000-0000-0000-0000-000000000219
Column is part of / contains Table 00000000-0000-0000-0000-000000007042