Synchronized Snowflake data and characteristics mapping

This page gives an overview of the integrated assets from Snowflake via the Catalog JDBC ingestion synchronization, including, the naming conventions, ingested metadata per asset, and the code to create a hierarchy in Collibra.

Naming conventions for created assets

The assets receive a unique full name (fully qualifying name) based on the following naming convention: [asset parent full name]>[asset name]

You can view the full name of an asset by editing the asset.

Warning Avoid editing the full name of an asset, because the full name is used to synchronize and refresh data sources. Changing the full name may cause unexpected results and break the synchronization or refresh process.

Snowflake source of synchronized metadata

For Table and Columns assets, the Snowflake source depends on the value defined for the engine-strategy property.

• Tables:
  • If the engine-strategy value is SNOWFLAKE_INFORMATION_SCHEMA, the source is INFORMATION_SCHEMA.TABLES.
  • If the engine-strategy value is SNOWFLAKE_ACCOUNT_USAGE, the source is SNOWFLAKE.ACCOUNT_USAGE.TABLES.
  • If the engine-strategy value is JAVA_API, the tables are retrieved via command SHOW TABLES IN SCHEMA.
• Columns:
  • If the engine-strategy value is SNOWFLAKE_INFORMATION_SCHEMA, the source is INFORMATION_SCHEMA.COLUMNS.
  • If the engine-strategy value is SNOWFLAKE_ACCOUNT_USAGE, the source is SNOWFLAKE.ACCOUNT_USAGE.COLUMNS.
  • If the engine-strategy value is JAVA_API, the columns are retrieved via command SHOW COLUMNS.

For Source tags, the Snowflake source depends on the value defined for tags-strategy property.

• If the tags-strategy value is SINGLE_CALL, the source is ACCOUNT_USAGE.tag_references table.
• If the tags-strategy value is CALL_PER_TABLE, the sources are INFORMATION_SCHEMA.tag_references_all_columns and INFORMATION_SCHEMA.tag_references tables.
  
• If the tags-strategy value is SKIP, no values are retrieved.

Also other requirements need to be in place to retrieve source tags. For more information, go to Preflight checks for Snowflake metadata integration (Edge).

Characteristics mapping

For generic information, go to Synchronized metadata.

Hierarchy and diagram view in Collibra

By default, the resulting assets are shown in a plain list, but you can enable a multi-path hierarchy to show it in a tree structure. Use the following relations to create the hierarchy.

• Technology asset has Schema
• Schema contains Table
  Because Database View and Semantic View are both child asset types of Table, they are included automatically in this hierarchy.
• Tables contains Column

You can also create a diagram view that shows the hierarchy.

{
  "nodes": [
    {
      "id": "Database",
      "type": {
        "id": "00000000-0000-0000-0000-000000031006"
      }
    },
    {
      "id": "Schema",
      "type": {
        "id": "00000000-0000-0000-0001-000400000002"
      }
    },
    {
      "id": "Table",
      "type": {
        "id": "00000000-0000-0000-0000-000000031007"
      }
    },
    {
      "id": "Column",
      "type": {
        "id": "00000000-0000-0000-0000-000000031008"
      }
    }
  ],
  "edges": [
    {
      "from": "Database",
      "to": "Schema",
      "label": "",
      "style": "arrow",
      "type": {
        "id": "00000000-0000-0000-0000-000000007024"
      },
      "roleDirection": true
    },
    {
      "from": "Schema",
      "to": "Table",
      "label": "",
      "style": "arrow",
      "type": {
        "id": "00000000-0000-0000-0000-000000007043"
      },
      "roleDirection": true
    },
    {
      "from": "Table",
      "to": "Column",
      "label": "",
      "style": "arrow",
      "type": {
        "id": "00000000-0000-0000-0000-000000007042"
      },
      "roleDirection": false
    }
  ],
  "showOverview": false,
  "enableFilters": true,
  "showLabels": false,
  "showFields": true,
  "showLegend": true,
  "showPreview": true,
  "visitStrategy": "completeGraph",
  "layout": "HierarchyTopBottom",
  "maxNodeLabelLength": 50,
  "maxEdgeLabelLength": 30,
  "layoutOptions": {
    "compactGroups": false,
    "componentArrangementPolicy": "topmost",
    "edgeBends": true,
    "edgeBundling": true,
    "edgeToEdgeDistance": 5,
    "minimumLayerDistance": "auto",
    "nodeToEdgeDistance": 5,
    "orthogonalRouting": true,
    "preciseNodeHeightCalculation": true,
    "recursiveGroupLayering": true,
    "separateLayers": true,
    "webWorkers": true,
    "nodePlacer": {
      "barycenterMode": true,
      "breakLongSegments": true,
      "groupCompactionStrategy": "none",
      "nodeCompaction": false,
      "straightenEdges": true
    }
  }
}

About Semantic View assets

The Semantic View asset type is a child of the Table asset type that represents a Snowflake Semantic View. The fields in the Snowflake Semantic View are ingested as Column assets linked to the Semantic View asset.

A Semantic View asset page includes a Definition tab, which shows the full YAML representation of the Semantic View as ingested from Snowflake. For information, go to the Snowflake Semantic View YAML documentation.

Related topics

About the Snowflake metadata integration via Edge