Integrations best practices

Applies to: integrating with Collibra.

Recommendation

Collibra offers a wide range of ways to connect to external data sources. Follow the recommendations and key component overview in this article to optimize your Collibra Data Intelligence Cloud metadata connector or integration approach.

Impact

  • Reduce time and resource costs for integration delivery, installation and support.
  • Leverage Collibra out-of-the-box features.

Background - integration types overview

  • Native/Direct integrations are available in Collibra as out-of-the-box features, such as S3 and Tableau.
  • Metadata connectors are mostly available in (Undefined variable: CollibraProducts.CollibraMarketplace) or on the technology vendor website:
    • JDBC connectors certified by Collibra.
    • Vendor supported JDBC connectors non-certified by Collibra.
    • Spring Boot applications installed as a standalone integration app from (Undefined variable: CollibraProducts.CollibraMarketplace).
  • Data Lineage is available as a standalone application:
    • Technical Lineage Harvester as a standalone JAVA application. We support three types:
      • Data Source as JDBC.
      • Data Source as ETL.
      • BI Harvesters such as PowerBI, Tableau and MicroStrategy.
    • Custom integration usually developed on any ESB (Enterprise Service Bus) or using object-oriented language for example Java Spring Boot service or Python service:
      • Custom Integrations using Java or Python can fully utilize REST IMPORT APIs to implement non-supported integrations.
      • We can use any ESB tools and Collibra provided Connectors called “ Collibra Connect” specially built for Mule for integrations.
      • We can extend use cases and use Collibra workflows in these integrations which can be called by custom integrations using REST API for bidirectional integrations and workflow task assignments to specific roles inside Collibra.
  • Insights Data Access which provide APIs can be used to extract data from Collibra. This helps in giving an overview of Collibra Maturity Assessments.
  • Manual data import as Excel files and .csv files from Collibra UI.
  • Collibra Data Quality & Observability Connectors used to extract the DQ Rules from Collibra DQ into (Undefined variable: migration-to-cloud.dgc-name-acronym). This is currently only supported in Edge.

Best practice recommendations

Integration type selection and key decision factors

This section aims to help you chose the best integration method to minimize your resource time and cost, and leverage the out-of-the-box feature set for your unique use case.

We recommend that you evaluate the different integration options in the following order of priority. Stop at the first one that provides the functionality you require:

Priority Integration approach
1 Native/Direct integration.
2 JDBC connectors certified by Collibra.
3 Vendor supported JDBC connectors non-certified by Collibra.
4 Spring Boot applications installed as a standalone integration app from the (Undefined variable: CollibraProducts.CollibraMarketplace).
5 Custom integration developed by any ESB or custom integration service.

Integration selection key factors

Below is a table of integration type and decision-making factors you should consider when choosing a new integration method:

  Collibra Supported Time to install and configure Jobserver or Edge required Upgrade process License vendor Collibra features enabled
Collibra native integration Yes Low S3 integration: Edge or Jobserver required. Automatic, during DGC releases timeline. Collibra  
Jobserver metadata connectors Supporting party has to be checked on (Undefined variable: CollibraProducts.CollibraMarketplace). Low Jobserver Manual update once new version released. Metadata connector provider (might not be supported when Free of charge) Profiling, Data Sampling, Classification
Edge capability templates Yes Low Edge Automatic, during Collibra releases timeline. Collibra Profiling, Data Sampling , Classification
Collibra Data Quality & Observability connector Yes Low Edge Automatic, during Edge release. Collibra  
Spring Boot applications Supporting party has to be checked on(Undefined variable: CollibraProducts.CollibraMarketplace). Medium Not needed Manual Metadata connector provider  
Data Lineage Yes Medium Not needed Manual Collibra Technical Lineage, Business Lineage
Reporting Insights Yes Medium Not needed Manual Collibra Reporting insights
Custom integration as any EBS or custom integration service No High Not needed Depending on ESB Platform or Custom Integration Technology ESB Platform Vendor or Technology Vendor Custom integration as any EBS or custom integration service

Validation Criteria

One of the important validation criteria is “Time to Install and Configure”, where values are:

  • Low is < 1 day
  • Medium is 1 - 5 days (usually 1-2 days)
  • High is 5 - 20 or more days (usually 10+ days).

Please refer to the above section for details about time needed.

Additional Information

For more information see the following resources: