System requirements of an Edge site

To use Edge, you must ensure that the following system requirements are met.

Software requirements

Tip If you are an early adopter or you use Edge for beta testing purposes, we highly recommend that you disable SELinux.

Hardware requirements

Note When installing on k3s, the Virtual Machine (VM) must be dedicated to a single Edge site installer.

You need the following minimum hardware requirements:

  • 64 GB memory.
  • 16-core CPU with x86_64 architecture.
  • At least 60 GB of free storage for Edge application storage requirements:
    • You have at least 50 GB of free storage on the partition that contains /var/lib/rancher/k3s. The partition mountpoint should not have the noexec option.

      Warning Any data in this location is fully managed by the Edge site. Do not save any other data in this location as the data can be removed by Edge without notification.

    • You have at least an additional 5 GB of space in /var/log for Edge components. Edge uses hardcoded /var/log to write logs:
      • Up to 1.1 GB of space for writing K3S audit logs.
      • Maximum of 60 MB per container for pod logs. The number of containers depends on the workload.
    • You have at least an additional 5 GB of space on the partition that holds /var/lib/kubelet. Edge uses hardcoded /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/* to write ephemeral data related to kubernetes.
  •  At least 500 GB of dedicated storage for Edge data storage requirements:
    • You have mounted at least 500 GB of dedicated storage for the Edge site data on a freely chosen mountpoint, for example, /var/edge/storage.

      Warning This dedicated storage must not be shared with other services because Edge can delete and overwrite files on this location without notice. Therefore, do not use /home/<username> or /var.

  • If you run the Linux server on AWS, Azure, or GCP, disable the services nm-cloud-setup.service and nm-cloud-setup.timer.

Warning When new capabilities are added in the future, the hardware requirements may change.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.
Note 
  • Ensure that the network connectivity between the internal cluster and the service CIDRs use by k3s (which are by default 10.42.0.0/16 and 10.43.0.0/16) is not blocked.
  • In case firewalld is enabled, run the following commands to add the cni0 and loopback interfaces to a trusted zone, so that Kubernetes can use it between its services:
    firewall-cmd --zone=trusted --change-interface=cni0 --permanent
    firewall-cmd --zone=trusted --change-interface=lo --permanent
    firewall-cmd --reload

FedRAMP

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.ddog-gov.com
    • https://*.artifactory-gov2prod.collibra.com/
      Note If the allowlist does not accept wildcards:
      • https://artifactory-gov2prod.collibra.com
      • https://edge-docker-delivery.artifactory-gov2prod.collibra.com
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.
Note 
  • Ensure that the network connectivity between the internal cluster and the service CIDRs use by k3s (which are by default 10.42.0.0/16 and 10.43.0.0/16) is not blocked.
  • In case firewalld is enabled, run the following commands to add the cni0 and loopback interfaces to a trusted zone, so that Kubernetes can use it between its services:
    firewall-cmd --zone=trusted --change-interface=cni0 --permanent
    firewall-cmd --zone=trusted --change-interface=lo --permanent
    firewall-cmd --reload

Select the managed, dedicated Kubernetes cluster you want to install your Edge site on:

EKS requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated to a single Edge site installer, do not use the cluster for other purposes.

  • AWS EKS 1.27 is supported for new and existing Edge sites.
    Note  
  • AWS EKS worker nodes use the EKS optimized Amazon Linux 2 AMI
  • EKS cluster has IRSA enabled
  • Set up security groups to ensure that worker nodes can communicate with each other on non-privileged ports.

Software requirements

  • A Linux server with bash available. This is the server from which you install the Edge software on EKS.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to the EKS cluster using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Ensure your Kubectl client is compatible with the relevant EKS version.

Hardware requirements

You need an operational EKS cluster with at least 1 worker node. The cluster must meet the following requirements:

  • The total cluster capacity has at least 16 core CPU and 64 GB memory, for example, 4 worker nodes each with 4 core CPU and 16 GB.
  • Each worker node needs at least 100 GB free disk space to store Docker images.
  • We recommend you have at least 2 worker nodes in the EKS cluster.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

FedRAMP

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.ddog-gov.com
    • https://*.artifactory-gov2prod.collibra.com/
      Note If the allowlist does not accept wildcards:
      • https://artifactory-gov2prod.collibra.com
      • https://edge-docker-delivery.artifactory-gov2prod.collibra.com
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

GKE requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated to a single Edge site installer, do not use the cluster for other purposes.

  • GKE 1.27 is supported for new Edge sites.
    Note Only new Edge sites can be installed on a GKE cluster. Existing Edge sites installed on EKS or k3s cannot be migrated to a GKE cluster.

Software requirements

  • A Linux server with bash available. This is the server from which you install the Edge software on GKE.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to the GKE cluster using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Ensure your Kubectl client is compatible with the relevant GKE version.

Hardware requirements

You need an operational GKE cluster with at least 1 worker node. The cluster must meet the following requirements:

  • The total cluster capacity has at least 16 core CPU and 64 GB memory, for example, 2 worker nodes each with 8 core CPU and 32 GB or 4 work nodes each with 4 core CPU and 16 GB.
  • Each worker node needs at least 100 GB free disk space to store Docker images.
  • We recommend you have at least 2 worker nodes in the GKE cluster.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

FedRAMP

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.ddog-gov.com
    • https://*.artifactory-gov2prod.collibra.com/
      Note If the allowlist does not accept wildcards:
      • https://artifactory-gov2prod.collibra.com
      • https://edge-docker-delivery.artifactory-gov2prod.collibra.com
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

AWS Fargate using EKS requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated to a single Edge site installer, do not use the cluster for other purposes.

  • AWS Fargate using EKS on Kubernetes 1.27 is supported for new Edge sites.
    Note Only new Edge sites can be installed on an AWS Fargate using EKS cluster. Existing Edge sites installed on EKS or k3s cannot be migrated to an AWS Fargate using EKS cluster.
  • EKS cluster has IRSA enabled
  • You must create an AWS Fargate profile for your cluster with the following namespace selectors:
    • kube-system
    • default
    • collibra-*
    • edge-kube-installer
  • EKS cluster has CoreDNS enabled and running on a Fargate Node(s).

Software requirements

  • A Linux server with bash available. This is the server from which you install the Edge software on AWS Fargate using EKS.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to the cluster using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Ensure your Kubectl client is compatible with the relevant EKS version.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

OpenShift requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated to a single Edge site installer, do not use the cluster for other purposes.

  • OpenShift 4.14.7 is supported for new Edge sites.
    Note Only new Edge sites can be installed on a OpenShift cluster. Existing Edge sites installed on EKS or k3s cannot be migrated to an OpenShift cluster.

Software requirements

Note Currently, Edge is unable to generate logs for OpenShift clusters. If you need to generate logs for your Edge sites installed on an OpenShift cluster, you need to use Kubernetes logging. With the 2024.05 release, Edge will be able to generate logs for OpenShift clusters.
  • A Linux server with bash available. This is the server from which you install the Edge software on OpenShift.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to OpenShift using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Ensure your Kubectl client is compatible with the relevant OpenShift version.
  • Create Security Context Constraints (SCC) which provide Edge service accounts with the required permissions.
    Note Currently, Edge provides 2 SCC files associated with the Edge namespaces. These files are based on the permissions required by the service accounts in these namespaces.
    • A non-privileged SCC is associated with both collibra-fast and collibra-edge.
    • A privileged SCC is associated with the edge-kube-installer and collibra-telemetry namespaces.

Hardware requirements

You need an operational OpenShift cluster with at least 1 worker node. The cluster must meet the following requirements:

  • The total cluster capacity has at least 16 core CPU and 64 GB memory, for example, 4 worker nodes each with 4 core CPU and 16 GB.
  • Each worker node needs at least 100 GB free disk space to store Docker images.
  • We recommend you have at least 2 worker nodes in the OpenShift cluster.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

FedRAMP

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.ddog-gov.com
    • https://*.artifactory-gov2prod.collibra.com/
      Note If the allowlist does not accept wildcards:
      • https://artifactory-gov2prod.collibra.com
      • https://edge-docker-delivery.artifactory-gov2prod.collibra.com
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

AKS requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated to a single Edge site installer, do not use the cluster for other purposes.

  • AKS 1.27 is supported for new Edge sites.
    Note Only new Edge sites can be installed on a AKS cluster. Existing Edge sites installed on EKS or k3s cannot be migrated to a AKS cluster.

Software requirements

  • A Linux server with bash available. This is the server from which you install the Edge software on AKS.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to the AKS cluster using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Ensure your Kubectl client is compatible with the relevant AKS version.

Hardware requirements

You need an operational AKS cluster with at least 1 worker node. The cluster must meet the following requirements:

  • The total cluster capacity has at least 16 core CPU and 64 GB memory, for example, 2 worker nodes each with 8 core CPU and 32 GB or 4 work nodes each with 4 core CPU and 16 GB.
  • Each worker node needs at least 100 GB free disk space to store Docker images.
  • We recommend you have at least 2 worker nodes in the AKS cluster.

Network requirements

Commercial

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.datadoghq.com: This URL is used to collect some of the logs from Edge for issue diagnosis. We do not send JDBC driver logs from Edge to Datadog.
    • https://*.repository.collibra.io: This URL serves as the primary source for downloading the latest Edge docker images from Collibra's docker registry and helm-chart repository.
      Note If the allowlist does not accept wildcards:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/: This URL is used to ingest metrics and traces for monitoring the health and usage of Edge sites.
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

FedRAMP

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Platform environment.
    • https://http-intake.logs.ddog-gov.com
    • https://*.artifactory-gov2prod.collibra.com/
      Note If the allowlist does not accept wildcards:
      • https://artifactory-gov2prod.collibra.com
      • https://edge-docker-delivery.artifactory-gov2prod.collibra.com
  • Access to all data sources you need to connect to your Edge sites.
  • Your Edge site has to be able to connect to port 443.
  • If you intend to use a man-in-the-middle (MITM) proxy, you need to add the specific truststores customization to the ca.pem, because Edge does not use the host TLS trustsore. For more information, go to Configure a forward proxy.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.