System requirements of an Edge site

To use Edge, you must ensure that the following system requirements are met.

  • K3S
  • EKS cluster

Software requirements

  • You must be able to install the Edge software on the latest version of RedHat Enterprise Linux 8.
  • The sudo package is installed on the Linux host.
  • The user who installs Edge has full sudo access (ALL=(ALL) ALL)
  • Optionally, if you want SE Linux enabled, you will need to install the following policy packages before installing Edge:
    Packages

    These packages are not hosted by Collibra. If you have any questions, please reach out to your internal teams.


Hardware requirements

You need the following minimum hardware requirements:

  • 64 GB memory
  • 16 core CPU with x86_64 architecture
  • You have at least 50 GB of free storage on the partition that must contain /var/lib/rancher/k3s.
    mkdir -p /var/lib/rancher/k3s
mkfs.xfs /dev/<block-device-name>
mount /dev/<block-device-name> /var/lib/rancher/k3s
echo '/dev/<block-device-name> /var/lib/rancher/k3s xfs defaults 0 0' >> /etc/fstab

    Note This is the default install path. If it is not created as a separate mount point following the steps above, the install will use 50 GB of disk space from either /var, or if not present, the root level of the drive.

    Warning Any data in this location is fully managed by the Edge site, do not save any other data in here as it can be removed by Edge without notification.

  • You have mounted at least 500 GB of dedicated storage for the Edge site data on a freely chosen mountpoint, for example /var/edge/storage.

    mkdir -p /var/edge/storage
mkfs.xfs /dev/<block-device-name>
mount /dev/<block-device-name> /var/edge/storage
echo '/dev/<block-device-name> /var/edge/storage xfs defaults 0 0' >> /etc/fstab

    Note  Change <block-device-name> to the name of the device that contains the storage.

    Warning This dedicated storage should not be shared with other services because Edge can delete and overwrite files on this location without notice, so don't use /home/<username> or /var.

  • If you run the Linux server on AWS or GCP, disable the services nm-cloud-setup.service and nm-cloud-setup.timer. 

    systemctl disable nm-cloud-setup.service nm-cloud-setup.timer 
reboot

Warning When new capabilities are added in the future, the hardware requirements may change.

Network requirements

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Cloud environment
    • https://*.datadoghq.com
      Note If wildcard whitelisting is not accepted:
      • https://http-intake.logs.datadoghq.com
      • https://7-36-1-app.agent.datadoghq.com
      • https://agent-http-intake.logs.datadoghq.com
      • https://api.datadoghq.com
    • https://*.repository.collibra.io
      Note If wildcard whitelisting is not accepted:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/
  • Your Edge site has to be able to connect to port 443.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

Note If a firewall is enabled, run the following commands to add the cni0 and loopback interfaces to a trusted zone, so that Kubernetes can use it between its services: 

firewall-cmd --zone=trusted --change-interface=cni0 --permanent
firewall-cmd --zone=trusted --change-interface=lo --permanent
firewall-cmd --reload

EKS requirements

You can install the Edge software on managed Kubernetes clusters.

Important A managed Kubernetes cluster must be fully dedicated for Edge, do not use the cluster for other purposes.

  • AWS EKS 1.21 or 1.22 (both only with --container-runtime containerd)
  • AWS EKS worker nodes use the EKS optimized Amazon Linux 2 AMI
  • EKS cluster has IRSA enabled
  • AWS EKS worker nodes need to be in the same (one) Availability Zone!
    This can be implemented by creating just one node group for the EKS cluster, which limits the subnets to just one subnet, that is one of the subnets of the EKS cluster.
    module "eks" {
	source            = "terraform-aws-modules/eks/aws"
	version           = "17.24.0"
	cluster_name      = "${var.vpc_name}-${var.cluster_name}-eks"
    	cluster_version   = "1.21"
	vpc_id            = var.vpc_id
	subnets           = data.aws_subnet_ids.public_subnet_ids.ids  # Subnets specified must be in at least two different AZs
	worker_additional_security_group_ids = [aws_security_group.worker_sg.id]
	enable_irsa       = true  # enable iam role for service account, for later use
	worker_groups = [
		{
    			name                = "${var.vpc_name}-${var.cluster_name}-eks-workers"
			instance_type        = var.worker_type
			asg_desired_capacity = var.instance_count_workers
			key_name             = aws_key_pair.cluster-ssh-keypair.key_name
			bootstrap_extra_args = "--container-runtime containerd"  # mandatory to run with containerd if on 1.21
			subnets              = [subnet1]                             # restriction for now to use only 1 subnet due to EBS tied to AZ
    		},
		]
	map_accounts = [
		data.aws_caller_identity.current.account_id
		]

	tags = {
		Name            = "${var.vpc_name}-${var.cluster_name}-eks"
    		}
}

Software requirements

  • A Linux server with bash available. This is the server from which you install the Edge software on EKS.

    Tip This server will also contain the Edge tools.

  • Plain cluster_admin kubectl access to the EKS cluster using its kubeconfig. With this kubeconfig, you must be able to use the kubectl command to communicate with the Kubernetes API server with full cluster access.
  • Kubectl client version 1.22.9, supports EKS 1.21 and 1.22.

Hardware requirements

You need an operational EKS cluster with at least 1 worker node. The cluster must meet the following requirements:

  • The total cluster capacity has at least 16 core CPU and 64 GB memory, for example 4 worker nodes with each 4 core CPU and 16 GB.
  • Each worker node needs at least 100 GB free disk space to store Docker images.
  • Ability to create EBS-based persistent volumes as a default storage class, at least 500 GB in total.

Network requirements

  • An Edge site needs outbound connections to all of the following:
    • The URL of your Collibra Data Intelligence Cloud environment
    • https://*.datadoghq.com
      Note If wildcard whitelisting is not accepted:
      • https://http-intake.logs.datadoghq.com
      • https://7-36-1-app.agent.datadoghq.com
      • https://agent-http-intake.logs.datadoghq.com
      • https://api.datadoghq.com
    • https://*.repository.collibra.io
      Note If wildcard whitelisting is not accepted:
      • https://repository.collibra.io
      • https://edge-docker-delivery.repository.collibra.io
      • https://mirror-docker.repository.collibra.io
    • https://otlp-http.observability.collibra.dev/
  • Your Edge site has to be able to connect to port 443.
  • The resolve configuration file of your Linux host has maximum three search domains and two name servers.

Note If a firewall is enabled, run the following commands to add the cni0 and loopback interfaces to a trusted zone, so that Kubernetes can use it between its services: 

firewall-cmd --zone=trusted --change-interface=cni0 --permanent
firewall-cmd --zone=trusted --change-interface=lo --permanent
firewall-cmd --reload