Zarf installation and upgrade for CPSH extended capabilities

This following instructions detail the process of installing the Zarf command-line interface (CLI) and initializing the target Kubernetes cluster for the CPSH extended capabilities deployment.

To begin:

  1. Download and install Zarf if you haven't done so already.
  2. (Optional) Run the Zarf initialization.

Initialize the cluster

This step bootstraps the cluster by installing the infrastructure necessary for Zarf (internal registry, agent, etc.).

  1. Download the standard Zarf init package to your local working directory.

    2. Copy 
    zarf tools download-init -a amd64

    Note The -a flag specifies the architecture for the package. By default, zarf tools download-init downloads the package for the architecture of your local machine. For example, if you are using a Mac with arm64, it downloads the arm64 version. However, if your local machine uses a different architecture than your cluster (such as EKS, which is amd64), you must include the -a flag for both the download and initialization steps to avoid an architecture mismatch.

  2. Optional, if you are installing on an OpenShift cluster, you must perform the following steps before running zarf init .These steps create and scope the OpenShift security constraints to the zarf namespace.
    1. Create file called zarf-init-prereqs.yaml.
    2. Copy and paste the following into the new file:
      Copy
      ---
      # OpenShift pre-requisite for CPSH deployment — zarf namespace only.
      #
      # Apply this manifest ONCE before running `zarf init` on an OpenShift cluster:
      #
      #   kubectl apply -f cicd-pipeline/resources/openshift/zarf-init-prereqs.yaml
      #
      # Then run zarf init with the proxy host-network flag (required on OpenShift
      # because OVN-Kubernetes does not support hostPort with hostIP: 127.0.0.1):
      #
      #   zarf init --registry-mode=proxy -a amd64 --set-variables HOST_NETWORK_PROXY=true
      #
      # ─────────────────────────────────────────────────────────────────────────────
      # Why this must be a pre-step (not bundled in the Zarf package)
      # ─────────────────────────────────────────────────────────────────────────────
      # The Zarf registry-proxy DaemonSet is created by `zarf init` — before the
      # platform package is ever deployed. On OpenShift, OVN-Kubernetes ignores
      # hostPort with hostIP:127.0.0.1, so the proxy DaemonSet must use
      # hostNetwork:true instead. OpenShift's SCC admission blocks hostNetwork:true
      # unless the pod's service account is bound to an SCC that explicitly allows it.
      # cpsh-zarf-scc must therefore exist before `zarf init` runs.
      #
      # ─────────────────────────────────────────────────────────────────────────────
      # cpsh-zarf-scc field rationale
      # ─────────────────────────────────────────────────────────────────────────────
      # allowHostNetwork / allowHostPorts: true
      #   Required for the Zarf registry-proxy DaemonSet. Scoped exclusively to the
      #   zarf namespace so no application pod in any collibra-* namespace can request
      #   host networking.
      #
      # runAsUser: RunAsAny
      #   Zarf agent runs as UID 65532, registry as UID 1000 — both outside
      #   OpenShift's per-namespace UID range.
      #
      # seLinuxContext: MustRunAs
      #   OpenShift allocates an MCS label per pod and relabels PVCs to match —
      #   correct behavior if the Zarf registry uses persistent storage.
      #
      # No capabilities, no unsafe sysctls, no hostPath volumes.

      # ─────────────────────────────────────────────────────────────────────────────
      # cpsh-zarf-scc — Zarf proxy components (zarf namespace only)
      # ─────────────────────────────────────────────────────────────────────────────
      apiVersion: security.openshift.io/v1
      kind: SecurityContextConstraints
      metadata:
        name: cpsh-zarf-scc
        annotations:
          kubernetes.io/description: >
            SCC for Zarf proxy-mode components in the zarf namespace. Grants
            allowHostNetwork and allowHostPorts so the registry-proxy DaemonSet can
            use hostNetwork:true on RHCOS (OVN-Kubernetes ignores hostPort with
            hostIP:127.0.0.1). Allows arbitrary UIDs for the Zarf agent (65532) and
            registry (1000). No Linux capabilities, no unsafe sysctls, no hostPath
            volumes. Scoped to the zarf namespace only so application pods in
            collibra-* namespaces cannot use host networking.
      allowPrivilegedContainer: false
      allowHostDirVolumePlugin: false
      allowHostNetwork: true
      allowHostPorts: true
      allowHostIPC: false
      allowHostPID: false
      readOnlyRootFilesystem: false
      runAsUser:
        type: RunAsAny
      fsGroup:
        type: RunAsAny
      supplementalGroups:
        type: RunAsAny
      seLinuxContext:
        type: MustRunAs
      seccompProfiles:
        - '*'
      volumes:
        - configMap
        - downwardAPI
        - emptyDir
        - persistentVolumeClaim
        - projected
        - secret
      allowPrivilegeEscalation: false
      allowedCapabilities: []
      defaultAddCapabilities: []
      requiredDropCapabilities:
        - ALL
      users: []
      groups: []

      ---
      # ─────────────────────────────────────────────────────────────────────────────
      # RBAC — grants 'use' on cpsh-zarf-scc (zarf namespace only)
      # ─────────────────────────────────────────────────────────────────────────────
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: cpsh-zarf-scc-role
      rules:
        - apiGroups: ["security.openshift.io"]
          resources: ["securitycontextconstraints"]
          resourceNames: ["cpsh-zarf-scc"]
          verbs: ["use"]

      ---
      # Binds cpsh-zarf-scc exclusively to the zarf namespace.
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: cpsh-zarf-scc-binding
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: cpsh-zarf-scc-role
      subjects:
        - kind: Group
          apiGroup: rbac.authorization.k8s.io
          name: "system:serviceaccounts:zarf"
    3. Run the following command:
      Copy
      kubectl apply -f zarf-init-prereqs.yaml
       
  3. Execute the following initialization command to initialize the cluster, enable the registry proxy, and apply specific Kubernetes taints and tolerations required for CPSH node groups:
    Note If you are installing on an OpenShift cluster, you must add the following to the following command:

    --set HOST_NETWORK_PROXY=true \

    Copy
    zarf --log-level=debug init \
      --features=registry-proxy=true \
      --registry-mode=proxy \
      -a amd64 \
      --set REGISTRY_PVC_SIZE=100Gi \
      --set REGISTRY_HPA_MAX=1 \
      --set PROXY_TOLERATIONS='[{"effect":"NoSchedule","key":"collibra.com/node-storage","operator":"Equal","value":"local"},{"effect":"NoSchedule","key":"collibra.com/team","operator":"Equal","value":"techlin"}]' \
      --confirm

Initialization Command Details

Parameter Value/Setting Purpose
--features=registry-proxy=true true Enables the Zarf registry proxy feature.
--set HOST_NETWORK_PROXY=true \ true

If you use an OpenShift cluster, you must set this to true.As OVN-Kubernetes ignores hostPort with hostIP: 127.0.0.1 on OpenShift, setting this parameter to true switches the registry-proxy DaemonSet to hostNetwork: true.
--registry-mode=proxy proxy Sets the internal Zarf registry to run in proxy mode.
--set REGISTRY_HPA_MAX=1 1 Sets the maximum Horizontal Pod Autoscaler replicas for the Zarf registry to 1.
--set PROXY_TOLERATIONS='[...]' Specific JSON array Applies Kubernetes tolerations to the Zarf proxy components, ensuring they run on the dedicated techlin node group with local storage.
--set REGISTRY_PVC_SIZE=100Gi 100Gi If you are going to use all Zarf images, such as Collibra Platform, Edge, and Technical lineage, set the Persistent Volume Claim (PVC) to 100 GB. If not, the Zarf default of 20 GB is used and you may run out of storage space.
--confirm   Skips interactive prompts and automatically confirms the installation.

Update

Standard method

To update Zarf using the standard method, reinstall the new Zarf version and re-run the initialization steps using the same parameters:

Note This is applicable if the newer Zarf version is backward compatible.

Copy 
zarf init [...]

Hard method

If Zarf needs to be updated and the new version is incompatible with the previous, or init parameters needs to be changed, you must first remove and then reinstall.

Note This method forces Zarf to forget about previously installed packages, potentially leading to issues.

Run the following if current installed packages can be removed:

Copy 
zarf destroy --remove-components --confirm
zarf init [...]

If current installed packages can not be removed:

  1. Backup the current Zarf secrets matching previously installed packages:

    2. Copy 
    kubectl get secrets -n zarf -o name |grep 'zarf-package-cpsh'
    kubectl get secrets -n zarf zarf-package-cpsh-platform -o yaml >zarf-package-cpsh-platform-secret.yaml
    kubectl get secrets -n zarf zarf-package-cpsh-lineage -o yaml >zarf-package-cpsh-lineage-secret.yaml
    kubectl get secrets -n zarf zarf-package-cpsh-edge-fips -o yaml >zarf-package-cpsh-edge-fips-secret.yaml
    kubectl get secrets -n zarf zarf-package-cpsh-observability -o yaml >zarf-package-cpsh-observability-secret.yaml

  2. Run Zarf destroy:

    3. Copy 
    zarf destroy --confirm

  3. Re-run Zarf init:

    4. Copy 
    zarf init [...]

  4. Restore the Zarf secrets for previously installed packages:

    5. Copy 
    kubectl create -n zarf -f zarf-package-cpsh-platform-secret.yaml
    kubectl create -n zarf -f zarf-package-cpsh-lineage-secret.yaml
    kubectl create -n zarf -f zarf-package-cpsh-edge-fips-secret.yaml
    kubectl create -n zarf -f zarf-package-cpsh-observability-secret.yaml

  5. Remove the Zarf ignore annotation from existing namespaces:

    6. Copy 
    kubectl label ns collibra-control zarf.dev/agent-
    kubectl label ns collibra-edge zarf.dev/agent-
    kubectl label ns collibra-instance zarf.dev/agent-
    kubectl label ns collibra-observability zarf.dev/agent-

  6. Re-install the cpsh-* packages (the Zarf registry is empty now, this will re-push the container images). The workload should not usually be updated.

Operations

Periodically, you should prune the internal Zarf container registry to manage its size. To do this, run the following command:

Copy 
zarf tools registry prune

This command will remove the container images that are not currently being used by any Zarf packages (such as old versions of the cpsh-* packages). These images can be re-pushed to the registry by re-installing the corresponding Zarf package.