Configure a forward proxy

Important 

In Collibra 2024.05, we launched a new user interface (UI) for Collibra Data Intelligence Platform! You can learn more about this latest UI in the UI overview.

Use the following options to see the documentation in the latest UI or in the previous, classic UI:

Important The following procedure is one example for how to configure a proxy for your Edge site.

For security reasons, it is possible that an Edge site has to connect cloud services via a forward HTTP proxy. Complete steps 1-3 to update proxy.properties before installing the Edge site.

If you use a forward proxy that decrypts TLS traffic, a so-called man-in-the-middle (MITM) proxy, complete all 4 steps to configure the forward proxy and enable the MITM proxy.

Steps

  1. Download the Edge site installer:
    1. Open an Edge site.
      1. On the main toolbar, click Products icon, and then click Cogwheel icon Settings.
        The Collibra settings page opens.
      2. In the tab pane, click Edge.
        The Sites tab opens and shows a table with an overview of the Edge sites.
      3. In the Edge site overview, click the name of an Edge site.
        The Edge site page appears.
    2. In the Installer and properties files section, click Download.Click Download Installer.
    3. Depending on your operating system and browser, follow the regular steps for downloading files.
      The installer file is a TGZ archive that contains the files proxy.properties, properties.yaml and registries.yaml.

      Note If you download an installer, all previously downloaded installers become invalid.

  2. Open the proxy.properties file.
  3. Uncomment and update the outbound-proxy properties by removing "#" at the beginning of the following lines:
    • Proxy configuration K3S
    • Proxy configuration managed Kubernetes
    #noProxy=<host IP addresses>,<host DNS names>,<k8s-svc-ip-addresses>,<k8s-pod-ip-addresses,<others>
#proxyHost=<proxy domain name or IP address>
    #proxyPort=<proxy-port>
#proxyUsername=<proxy username>
#proxyPassword=<proxy password>

    Setting

    Value

    noProxy

    A comma-separated list of IP or DNS addresses that can bypass the proxy server.

    This list must include at least the Kubernetes cluster's internal IP addresses and the Kubernetes nodes' IP and DNS addresses.

    The list may not contain spaces.

    Tip To get the values for this setting, you can use the edge-get-noproxy.sh script, which you can find in the extracted installer directory under /resources/tools. However, make sure that your network administrator reviews these values.

    where:

    • <host-ip-addresses>: for example 172.20.0.0/16.
    • <host-dns-names>: for example *.compute.internal.
    • <k8s-svc-ip-addresses>: is by default 10.43.0.0/16, but this can differ for other k8s flavors or configurations.
    • <k8s-pod-ip-addresses>: is by default 10.42.0.0/16, but this can differ for other k8s flavors or configurations.
    • <others>: other IP addresses that don't need to be proxied. Add at least 169.254.169.254. for AWS.

    Example 
    noProxy=172.20.0.0/16,*.compute.internal,10.43.0.0/16,10.42.0.0/16,169.254.169.254

    proxyHost

    The IP or DNS address of the proxy server.

    Example proxyHost=site4-proxy.shared.edge.collibra.dev

    proxyPort

    The TCP port of the proxy server. This value must be a quoted string and not an integer value.

    Example "proxyPort=3128"

    proxyUsername

    The username to authenticate at the proxy server.

    Example proxyUsername=edge

    Note  Usernames with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if the username is ge'smith\, it would need to be entered into proxy.properties as username: ge\'smith\\.

    proxyPassword

    The password to authenticate at the proxy server.

    Example proxyPassword=la;fs90jpo4j3rR%

    Note Passwords with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if the password is te"st\1234', it would need to be entered into proxy.properties as password: te\"st\\1234\'.

    Tip 
    What is your Edge site installation method?
    Which Edge CLI method do you want to use?

    Warning Collibra Support will not assist with custom Helm or Kubernetes configurations. The following steps are an example, and any assistance for configurations or issues outside of these steps is unsupported. We recommend using the Edge CLI method for managed Kubernetes installations.

    A common example of custom Helm configurations is, but not limited to, using an unsupported private repository. At this time, we only support a JFrog repository.

    Note Edge supports the following managed Kubernetes clusters:
    • AKS
    • AWS Fargate using EKS
    • EKS
    • GKE
    • OpenShift

    Follow the steps below if you use a supported Kubernetes cluster.

    #noProxy=<host IP addresses>,<host DNS names>,<k8s-svc-ip-addresses>,<k8s-pod-ip-addresses,<others>
#proxyHost=<proxy domain name or IP address>
    #proxyPort=<proxy-port>
#proxyUsername=<proxy username>
#proxyPassword=<proxy password>

    Setting

    Value

    noProxy

    A comma-separated list of IP or DNS addresses that can bypass the proxy server.

    This list must include at least the Kubernetes cluster's internal IP addresses and the Kubernetes nodes' IP and DNS addresses.

    The list may not contain spaces.

    Tip To get the values for this setting, you can use the edge-get-noproxy.sh script. However, make sure that your network administrator reviews these values.

    where:

    • <host-ip-addresses>: for example 172.20.0.0/16.
    • <host-dns-names>: for example *.compute.internal.
    • <k8s-svc-ip-addresses>: depends on your managed kubernetes installation. Typically this is 10.100.0.0/16 or 172.20.0.0/16.
    • <k8s-pod-ip-addresses>: depends on your managed kubernetes installation. Typically they are the same subnets as in the VPC, for example 172.20.0.0/16.
    • <others>: other IP addresses that don't need to be proxied, for EKS, always add 169.254.169.254..

    Example 
    noProxy=172.20.0.0/16,*.compute.internal,10.100.0.0/16,169.254.169.254

    proxyHost

    The IP or DNS address of the proxy server.

    Example proxyHost=site4-proxy.shared.edge.collibra.dev

    proxyPort

    The TCP port of the proxy server. This value must be a quoted string and not an integer value.

    Example "proxyPort=3128"

    proxyUsername

    The username to authenticate at the proxy server.

    Example proxyUsername=edge

    Note Usernames with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if my username is ge'smith\, it would need to be entered into proxy.properties as username:ge\'smith\\.

    proxyPassword

    The password to authenticate at the proxy server.

    Example proxyPassword=la;fs90jpo4j3rR%

    Note Passwords with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if my password is te"st\1234', it would need to be entered into proxy.properties as password: te\"st\\1234\'.

    Important When you add a new node to a cluster, review and update, if necessary, the noProxy and implicitly forward proxy settings, unless the subnet used for nodes and their DNS suffix are added to noProxy.

    1. Base64 encode values the following values from the proxy.properties table and insert them into the secret template that applies to your setup:
      • Authenticated
        • http://"<proxyUsername>":"<proxyPassword>"@"<proxyHost>":"<proxyPort>"
        • noProxy
        • proxyHost
        • proxyPort
        apiVersion: v1
kind: Secret
metadata:
  name: outbound-proxy
  namespace: collibra-edge
data:                                                                                                                                                                                                                                                                      
  http_proxy: "<base64_from_bullet_1>"                                                                                                                                                                                                                           
  no_proxy: "<base_64_of_noProxy>"                                                                                                                                                                                                                          
  spring.cloud.gateway.httpclient.proxy.host: "<base_64_of_proxyHost>"                                                                                                                                                                                             
  spring.cloud.gateway.httpclient.proxy.port: "<base_64_of_proxyPort>"
type: Opaque
      • Unauthenticated
        • http://"<proxyHost>":"<proxyPort>"
        • noProxy
        • proxyHost
        • proxyPort
        
apiVersion: v1
kind: Secret
metadata:
  name: outbound-proxy						
  namespace: collibra-edge
data:                                                                                                                                                                                                                                                                      
  http_proxy: "<base64_from_bullet_1>"                                                                                                                                                                                                                           
  no_proxy: "<base_64_of_noProxy>"                                                                                                                                                                                                                          
  spring.cloud.gateway.httpclient.proxy.host: "<base_64_of_proxyHost>"                                                                                                                                                                                             
  spring.cloud.gateway.httpclient.proxy.port: "<base_64_of_proxyPort>"
type: Opaque
    2. Apply the newly created secret yaml file with kubectl:
      Copy
      kubectl apply -f "<name_of_secret_file>".yaml -n collibra-edge
    3. Optionally, if you're using a custom ca.pem (MITM proxy):
      1. Add the base64 output of your ca.pem file to the following template:
        Copy
        apiVersion: v1
        kind: Secret
        metadata:
          name: edge-ca.pem
          namespace: collibra-edge
        data:
          ca.pem: "<base64_of_your_ca.pem_file>"
        type: Opaque
      2. Apply the newly created ca secret:
        Copy
        kubectl apply -f edge-ca.yaml -n collibra-edge
  4. To enable Edge via a MITM proxy (a forward proxy that decrypts TLS traffic), follow the steps below:

    Note On-the-fly TLS certificates that are generated by the MITM proxy must use the subjectAltName (SAN) extension.

    1. Export your proxy server's CA certificate in PEM format.
      • When using your own "ca.pem" file be sure to only include the certificate or certificate chain of the MITM proxy. A custom "ca.pem" file cannot exceed 100kb.
    2. Save this certificate as "ca.pem" in the same directory as the Edge site installer.

      Note If you save the certificate in another directory, use the --ca argument in the Edge site installation command.

What's next?

  • If this is a new Edge installation, install the Edge site.
  • If you use a MITM proxy and the "ca.pem" has changed or was not included in the initial Edge installation, you must reinstall your Edge site.
  • If you want to update the forward proxy afterwards, you can use the update script.