Configure a forward proxy

Important 

In Collibra 2024.05, we've launched a new user interface (UI) for Collibra Data Intelligence Platform! You can learn more about this latest UI in the UI overview.

Use the following options to see the documentation in the latest UI or in the previous, classic UI:

For security reasons, it is possible that an Edge site has to connect cloud services via a forward HTTP proxy. Complete steps 1-3 to update proxy.properties before installing the Edge site.

If you use a forward proxy that decrypts TLS traffic, a so-called man-in-the-middle (MITM) proxy, complete all 4 steps to configure the forward proxy and enable the MITM proxy.

Warning MITM is not supported by all capabilities. Review the capability documentation to confirm if MITM is supported for your capability.

Steps

  1. Download the Edge site installer:
    1. Open an Edge site.
      1. On the main toolbar, click Products icon, and then click Cogwheel icon Settings.
        The Collibra settings page opens.
      2. Click Edge.
        The Edge sites overview opens.
      3. Click the name of an Edge site in the Edge site overview.
        The Edge site page opens.
    2. In the Installer and properties files section, click Download.Click Download Installer.
    3. Depending on your operating system and browser, follow the regular steps for downloading files.
      The installer file is a TGZ archive that contains the files proxy.properties, properties.yaml and registries.yaml.

      Note If you download an installer, all previously downloaded installers become invalid.

  2. Open the proxy.properties file.
  3. Uncomment and update the outbound-proxy properties by removing "#" at the beginning of the following lines:
    • Proxy configuration K3S
    • Proxy configuration managed Kubernetes
    #noProxy=<host IP addresses>,<host DNS names>,<k8s-svc-ip-addresses>,<k8s-pod-ip-addresses,<others>
#proxyHost=<proxy domain name or IP address>
    #proxyPort=<proxy-port>
#proxyUsername=<proxy username>
#proxyPassword=<proxy password>

    Setting

    Value

    noProxy

    A comma-separated list of IP or DNS addresses that can bypass the proxy server.

    This list must include at least the Kubernetes cluster's internal IP addresses and the Kubernetes nodes' IP and DNS addresses.

    The list may not contain spaces.

    Tip To get the values for this setting, you can use the edge-get-noproxy.sh script, which you can find in the extracted installer directory under /resources/tools. However, make sure that your network administrator reviews these values.

    where:

    • <host-ip-addresses>: for example 172.20.0.0/16.
    • <host-dns-names>: for example *.compute.internal.
    • <k8s-svc-ip-addresses>: is by default 10.43.0.0/16, but this can differ for other k8s flavors or configurations.
    • <k8s-pod-ip-addresses>: is by default 10.42.0.0/16, but this can differ for other k8s flavors or configurations.
    • <others>: other IP addresses that don't need to be proxied. Add at least 169.254.169.254. for AWS.

    Example 
    noProxy=172.20.0.0/16,*.compute.internal,10.43.0.0/16,10.42.0.0/16,169.254.169.254

    proxyHost

    The IP or DNS address of the proxy server.

    Example proxyHost=site4-proxy.shared.edge.collibra.dev

    proxyPort

    The TCP port of the proxy server. This value must be a quoted string and not an integer value.

    Example proxyPort=3128

    proxyUsername

    The username to authenticate at the proxy server.

    Example proxyUsername=edge

    Note  Usernames with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if the username is ge'smith\, it would need to be entered into proxy.properties as username: ge\'smith\\.

    proxyPassword

    The password to authenticate at the proxy server.

    Example proxyPassword=la;fs90jpo4j3rR%

    Note Passwords with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if the password is te"st\1234', it would need to be entered into proxy.properties as password: te\"st\\1234\'.

    Tip 
    What is your Edge site installation method?

    Important You should only use the Helm Chart installation method if you are familiar with helm and Kubernetes. The following information is only an example, and as such, Collibra Support is limited to this specific setup and cannot assist with custom helm and Kubernetes configurations.

    Note Edge supports the following managed Kubernetes clusters:
    • AKS
    • AWS Fargate using EKS
    • EKS
    • GKE
    • OpenShift

    Follow the steps below if you use a supported Kubernetes cluster.

    #noProxy=<host IP addresses>,<host DNS names>,<k8s-svc-ip-addresses>,<k8s-pod-ip-addresses,<others>
#proxyHost=<proxy domain name or IP address>
    #proxyPort=<proxy-port>
#proxyUsername=<proxy username>
#proxyPassword=<proxy password>

    Setting

    Value

    noProxy

    A comma-separated list of IP or DNS addresses that can bypass the proxy server.

    This list must include at least the Kubernetes cluster's internal IP addresses and the Kubernetes nodes' IP and DNS addresses.

    The list may not contain spaces.

    Tip To get the values for this setting, you can use the edge-get-noproxy.sh script. However, make sure that your network administrator reviews these values.

    where:

    • <host-ip-addresses>: for example 172.20.0.0/16.
    • <host-dns-names>: for example *.compute.internal.
    • <k8s-svc-ip-addresses>: depends on your managed kubernetes installation. Typically this is 10.100.0.0/16 or 172.20.0.0/16.
    • <k8s-pod-ip-addresses>: depends on your managed kubernetes installation. Typically they are the same subnets as in the VPC, for example 172.20.0.0/16.
    • <others>: other IP addresses that don't need to be proxied, for EKS, always add 169.254.169.254..

    Example 
    noProxy=172.20.0.0/16,*.compute.internal,10.100.0.0/16,169.254.169.254

    proxyHost

    The IP or DNS address of the proxy server.

    Example proxyHost=site4-proxy.shared.edge.collibra.dev

    proxyPort

    The TCP port of the proxy server. This value must be a quoted string and not an integer value.

    Example proxyPort=3128

    proxyUsername

    The username to authenticate at the proxy server.

    Example proxyUsername=edge

    Note Usernames with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if my username is ge'smith\, it would need to be entered into proxy.properties as username:ge\'smith\\.

    proxyPassword

    The password to authenticate at the proxy server.

    Example proxyPassword=la;fs90jpo4j3rR%

    Note Passwords with single quotations ', double quotations ", and backslashes \ need to be escaped using an additional backslash. For example, if my password is te"st\1234', it would need to be entered into proxy.properties as password: te\"st\\1234\'.

    Important When you add a new node to a cluster, review and update, if necessary, the noProxy and implicitly forward proxy settings, unless the subnet used for nodes and their DNS suffix are added to noProxy.

    1. Base64 encode values the following values from the proxy.properties table and insert them into the secret template that applies to your setup:
      • Authenticated
        • http://"<proxyUsername>":"<proxyPassword>"@"<proxyHost>":"<proxyPort>"
        • noProxy
        • proxyHost
        • proxyPort
        apiVersion: v1
kind: Secret
metadata:
  name: outbound-proxy
  namespace: collibra-edge
data:                                                                                                                                                                                                                                                                      
  http_proxy: "<base64_from_bullet_1>"                                                                                                                                                                                                                           
  no_proxy: "<base_64_of_noProxy>"                                                                                                                                                                                                                          
  spring.cloud.gateway.httpclient.proxy.host: "<base_64_of_proxyHost>"                                                                                                                                                                                             
  spring.cloud.gateway.httpclient.proxy.port: "<base_64_of_proxyPort>"
type: Opaque
      • Unauthenticated
        • http://"<proxyHost>":"<proxyPort>"
        • noProxy
        • proxyHost
        • proxyPort
        apiVersion: v1
kind: Secret
metadata:
  name: outbound-proxy						
  namespace: collibra-edge
data:                                                                                                                                                                                                                                                                      
  http_proxy: "<base64_from_bullet_1>"                                                                                                                                                                                                                           
  no_proxy: "<base_64_of_noProxy>"                                                                                                                                                                                                                          
  spring.cloud.gateway.httpclient.proxy.host: "<base_64_of_proxyHost>"                                                                                                                                                                                             
  spring.cloud.gateway.httpclient.proxy.port: "<base_64_of_proxyPort>"
type: Opaque
    2. Apply the newly created secret yaml file with kubectl:
      kubectl apply -f "<name_of_secret_file>".yaml -n collibra-edge
    3. Optionally, if you're using a custom ca.pem (MITM proxy):
      1. Add the base64 output of your ca.pem file to the following template: 
        apiVersion: v1
kind: Secret
metadata:
  name: edge-ca.pem
  namespace: collibra-edge
data:
  ca.pem: "<base64_of_your_ca.pem_file>"
type: Opaque
      2. Apply the newly created ca secret:
        kubectl apply -f edge-ca.yaml -n collibra-edge
  4. To enable Edge via a MITM proxy (a forward proxy that decrypts TLS traffic), follow the steps below:

    Note On-the-fly TLS certificates that are generated by the MITM proxy must use the subjectAltName (SAN) extension.

    1. Export your proxy server's CA certificate in PEM format.
      • When using your own ca.pem file be sure to only include the certificate or certificate chain of the MITM proxy. A custom ca.pem file cannot exceed 100kb.
    2. Save this certificate as ca.pem in the same directory as the Edge site installer.

      Note If you save the certificate in another directory, use the --ca argument in the Edge site installation command.

What's next?

  • If this is a new Edge installation, install the Edge site.
  • If you use a MITM proxy and the ca.pem has changed or was not included in the initial Edge installation, you must reinstall your Edge site.
  • If you want to update the forward proxy afterwards, you can use the update script.