Load Balance East-West Traffic in Kubernetes Environment Using Citrix ADC CPX

You can deploy Citrix ADC CPX in a Kubernetes cluster to load balance containerized applications in the cluster. Citrix ADC CPX is supported on the following Kubernetes versions:

  • 1.5.x

  • 1.6.x

For information about Kubernetes, see http://kubernetes.io/docs/.

By default, when you deploy Citrix ADC CPX in the Kubernetes cluster, it replaces the Kubernetes’ kube-proxy that provides basic load balancing functionality. Replacing the kube-proxy with Citrix ADC CPX, in addition to the load balancing functionality, you can use the NetScaler Management and Analytics System (MAS) for:

  • Visibility into the application environment in the cluster

  • Managing and monitoring the Citrix ADC CPX instances in the cluster

  • Using the Stylebooks feature to simplify the task of managing complex NetScaler configurations for your applications

For more information on NetScaler MAS, see NetScaler Management and Analytics System Product Documentation.

How Citrix ADC CPX Load Balances East-West Traffic Flow in Kubernetes Environment

After you have deployed the Kubernetes cluster, you must integrate the cluster with NetScaler MAS by providing the details of the Kubernetes environment in NetScaler MAS. NetScaler MAS monitors the changes in Kubernetes resources, such as services, endpoints, and Ingress rules.

When you deploy a Citrix ADC CPX instance in the Kubernetes cluster, it automatically registers with NetScaler MAS. As part of the registration process, NetScaler MAS learns about the Citrix ADC CPX instance IP address and the port on which it can reach the instance to configure it by using NITRO REST APIs.

The Stylebook engine in NetScaler MAS processes all the information that NetScaler MAS collects from Kubernetes, such as services, endpoints, and Ingress rules. Using an existing provisioned Stylebook (com.citrix.adc.stylebooks/1.0/cs-lb-mon), the Stylebook engine generates NetScaler-specific configurations, such as the virtual servers and service groups required for load balancing, and applies the configurations to the Citrix ADC CPX instances. For more information about Stylebook, see Stylebooks.

The following figure shows how Citrix ADC CPX load balances east-west traffic flow in a Kubernetes cluster. localized image

In this example, Node 1 and Node 2 of the Kubernetes clusters contains instances of a front-end service and a back-end service. When the Citrix ADC CPX instances are deployed in Node 1 and Node 2, the Citrix ADC CPX instances are automatically registered with NetScaler MAS. You must manually integrate the Kubernetes cluster with NetScaler MAS by configuring the Kubernetes cluster details in NetScaler MAS.

When a client requests the front-end service, the ingress resource load balances the request between the instances of the front-end service on the two nodes. When an instance of the front-end service needs information from the back-end services in the cluster, it directs the requests to the Citrix ADC CPX instance in its node. That Citrix ADC CPX instance load balances the requests between the back-end services in the cluster, thereby providing east-west traffic flow.

Deploying a Citrix ADC CPX Instance on a Node in Kubernetes Cluster

You can deploy Citrix ADC CPX instances as Kubernetes pods on the nodes in a Kubernetes cluster. A Citrix ADC CPX instance can be deployed as a daemon set or as a manifest.

  • Daemon set – Deploying a Citrix ADC CPX instance as a daemon set resource enables you to deploy a Citrix ADC CPX instance as a pod in the node and also ensures that a Citrix ADC CPX instance is deployed on new nodes that join the Kubernetes cluster. When the new node joins the cluster, the Citrix ADC CPX instance specified in the daemons set is installed automatically on the node.

  • Manifest – A Kubernetes manifest is a YAML or JSON formatted file containing Kubernetes object deployment and configuration instructions. You can create a Kubernetes manifest of a Citrix ADC CPX instance and place it in a particular directory on the nodes. A kubelet on every node monitors this directory and creates objects, that is, Citrix ADC CPX instances, as specified by the manifest.

Prerequisites

For this type of deployment, make sure that you:

Deploying Citrix ADC CPX Instances as a Daemon Set

With the daemon set approach, you can deploy a Citrix ADC CPX instance as a pod on a node, and it is then automatically deployed as a pod on each new node that joins the Kubernetes cluster.

To deploy a Citrix ADC CPX instance as a daemon set, you must write a YAML file or a JSON script. The file or script specifies the container type, CPX image file name, NetScaler MAS server IP address, and NetScaler MAS server fingerprint.

The following is a sample YAML file:

    apiVersion: extensions/v1beta1

    kind: DaemonSet

    metadata:

      name: cpx

    spec:

      template:

        metadata:

          name: cpx

          labels:

            app: cpx-daemon

          annotations:

            NETSCALER_AS_APP: "True"

        spec:

          hostNetwork: true

          containers:

            - name: cpx

              image: "<repository>/cpx:12.0-64"

              securityContext:

                 privileged: true

              env:

              - name: "EULA"

                value: "yes"

              - name: "NS_NETMODE"

                value: "HOST"

              - name: "kubernetes_url"

                value: "https://10.217.212.231:6443"

              - name: "NS_MGMT_SERVER"

                value: "10.217.212.226"

              - name: "NS_MGMT_FINGER_PRINT"

                value: "74:EA:04:90:2C:FA:BF:7A:31:C9:52:64:D3:9C:BC:D3:O8:9F:9A:O4"

              - name: "NS_ROUTABLE"

                value: "FALSE"

              - name: "KUBERNETES_TASK_ID"

                valueFrom:

                   fieldRef:

                      fieldPath: metadata.name

              volumeMounts:

              imagePullPolicy: Always

The following table describes the sections, parameters, and environment variables used in the sample daemon set:

Section Parameter Description
container name Name of the Citrix ADC CPX container.
  image Specifies the image for container creation.
SecurityContext privileged: true Specifies that the Citrix ADC CPX container is run in privileged mode.
  name: “EULA” A Citrix ADC CPX specific environment variable, which is required for verification that you have read and understand the End User License Agreement (EULA) available at: https://www.citrix.com/products/netscaler-adc/cpx-express.html.
  name: “NS_NETMODE” A Citrix ADC CPX specific environment variable that allows you to specify that the Citrix ADC CPX instance is started in host mode. After the instance starts in host mode, it configures 4 default iptable rules on the host machine for management access to the instance. It uses the following ports: 9995 for HTTP, 9996 for HTTPS, 9997 for SSH and 9998 for SNMP. Also, If you want to specify different ports, you can use the following environment variables: -e NS_HTTP_PORT, -e NS_HTTPS_PORT, -e NS_SSH_PORT, and -e NS_SNMP_PORT.
  name: “kubernetes_url” A Citrix ADC CPX specific environment variable that specifies the Kubernetes URL.
  name: “NS_MGMT_SERVER” A Citrix ADC CPX specific environment variable that describes the NetScaler MAS server IP address. When the Citrix ADC CPX instance is deployed, it automatically registers with the NetScaler MAS server at this IP address.
  name: “NS_MGMT_FINGER_PRINT” A Citrix ADC CPX specific environment variable that defines the NetScaler MAS fingerprint.
  name: “NS_ROUTABLE” A Citrix ADC CPX specific environment variable specifying whether the Citrix ADC CPX container is run in non-IP-per-container mode. Be sure to set the value to FALSE.
  name: “KUBERNETES_TASK_ID” Identifies the Citrix ADC CPX ID in the Kubernetes cluster.
imagePullPolicy   Specifies how Kubernetes pulls the image.

Deploying a Citrix ADC CPX Instance Using a Manifest

A Kubernetes manifest is a YAML or JSON formatted file containing Kubernetes object deployment and configuration instructions. You can create a Kubernetes manifest of a Citrix ADC CPX instance and place it in a particular directory on the nodes. A kubelet on every node monitors this directory and creates objects, that is, Citrix ADC CPX instances, as specified by the manifest.

The following is a sample manifest:

    apiVersion: v1

    kind: Pod

    metadata:

        name: cpx

    annotations:

        NETSCALER_AS_APP: "True"

    spec:

        hostNetwork: true

        containers:

            - name: cpx

              image: "<repository>/cpx:12.0-64"

        securityContext:

        privileged: true

        env:

            - name: "EULA"

            value: "yes”

            - name: "NS_NETMODE"

            value: "HOST"

            - name: "kubernetes_url"

            value: "https://10.217.212.231:6443"

            - name: "NS_MGMT_SERVER"

            value: "10.217.212.226"

            - name: "NS_MGMT_FINGER_PRINT"

            value: "74:EA:04:90:2C:FA:BF:7A:31:C9:52:64:D3:9C:BC:D3:O8:9F:9A:O4"

            - name: "NS_ROUTABLE"

            value: "FALSE"

            - name: "KUBERNETES_TASK_ID"

            valueFrom:

                fieldRef:

                fieldPath: metadata.name

        imagePullPolicy: Always

The following table describes the sections, parameters, and environment variables used in the sample manifest:

Section Parameter Description
container name Name of the Citrix ADC CPX container.
  image Specifies the image for container creation.
SecurityContext privileged: true Specifies that the Citrix ADC CPX container is run in privileged mode.
  name: “EULA” A Citrix ADC CPX specific environment variable, which is required for verification that you have read and understand the End User License Agreement (EULA) available at: https://www.citrix.com/products/netscaler-adc/cpx-express.html.
  name: “NS_NETMODE” A Citrix ADC CPX specific environment variable that allows you to specify that the Citrix ADC CPX instance is started in host mode. After the instance starts in host mode, it configures 4 default iptable rules on the host machine for management access to the instance. It uses the following ports: 9995 for HTTP, 9996 for HTTPS, 9997 for SSH and 9998 for SNMP. Also, If you want to specify different ports, you can use the following environment variables: -e NS_HTTP_PORT, -e NS_HTTPS_PORT, -e NS_SSH_PORT, and -e NS_SNMP_PORT.
  name: “kubernetes_url” A Citrix ADC CPX specific environment variable that specifies the Kubernetes URL.
  name: “NS_MGMT_SERVER” A Citrix ADC CPX specific environment variable that describes the NetScaler MAS server IP address. When the Citrix ADC CPX instance is deployed, it automatically registers with the NetScaler MAS server at this IP address.
  name: “NS_MGMT_FINGER_PRINT” A Citrix ADC CPX specific environment variable that defines the NetScaler MAS fingerprint.
  name: “NS_ROUTABLE” A Citrix ADC CPX specific environment variable specifying whether the Citrix ADC CPX container is run in non-IP-per-container mode. Be sure to set the value to FALSE.
  name: “KUBERNETES_TASK_ID” Identifies the Citrix ADC CPX ID in the Kubernetes cluster.
imagePullPolicy   Specifies how Kubernetes pulls the image.

Load Balance East-West Traffic in Kubernetes Environment Using Citrix ADC CPX