NetScaler ingress controller

Deploy NetScaler Ingress Controller using YAML

You can deploy NetScaler Ingress Controller in the following modes on your bare metal and cloud deployments:

  • As a standalone pod in the Kubernetes cluster. Use this mode if you are controlling NetScalers (NetScaler MPX or NetScaler VPX) outside the cluster. For example, with dual-tier topologies, or single-tier topology where the single tier is a NetScaler MPX or VPX.

  • As a sidecar (in the same pod) with NetScaler CPX in the Kubernetes cluster. The sidecar controller is only responsible for the associated NetScaler CPX within the same pod. This mode is used in dual-tier or cloud topologies.

Deploy NetScaler Ingress Controller as a standalone pod in the Kubernetes cluster for NetScaler MPX or VPX appliances

Use the citrix-k8s-ingress-controller.yaml file to run the NetScaler Ingress Controller as a standalone pod in your Kubernetes cluster.

Note:

The NetScaler MPX or VPX can be deployed in standalone, high-availability, or clustered modes.

Prerequisites

  • Determine the NS_IP IP address needed by the controller to communicate with the appliance. The IP address might be anyone of the following depending on the type of NetScaler deployment:
    • (Standalone appliances) NSIP - The management IP address of a standalone NetScaler appliance. For more information, see IP Addressing in NetScaler
    • (Appliances in High Availability mode) SNIP - The subnet IP address. For more information, see IP Addressing in NetScaler
    • (Appliances in Clustered mode) CLIP - The cluster management IP (CLIP) address for a clustered NetScaler deployment. For more information, see IP addressing for a cluster
  • The user name and password of the NetScaler VPX or MPX appliance used as the Ingress device. The NetScaler appliance must have a system user account (non-default) with certain privileges so that NetScaler Ingress Controller can configure the NetScaler VPX or MPX appliance. For instructions to create the system user account on NetScaler, see Create System User Account for NetScaler Ingress Controller in NetScaler

    You can directly pass the user name and password as environment variables to the controller, or use Kubernetes secrets (recommended). If you want to use Kubernetes secrets, create a secret for the user name and password using the following command:

     kubectl create secret  generic nslogin --from-literal=username=<username> --from-literal=password=<password>
    

Create System User Account for NetScaler Ingress Controller in NetScaler

NetScaler Ingress Controller configures the NetScaler appliance (MPX or VPX) using a system user account of the NetScaler. The system user account should have certain privileges so that the NetScaler Ingress Controller has permission to configure the following on the NetScaler:

  • Add, Delete, or View Content Switching (CS) virtual server
  • Configure CS policies and actions
  • Configure Load Balancing (LB) virtual server
  • Configure Service groups
  • Cofigure SSL certkeys
  • Configure routes
  • Configure user monitors
  • Add system file (for uploading SSL certkeys from Kubernetes)
  • Configure Virtual IP address (VIP)
  • Check the status of the NetScaler appliance

To create the system user account, perform the following:

  1. Log on to the NetScaler appliance. Perform the following:
    1. Use an SSH client, such as PuTTy, to open an SSH connection to the NetScaler appliance.

    2. Log on to the appliance by using the administrator credentials.

  2. Create the system user account using the following command:

    add system user <username> <password>
    

    For example:

    add system user cic mypassword
    
  3. Create a policy to provide required permissions to the system user account. Use the following command:

    add cmdpolicy cic-policy ALLOW '^\(\?!shell)\(\?!sftp)\(\?!scp)\(\?!batch)\(\?!source)\(\?!.\*superuser)\(\?!.\*nsroot)\(\?!install)\(\?!show\s+system\s+\(user|cmdPolicy|file))\(\?!\(set|add|rm|create|export|kill)\s+system)\(\?!\(unbind|bind)\s+system\s+\(user|group))\(\?!diff\s+ns\s+config)\(\?!\(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).\*|\(^install\s\*\(wi|wf))|\(^\S+\s+system\s+file)^\(\?!shell)\(\?!sftp)\(\?!scp)\(\?!batch)\(\?!source)\(\?!.\*superuser)\(\?!.\*nsroot)\(\?!install)\(\?!show\s+system\s+\(user|cmdPolicy|file))\(\?!\(set|add|rm|create|export|kill)\s+system)\(\?!\(unbind|bind)\s+system\s+\(user|group))\(\?!diff\s+ns\s+config)\(\?!\(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).\*|\(^install\s\*\(wi|wf))|\(^\S+\s+system\s+file)'
    

    Note:

    The system user account would have privileges based on the command policy that you define.

    The command policy mentioned in step 3 is similar to the built-in sysAdmin command policy with additional permission to upload files.

    In the command policy specification provided, special characters which need to be escaped are already omitted to easily copy-paste into the NetScaler command line.

    For configuring the command policy from the NetScaler configuration wizard (GUI), use the following command policy specification.

    ^\(?!shell)\(?!sftp)\(?!scp)\(?!batch)\(?!source)\(?!.\*superuser)\(?!.\*nsroot)\(?!install)\(?!show\s+system\s+\(user|cmdPolicy|file))\(?!\(set|add|rm|create|export|kill)\s+system)\(?!\(unbind|bind)\s+system\s+\(user|group))\(?!diff\s+ns\s+config)\(?!\(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).\*|\(^install\s\*\(wi|wf))|\(^\S+\s+system\s+file)^\(?!shell)\(?!sftp)\(?!scp)\(?!batch)\(?!source)\(?!.\*superuser)\(?!.\*nsroot)\(?!install)\(?!show\s+system\s+\(user|cmdPolicy|file))\(?!\(set|add|rm|create|export|kill)\s+system)\(?!\(unbind|bind)\s+system\s+\(user|group))\(?!diff\s+ns\s+config)\(?!\(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).\*|\(^install\s\*\(wi|wf))|\(^\S+\s+system\s+file)
    
  4. Bind the policy to the system user account using the following command:

    bind system user cic cic-policy 0
    

Deploy NetScaler Ingress Controller as a pod

Perform the following:

  1. Download the citrix-k8s-ingress-controller.yaml using the following command:

    wget  https://raw.githubusercontent.com/citrix/citrix-k8s-ingress-controller/master/deployment/baremetal/citrix-k8s-ingress-controller.yaml
    
  2. Edit the citrix-k8s-ingress-controller.yaml file and enter the values for the following environmental variables:

    Environment Variable Mandatory or Optional Description
    NS_IP Mandatory The IP address of the NetScaler appliance. For more details, see Prerequisites.
    NS_USER and NS_PASSWORD Mandatory The user name and password of the NetScaler VPX or MPX appliance used as the Ingress device. For more details, see Prerequisites.
    EULA Mandatory The End User License Agreement. Specify the value as Yes.
    Kubernetes_url Optional The kube-apiserver url that NetScaler Ingress Controller uses to register the events. If the value is not specified, NetScaler Ingress Controller uses the internal kube-apiserver IP address.
    LOGLEVEL Optional The log levels to control the logs generated by NetScaler Ingress Controller. By default, the value is set to DEBUG. The supported values are: CRITICAL, ERROR, WARNING, INFO, and DEBUG. For more information, see Log Levels
    NS_PROTOCOL and NS_PORT Optional Defines the protocol and port that is used by the NetScaler Ingress Controller to communicate with NetScaler. By default, NetScaler Ingress Controller uses HTTP on port 80. You can also use HTTPS on port 443.
    ingress-classes Optional If multiple ingress load balancers are used to load balance different ingress resources. You can use this environment variable to specify the NetScaler Ingress Controller to configure NetScaler associated with specific ingress class. For information on Ingress classes, see Ingress class support
    NS_VIP Optional NetScaler Ingress Controller uses the IP address provided in this environment variable to configure a virtual IP address to the NetScaler that receives Ingress traffic. Note: NS_VIP acts as a fallback when the frontend-ip annotation is not provided in Ingress yaml. Not supported for Type Loadbalancer service.
    NS_APPS_NAME_PREFIX Optional By default, the NetScaler Ingress Controller adds “k8s” as prefix to the NetScaler entities such as, content switching (CS) virtual server, load balancing (LB) virtual server and so on. You can now customize the prefix using the NS_APPS_NAME_PREFIX environment variable in the NetScaler Ingress Controller deployment YAML file. You can use alphanumeric characters for the prefix and the prefix length should not exceed 8 characters.
    NAMESPACE Optional While running a NetScaler Ingress Controller with Role based RBAC, you must provide the namespace which you want to listen or get events. This namespace must be same as the one used for creating the service account. Using the service account, the NetScaler Ingress Controller can listen on a namespace. You can use the NAMESPACE environment variable to specify the namespace. For more information, see [Deploy the NetScaler Ingress Controller for a namespace(#deploy-the-citrix-ingress-controller-for-a-namespace).
    POD_IPS_FOR_SERVICEGROUP_MEMBERS Optional By default, while configuring services of type LoadBalancer and NodePort on an external tier-1 NetScaler the NetScaler Ingress Controller adds NodeIP and NodePort as service group members. If this variable is set as True, pod IP address and port are added instead of NodeIP and NodePort as service group members.
    IGNORE_NODE_EXTERNAL_IP Optional While adding NodeIP for services of type LoadBalancer or NodePort on an external tier-1 NetScaler, the NetScaler Ingress Controller prioritizes an external IP address over an internal IP address. When you want to prefer an internal IP address over an external IP address for NodeIP, you can set this variable to True.
    NS_DNS_NAMESERVER Optional Enables adding DNS nameservers on NetScaler VPX.
    NS_CONFIG_DNS_REC Optional Enables adding DNS records on NetScaler for Ingress resources. This variable is configured at the boot time and cannot be changed at runtime. Possible values are true or false. The default value is false and you need to set it as true to enable the DNS server configuration. When you set the value as ‘true’, the corresponding command add dns addrec <abc.com 1.1.1.1> is executed on NetScaler and an address record (mapping of the domain name to IP address) is created. For more information, see Create address records for a domain name.
    NS_SVC_LB_DNS_REC Optional Enables adding DNS records on NetScaler for services of type LoadBalancer. This variable is configured at the boot time and cannot be changed at runtime. Possible values are true or false. The default value is false and you need to set it as true to enable the DNS server configuration.
    SCOPE Optional Enables configuring the scope of NetScaler Ingress Controller as Role or ClusterRole binding. You can set the value of the SCOPE environment variable as local or cluster. When you set this variable as local, NetScaler Ingress Controller is deployed with Role binding that has limited privileges. You can use this option when you want to deploy NetScaler Ingress Controller with minimal privileges for a particular namespace with Role binding. By default, the value of SCOPE is set as cluster and NetScaler Ingress Controller is deployed with ClusterRole binding.
  3. Once you update the environment variables, save the YAML file and deploy it using the following command:

    kubectl create -f citrix-k8s-ingress-controller.yaml
    
  4. Verify if NetScaler Ingress Controller is deployed successfully using the following command:

    kubectl get pods --all-namespaces
    

Deploy NetScaler Ingress Controller as a sidecar with NetScaler CPX

Use the citrix-k8s-cpx-ingress.yaml file to deploy a NetScaler CPX with NetScaler Ingress Controller as a sidecar. The YAML file deploys a NetScaler CPX instance that is used for load balancing the North-South traffic to the microservices in your Kubernetes cluster.

Perform the following:

  1. Download the citrix-k8s-cpx-ingress.yaml using the following command:

    wget https://raw.githubusercontent.com/citrix/citrix-k8s-ingress-controller/master/deployment/baremetal/citrix-k8s-cpx-ingress.yml
    
  2. Deploy the citrix-k8s-cpx-ingress.yaml file using the following command:

    kubectl create -f citrix-k8s-cpx-ingress.yaml
    
  3. Verify if NetScaler Ingress Controller is deployed successfully using the following command:

    kubectl get pods --all-namespaces
    

Deploy NetScaler CPX with the NetScaler Ingress Controller as sidecar without the default credentials

Earlier, when you deploy NetScaler CPX with the NetScaler Ingress Controller as a sidecar without specifying the login credentials, the NetScaler Ingress Controller considers nsroot/nsroot as the default credentials.

With the latest NetScaler CPX versions (NetScaler CPX 13.0.64.35 and later), the default credentials are removed. So, when you deploy the NetScaler Ingress Controller as a sidecar with the latest versions of NetScaler CPX, the NetScaler Ingress Controller can get the credentials from NetScaler CPX through the /var/deviceinfo/random_id file in the NetScaler CPX. This file can be shared between the NetScaler CPX and the NetScaler Ingress Controller through the volume mount.

Depending on whether you are using the latest NetScaler CPX version or an older version, you need to choose one of the following deployment YAML files. For older versions of NetScaler CPX, you need to specify the credentials in the YAML file.

  • For NetScaler CPX 13.0.64.35 and later versions, use the following YAML:

citrix-k8s-cpx-ingress.yml

As provided in the YAML, the following is a snippet of the volume mount configuration required in the YAML file both for the NetScaler Ingress Controller and NetScaler CPX:

    volumeMounts:
    - mountPath: /var/deviceinfo
    name: shared-data

Following is a snippet of the shared volume configuration common for the NetScaler CPX and the NetScaler Ingress Controller.

    volumes:
    - name: shared-data
    emptyDir: {}
  • For earlier NetScaler CPX versions (versions earlier than 13.0.64.35), use the following YAML:

    cpx-ingress-previous.yaml

    Following is a snippet of the credential section in the NetScaler Ingress Controller:

     - name: "NS_USER"
     valueFrom:
     secretKeyRef:
             name: nslogin
             key: username
     - name: "NS_PASSWORD"
     valueFrom:
     secretKeyRef:
             name: nslogin
             key: password
    

Deploy the NetScaler Ingress Controller for a namespace

In Kubernetes, a role consists of rules that define a set of permissions that can be performed on a set of resources. In an RBAC enabled Kubernetes environment, you can create two kinds of roles based on the scope you need:

  • Role
  • ClusterRole

A role can be defined within a namespace with a Role, or cluster-wide with a ClusterRole. You can create a Role to grant access to resources within a single namespace.

In Kubernetes, you can create multiple virtual clusters on the same physical cluster. Namespaces provides a way to divide cluster resources between multiple users and useful in environments with many users spread across multiple teams, or projects.

By default, the NetScaler Ingress Controller monitors Ingress resources across all namespaces in the Kubernetes cluster. If multiple teams want to manage the same NetScaler, they can deploy a Role based NetScaler Ingress Controller to monitor only ingress resources belongs to a specific namespace. This namespace must be same as the namespace you have provided for creating the service account. You need to create a Role and bind the role to the service account for the NetScaler Ingress Controller. In this case, the NetScaler Ingress Controller listens only for events from the specified namespace and then configure the NetScaler accordingly.

You can use the SCOPE environment variable to configure the scope of NetScaler Ingress Controller as Role or ClusterRole binding. You can set the value of the SCOPE environment variable as local or cluster. When you set this variable as local, NetScaler Ingress Controller is deployed with minimal privileges for a particular namespace with Role binding. By default, the value of SCOPE is set as cluster and NetScaler Ingress Controller is deployed with the ClusterRole binding.

The following example shows a sample YAML file which defines a Role and RoleBinding for deploying a NetScaler Ingress Controller for a specific namespace.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: citrix
rules:
  - apiGroups: [""]
    resources:
      [
        "endpoints",
        "pods",
        "secrets",
        "nodes",
        "routes",
        "namespaces",
        "configmaps",
        "services",
      ]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["services/status"]
    verbs: ["patch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create"]
  - apiGroups: ["extensions"]
    resources: ["ingresses", "ingresses/status"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: ["networking.k8s.io"]
    resources: ["ingresses", "ingresses/status", "ingressclasses"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: ["apiextensions.k8s.io"]
    resources: ["customresourcedefinitions"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["citrix.com"]
    resources:
      [
        "rewritepolicies",
        "authpolicies",
        "ratelimits",
        "listeners",
        "httproutes",
        "continuousdeployments",
        "apigatewaypolicies",
        "wafs",
        "bots",
        "corspolicies",
        "appqoepolicies",
      ]
    verbs: ["get", "list", "watch", "create", "delete", "patch"]
  - apiGroups: ["citrix.com"]
    resources:
      [
        "rewritepolicies/status",
        "continuousdeployments/status",
        "authpolicies/status",
        "ratelimits/status",
        "listeners/status",
        "httproutes/status",
        "wafs/status",
        "apigatewaypolicies/status",
        "bots/status",
        "corspolicies/status",
        "appqoepolicies/status",
      ]
    verbs: ["patch"]
  - apiGroups: ["citrix.com"]
    resources: ["vips"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: ["route.openshift.io"]
    resources: ["routes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["crd.projectcalico.org"]
    resources: ["ipamblocks"]
    verbs: ["get", "list", "watch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: citrix
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: citrix
subjects:
  - kind: ServiceAccount
    name: citrix
    namespace: test
<!--NeedCopy-->

Restrictions

When the NetScaler Ingress Controller runs with a Role (scope with in a namespace), the following functionalities are not supported as they require global scope.

  • configuring static routes
  • watching on all namespaces
  • CRDs
Deploy NetScaler Ingress Controller using YAML