Product Documentation

Configuring layer 3 clustering

Understanding L3 Cluster

The demand to expand the high availability deployment and increase the scalability of the client traffic across different networks guided to establish L3 cluster. The L3 cluster lets you to group Citrix ADC appliances across individual subnets (L2 cluster).

L3 cluster is also referred to as “cluster in Independent Network Configuration (INC) mode”.  In L3 cluster deployment, the cluster nodes in the same network are grouped together to form a Nodegroup. L3 cluster uses GRE tunneling to steer the packets across networks. The heartbeat messages across L3 clusters will be routed.

This document includes the following details:

  • Architecture
  • Example

Architecture

The L3 cluster architecture comprises of the following components:

  • Nodegroup. The cluster nodes from each network (n1, n2) and (n3, n4), as depicted in below figure, are grouped together to form a Nodegroup. These Nodegroups are terminated to the layer 3 switch on either side of the network.
    • The cluster communicates with the client through the physical connections between the cluster node and the client-side connecting device. The logical grouping of these physical connections is called the client data plane.
    • The cluster communicates with the server through the physical connections between the cluster node and the server side connecting device. The logical grouping of these physical connections is called the server data plane.
  • Backplane Switch. Cluster nodes within the same network communicate with each other by using the cluster backplane. The backplane is a set of interfaces in which one interface of each node is connected to a common switch, which is called the cluster backplane switch.
  • GRE Tunnel. The ** packets between nodes in a L3 cluster are exchanged over an unencrypted GRE tunnel that uses the NSIP addresses of the source and destination nodes for routing. The steering mechanism will change for nodes belonging to different network. The packets are steered through a GRE tunnel to the node on the other subnet, instead of rewriting the MAC.

localized image

Example

Consider an example of an L3 cluster deployment consisting of the following:

  • Three Citrix ADC appliances (n1, n2, and n3) nodes are grouped together into Nodegroup1.
  • Similarly, the nodes n4 and n5 are grouped in Nodegroup2. In the third network, there are two nodegroups. Nodegroup3 includes n6 and n7 and Nodegroup4 includes n8 and n9.
  • The Citrix ADC appliances that belong to the same network are combined together to form a Nodegroup.

localized image

Points to Consider before Configuring L3 Cluster

Consider the following points before configuring L3 cluster on a Citrix ADC appliance:

  • The backplane is not mandatory while configuring L3 subnets. If the backplane is not specified, the node will not go to backplane fail state.

    Note

    If you have some cluster nodes in the L2 network, it is mandatory to enable steering on the cluster backplane, else the nodes will go to backplane fail state.

  • The external traffic distribution in L3 cluster supports only Equal Cost Multiple Path (ECMP).
  • The following parameters are processed when steering is disabled is an L3 cluster deployment:
    • ICMP errors
    • Fragmentation
    • Striped SNIPs or MIPs
  • The entities (route, route6, pbr, and pbr6) can be bound to configuration nodegroup.
  • VLAN, RNAT, and IP tunnel cannot be bound to a config nodegroup.
  • Config nodegroup should always have property  STRICT “YES.
  • The cluster nodes should not be added to a config nodegroup via “add cluster node” command.
  • The “clear config extended+” command will not clear the entities (route, route6, pbr, pb6, rnat, IP tunnel, ip6tunnel). These entities should be cleared when an “add cluster instance –INC enabled” command is configured.

Configuring L3 Cluster

In an L3 cluster configuration, the cluster command has different attributes to configure that is based on nodes, and nodegroups. The L3 cluster configuration also includes an IPv6 profile apart from IPv4 profiles.

Configuring L3 cluster on a Citrix ADC appliance consists of the following tasks:

  • Create a cluster instance
  • Create a nodegroup in L3 cluster
  • Add a Citrix ADC appliance to the cluster and group with nodegroup
  • Add cluster IP address to the node
  • Enable the cluster instance
  • Save the configuration
  • Add a new node to an existing nodegroup
  • Create a new nodegroup in L3 cluster
  • Group new nodes to the newly created nodegroup
  • Join the node to the cluster

Configuring the following by Using the Command Line

  • To create a cluster instance by using the Citrix ADC CLI

    add cluster instance <clid> -inc <ENABLED DISABLED> -processLocal <ENABLED DISABLED>
  • To create a nodegroup in L3 cluster

    add cluster nodegroup <ng>

  • To add a Citrix ADC appliance to the cluster and to associate with nodegroup

    add cluster node <nodeid> <nodeip> -nodegroup <ng>

  • To add the cluster IP address on this node

    add ns ip <IPAddress> <netmask> -type clip

  • Enable the cluster instance

    enable cluster instance <clId>

  • Save the configuration

    save ns config

  • Warm reboot the appliance

reboot -warm

  • To add a new node to an existing nodegroup

    add cluster node <nodeid> <nodeip> -nodegroup <ng>

  • To create a new nodegroup in L3 cluster

    add cluster nodegroup <ng>

  • To group new nodes to the newly created nodegroup

    add cluster node <nodeid> <nodeip> -nodegroup <ng>

  • To join the node to the cluster

   join cluster –clip \<ip\_addr\> -password \<password\>

    add cluster instance 1 –inc ENABLED –processLocal ENABLED

       Done

Note

The “inc” parameter should be ENABLED for an L3 cluster.

    > add cluster nodegroup ng1

       Done

    > add cluster node 0 1.1.1.1 –state ACTIVE –nodegroup ng1

       Done

    > add ns ip 1.1.1.100 255.255.255.255 –type clip

       Done

    > enable cluster instance 1

       Done

    > save ns config

       Done

    > add cluster node 1 1.1.1.2 –state ACTIVE –nodegroup ng1

       Done

    > add cluster nodegroup ng2

       Done

    > add cluster node 4 2.2.2.1 –state ACTIVE –nodegroup ng2

       Done

    > add cluster node 5 2.2.2.2 –state ACTIVE –nodegroup ng2

       Done

    > join cluster -clip 1.1.1.100 -password nsroot

Advertising Cluster IP address of a Layer 3 Cluster

You must configure the cluster IP address to be advertised to the upstream router to make the cluster configuration accessible from any subnet.  The cluster IP address is advertised as a kernel route by the dynamic routing protocols configured on a node.

Advertising the cluster IP address consists of the following tasks:

  • Enable the host route option of the cluster IP address. The host route option pushes the cluster IP address to ZebOS routing table for kernel route redistribution through dynamic routing protocols.
  • Configuring a dynamic routing protocol on a node. A dynamic routing protocol advertises the cluster IP address to the upstream router. For more information on configuring a dynamic routing protocol, see Configuring Dynamic Routes.

To enable the host route option of the cluster IP Address by using the Citrix ADC CLI

  At the command prompt, type:

-  add nsip \<IPAddress\> \<netmask\> -**hostRoute ENABLED**
-  **show nsip** \<IPAddress\>

    > add ns ip 10.102.29.60 255.255.255.255 -hostRoute ENABLED

       Done

Spotted, partially striped configurations on L3 cluster

The spotted and partially striped configurations on L3 cluster slightly differs from L2 cluster. The configuration might differ from node to node as the nodes reside on different subnets. The network configurations can be node specific in L3 cluster, hence you have to configure the spotted or partially striped configurations based on the below-mentioned parameters.

To configure spotted, partially striped configurations on a Citrix ADC appliance over L3 cluster perform the following tasks:

-  Add a cluster ownergroup to an IPv4 static routing table
-  Add a cluster ownergroup to an IPv6 static routing table  
-  Add a cluster ownergroup to an IPv4 policy based routing (PBR)  
-  Add a cluster ownergroup to an IPv6 PBR
-  Add a VLAN
-  Bind a VLAN to a specific ownergroup of cluster nodegroup

Configuring the following by Using the Command Line

  • To add a cluster ownergroup to an IPv4 static route table of the Citrix ADC appliance

    add route <network> <netmask> <gateway> -ownergroup <ng>

  • To add a cluster ownergroup to an IPv6 static route table of the Citrix ADC appliance

    add route6 <network> -ownergroup <ng>

  • To add a cluster ownergroup to an IPv4 PBR

    add pbr <name>  <action> -ownergroup <ng>

  • To add a cluster ownergroup to an IPv6 PBR

    add pbr6 <name>  <action> -ownergroup <ng>

  • To add a VLAN

    add vlan <id>

  • To bind a VLAN to a specific ownergroup of cluster nodegroup

    bind vlan \<id\> -ifnum – \[IPAddress \<ip\_addr | ipv6\_addr |\>  \[-ownergroup \<ng\>\]

    The following commands are sample examples of spotted and partially striped configurations which can be configured by using the Citrix ADC CLI.

    > add route 10.102.29.0 255.255.255.0 10.102.29.2 –ownergroup ng2

        Done

    > add route6 fe80::9404:60ff:fedd:a464/64 –ownergroup ng1

        Done

    > add pbr pbr1 allow –ownergroup ng1

        Done

    > add pbr6 pbr2 allow –ownergroup ng2

        Done

    > add vlan 2

        Done

    > bind vlan 2 –ifnum 1/2 –[IPAddress 10.102.29.80 | fe80::9404:60ff:fedd:a464/64-ownergroup ng1

        Done

Configure nodegroup

In an L3 cluster, to replicate the same set of configurations on more than one nodegroup, the following commands are used:

Configuring the following by Using the Command Line

  • To add an IPv4 static route to the routing table of the Citrix ADC appliance

    add route <network> <netmask> <gateway> -ownerGroup <ng>

Sample Configuration

add route 0 0 10.102.53.1 –ownerGroup ng1

add route 0 0 10.102.53.1 –ownerGroup ng2

To support the above configuration, a new nodegroup ‘all’ has to be defined and you have to configure the following commands:

Configuring the following by Using the Command Line

  • To add a new nodegroup to cluster with strict parameter

    add cluster nodegroup \<name\> -strict \<YES | NO\>

  • To bind a cluster node or an entity to the given nodegroup

    bind cluster nodegroup <name> -node <nodeid>

  • To add IPv4 static route to all ownergroup

    add route <network> <netmask> <gateway> -ownerGroup <ng>

Sample configuration:

add cluster nodegroup all –strict YES

bind cluster nodegroup all –node 1

bind cluster nodegroup all –node 2

add route 0 0 10.102.53.1 –ownerGroup all

Configuring layer 3 clustering