Configure high availability deployment

High Availability (HA) refers to a system that is always available to a user without any interruption to the services. High availability setup is crucial during system downtime, network or application failures, and is a key requirement to any enterprise. A high availability deployment of two Citrix ADM nodes in active-passive mode with same configurations provides uninterrupted operations.

Important

To access Citrix ADM 12.1 build 48.18 or later versions using HTTPS:

If you have configured a NetScaler appliance to load balance Citrix ADM in a high availability mode, first remove the NetScaler appliance. Then, configure a floating IP address to access Citrix ADM in high availability mode.

The following are the benefits of high availability deployment in Citrix ADM:

  • An improved mechanism to monitor heartbeats between the primary and secondary node.

  • Provides physical streaming replication of database instead of a logical bi-directional replication.

  • Ability to configure the floating IP address on the primary node to eliminate the need of separate NetScaler load balancer.

  • Provides easy access to the Citrix ADM user interface using the floating IP address.

  • Citrix ADM user interface is provided only on the primary node. By using the primary node, you can eliminate the risk of accessing and making changes to the secondary node.

  • Configuring the floating IP address handles the failover situation and reconfiguring the instances is not required.

  • Provides built in ability to detect and handle split-brain situation.

The following table describes the terms used in high availability deployment.

Terms Description
Primary node First node registered in the high availability deployment.
Secondary node Second node registered in the high availability deployment.
Heartbeat A mechanism used to exchange messages between primary and secondary node in the high availability setup. The messages determine status and health of the application on each individual node.
Floating IP address A floating IP is an IP address that can be instantly moved from one node to another in the same subnet. Internally it is set up as an alias on the network interface of the primary node. If there is a failover, the floating IP address is seamlessly moved from the old primary to the new one. It is useful in high availability setup because it allows clients to communicate with the high availability nodes using a single IP address.

Note

For more information on port and protocol details, see Ports.

Components of high availability architecture

The following figure displays the architecture of two Citrix ADM nodes deployed in high availability mode.

localized image

In high availability deployment, one Citrix ADM node is configured as the primary node (MAS 1) and the other as the secondary node (MAS 2). If the primary node goes down due to any reason, the secondary node takes over as the new primary node.

Monitoring tool

Monitoring tool is an internal process used to monitor, alert, and handle failover situations. The tool is active and running on each node in high availability. It is responsible for starting subsystems, initiating database on both the nodes, deciding on the primary, or secondary node if there is a failover, and so on.

Primary node (MAS 1)

The primary node accepts connections and manages the instances. All processes such as AppFlow, SNMP, LogStream, syslog, and so on is managed by the primary node. The Citrix ADM user interface access is available on primary node. The floating IP address is configured on the primary node.

Secondary node (MAS 2)

The secondary node listens to the heartbeat messages sent from the primary node. Database on the secondary node is in read-replica mode only. None of the processes are active in the secondary node and the Citrix ADM user interface is not accessible on the secondary node.

Physical streaming replication

The primary and secondary nodes synchronize through heartbeat mechanism. With the physical streaming replication of database, the secondary node starts in read-replica mode. The secondary node listens to the heartbeat messages received from the primary node. If the secondary node does not receive any heartbeats for a time period of 180 seconds, the primary node is considered to be down. Then, the secondary node takes over as the primary node.

Heartbeat messages

Heartbeat messages are User Datagram Packets (UDP) that are sent and received between primary and secondary node. It monitors all subsystems of Citrix ADM and database to exchange information about the node state, health, processes, and so on. The information is shared between the high availability nodes every second. Notifications are sent as alerts to the administrator if there is a failover or break up of high availability states.

Floating IP address

The floating IP address is associated with the primary node in the high availability setup. It is an alias given to the primary node IP address, that the client can use to connect to Citrix ADM in the primary node. Since the floating IP address is configured on the primary node, the instance reconfiguration is not required in case of failover. The instances reconnect to the same IP address to reach the new primary.

Key points to note

  • In a high availability setup, both the Citrix ADM nodes must be deployed in active-passive mode. They must be on the same subnets using the same software version and build, and have same configurations.

  • Floating IP address:

    • Floating IP address is configured on the primary node.

    • Instances need not be reconfigured if there is a failover.

    • You can access a high availability node from the user interface, either by using the primary node IP or floating IP address.

Note

Citrix recommends that you use the floating IP address to access the user interface.

  • Database:

    • In a high availability setup, all configuration files are synchronized automatically from the primary node to the secondary node at an interval of one minute.

    • Database synchronization happens instantly by physical replication of database.

    • Database on secondary node is in read-replica mode.

  • Citrix ADM upgrade:

    • Internal processes implicitly upgrades Citrix ADM from the earlier versions.

Note

After the upgrade is successful, you must configure the floating IP address.

  • UDP default port 5005 is available on both the nodes for heartbeats to be sent and for messages to be received.

  • You can make configuration changes on both the primary and secondary nodes. Citrix recommends making them on the primary node.

  • MAC address
    The setting for the “MAC Address Changes” option in a hypervisor affects the traffic that a virtual machine receives. Allow MAC address changes to be enabled on the virtual switch so that the floating IP address moves seamlessly to the new primary node after failover. For example, when deploying Citrix ADM on a high availability on Vmware ESXi, make sure you accept changes to MAC address. ESXi now allows requests to change the active MAC address to other than the initial MAC address.

    localized image

Prerequisites

Before you set up high availability for Citrix ADM nodes, note the following prerequisites:

  • The Citrix ADM high availability deployment is supported from Citrix ADM version 12.0 build 51.24.

  • Download the NetScaler Management and Analytics System image file (.xva) from the Citrix download site: https://www.citrix.com/downloads/

Citrix recommends that you set CPU priority (in virtual machine properties) at the highest level to improve scheduling behavior and network latency.

The following table lists the minimum requirements for the virtual computing resources:

Component Requirement
RAM 32 GB
  Note: The default value is 8 GB. Citrix recommends that you increase the default value to 32 GB for better performance.
Virtual CPU 8 CPUs
  Note: The default is 2 CPUs. Citrix recommends that you increase the default value to 8 CPUs for better performance.
Storage Space Citrix recommends using solid-state drive (SSD) technology for Citrix ADM deployments. The default value is 120 GB. Actual storage requirement depends on Citrix ADM sizing estimation. If your Citrix ADM storage requirement exceeds 120 GB, you have to attach an additional disk. Note: You can add only one additional disk. Citrix recommends you to estimate storage and attach additional disk at the time of initial deployment. For more information, see How to Attach an Additional Disk to Citrix ADM.
Virtual network interfaces 1
Throughput 1 Gbps or 100 Mbps
Hypervisor Versions
Citrix XenServer 6.2 and 6.5
VMware ESXi 5.5 and 6.0
Microsoft Hyper-V 2012 R2
Linux KVM Ubuntu and Fedora

To set up Citrix ADM in high availability mode

  1. Register and deploy the first server (primary node).

  2. Register and deploy the second server (secondary node).

  3. Deploy the primary and secondary node for high availability setup.

Register and deploy the first server (primary node)

To register the first node:

  1. Use the .xva image file downloaded from the Citrix download site and import it in to your hypervisor.

    Note

    It might take a few minutes for the .xva image file to import and get started. You can see the status on the bottom of the screen.

    localized image

  2. After the import is successful, right-click and click Start.

    localized image

  3. From the Console tab, configure NetScaler MAS with the initial network configurations as displayed in the following image.

    localized image

  4. After the initial network configuration is complete, the system prompts for login. Log on using following credentials – nsrecover/nsroot.

  5. To deploy the primary node, enter /mps/deployment_type.py. The NetScaler MAS deployment configuration menu is displayed.

    localized image

  6. Select 1 to register NetScaler MAS server as primary node.

    localized image

  7. The console prompts you to select the NetScaler MAS standalone deployment. Enter No to confirm the deployment as high availability.

    localized image

  8. The console prompts you to select the First Server Node. Enter Yes to confirm the node as the first node.

    localized image

  9. The console prompts you to restart the system. Enter Yes to restart.

    localized image

    The system restarts and is displayed as the primary node in the NetScaler MAS user interface.

Register and deploy the second server (secondary node)

  1. Use the .xva image file downloaded from the Citrix download site and import it in to your hypervisor.

  2. From the Console tab, configure NetScaler MAS with the initial network configurations as displayed in the following image.

  3. After the initial network configuration is completed, the system prompts for login. Log on using following credentials – nsrecover/nsroot.

  4. To deploy the secondary node, enter /mps/deployment_type.py. The NetScaler MAS deployment configuration menu is displayed.

  5. Select 1 to register NetScaler MAS server as secondary node.

  6. The console prompts you to select the NetScaler MAS as standalone deployment. Enter No to confirm the deployment as high availability.

  7. The console prompts you to select the first server node. Enter No to confirm the node as the second server.

    localized image

  8. The console prompts you to enter the IP address and password of the primary node.

    localized image

  9. The console prompts you to enter the floating IP address.

    localized image

  10. The console prompts you to restart the system. Enter Yes to restart.

    localized image

    Note

    • Floating IP address is mandatory for high availability deployment of nodes.

    • The system will show error messages if there are any issues in the configuration.

    • The system reboots and takes a few minutes for the configurations to take effect.

    • You may not be able to access the GUI immediately after the configuration is completed. You need to try accessing the GUI after a few minutes.

Deploy the primary and secondary node as a high availability pair

After the registration both primary and secondary nodes are displayed on the NetScaler MAS user interface. Deploy these nodes into a high availability pair.

Note

  • Before deploying the nodes into a high availability pair, ensure that the secondary node is completed with a reboot, after the initial network configuration.

  • After the high availability deployment is complete, use the floating IP address to access the NetScaler MAS user interface.

To deploy nodes as a high availability pair:

  1. Open a web browser and enter the IP address of the first NetScaler MAS server node.  

  2. In the user Name and password fields, enter the administrator credentials.

  3. Select the deployment type as Two Servers deployed in High Availability Mode, and click Next.

    localized image

  4. On the System tab, navigate to Deployment and click Deploy.

    localized image

  5. A confirmation message is displayed. Click Yes.

    localized image

After you deploy the NetScaler MAS in high availability mode, the primary and secondary nodes are displayed.

localized image

Disable high availability

You can disable high availability on a NetScaler MAS high availability pair and convert the nodes to standalone NetScaler MAS servers.

Note

Disable high availability from the primary node.

To disable the high availability:

  1. In a web browser, enter the IP address of the NetScaler MAS server primary node.

  2. In the User Name and Password fields, enter the administrator credentials.

  3. On the System tab, navigate to Deployment and click Break HA.

    localized image

A dialogue box is displayed. Click Yes to break the high availability deployment.

Redeploy high availability

After you disable the high availability to a standalone deployment, you can redeploy it to high availability mode again. Redeploying high availability is similar to the first time deployment of high availability. For more details see Deploy the primary and secondary node as a high availability pair.

High availability failover scenarios

A failover occurs if one of the following conditions is encountered:

  • Node failure: Primary node goes down, no heartbeat is detected from primary node for 180 seconds.

  • Application health failure: Primary node is up and running but one of the Citrix ADM processes is down.

Split-brain scenario

When there is no communication between both the nodes due to downtime in network link, then:

  • Primary node continues to operate as primary

  • Secondary node takes over as primary because of the failure to receive heartbeats

  • Both the nodes would run their individual database instances

For example, in an enterprise two Citrix ADM nodes have been deployed as primary and secondary. Due to a possible network link downtime, the communication between the two Citrix ADM nodes breaks completely. Since there is no heartbeat exchange for over 180 seconds, both the nodes consider themselves to be the primary node. Both nodes act as active nodes and run their own instances of database.

With Citrix ADM 12.1, this split-brain situation is handled gracefully after the network link and heartbeat is restored. High availability synchronization is restored automatically. The recovery time depends on the data and speed of the link between the nodes.

Note

During the split-brain condition, changes that occurred on the old primary node is reset with the new primary when it is rejoined in high availability. The changes that happened on new primary node during split-brain remains intact.