In Service Software Upgrade support for high availability for performing zero downtime upgrade

During a regular upgrade process in a high availability setup, at some point, both nodes run different software builds. These two builds can have the same or different internal high availability version numbers.

If both the builds have different high availability version numbers, connection failover (even if it is enabled) for existing data connections is not supported. In other words, all existing data connections are lost, which leads to downtime.

To address this issue, in Service Software Upgrade (ISSU) can be used for high availability set-ups. ISSU introduces a migration functionality, which replaces the force failover operation step in the upgrade process. The migration functionality takes care of honoring the existing connections and includes the force failover operation.

After migration operation is performed, the new primary node always receives traffic (request and response) related to the existing connections but steers them to the old primary node. The old primary node processes the data traffic and then sends them directly to the destination.

How the enhanced ISSU works

The regular upgrade process in a high availability setup consist of the following sequential steps:

  1. Upgrade the secondary node. This step includes software upgrade of the secondary node and restart of the node.

  2. Force Failover. Running the force failover makes the upgraded secondary node to primary, and the primary node to secondary.

  3. Upgrade the new secondary node. This step includes software upgrade of the new secondary node and restart of the node.

During the time frame between step 1 and step 3, both nodes run different software builds. These two builds can have the same or different internal high availability versions.

If both the builds have different high availability version numbers, connection failover (even if it is enabled) for existing data connections not supported. In other words, all existing data connections are lost, which leads to downtime.

The ISSU upgrade process in a high availability setup consists of the following steps:

  1. Upgrade the secondary node. This step includes software upgrade of the secondary node and restart of the node.

  2. ISSU migration operation. The step includes the force failover operation and takes care of the existing connections. After you perform the migration operation, the new primary node always receives traffic (request and response) related to the existing connections but steers them to the old primary node through the configured SYNC VLAN in GRE tunnel. The old primary node processes the data traffic and then sends them directly to the destination. The ISSU migration operation is completed when all the existing connections are closed.

  3. Upgrade the new secondary node. This step includes software upgrade of the new secondary node and restart of the node.

Before you begin

Before you start performing the ISSU process in a high availability setup, go through the following pre-requisites and limitations:

  • Make sure the SYNC VLAN is configured on both the nodes of the high availability setup. For more information, see Restricting high availability synchronization traffic to a VLAN.
  • ISSU is not supported in Microsoft Azure cloud because Microsoft Azure does not support GRE tunneling.
  • High availability config propagation and synchronization do not work during ISSU.
  • ISSU is not supported for IPv6 high availability setup.
  • ISSU is not supported for following sessions:

    • Jumbo frames
    • IPv6 sessions
    • Large scale NAT (LSN)

Configuration steps

ISSU includes a migration feature, which replaces the force failover operation in the regular upgrade process of a high availability setup. The migration functionality takes care of honoring the existing connections and includes the force failover operation.

During the ISSU process of a high availability setup, you run the migration operation just after you upgraded the secondary node. You can perform the migration operation from either of the two nodes.

CLI Procedure

To perform the high availability migration operation by using the CLI:

At the command prompt type:

start ns migration

GUI Procedure

To perform the high availability migration operation by using the GUI:

Navigate to System, click System Information tab, click Migration tab, and then click Start Migration.

Rollback of the ISSU process

High availability (HA) setups now support rollback of the In Service Software Upgrade (ISSU) process. The ISSU rollback feature is helpful if you observe that the HA setup during the ISSU migration operation is not stable, or is not performing at an optimum level as expected.

The ISSU rollback is applicable when the ISSU migration operation is in progress. The ISSU rollback does not work if ISSU migration operation is already completed. In other words, you must run the ISSU rollback operation when ISSU migration operation is in progress.

The ISSU rollback functions differently based on the state of the ISSU migration operation when the ISSU rollback operation is triggered:

  • Force failover has not yet happened during ISSU migration operation. The ISSU rollback stops the ISSU migration operation, and removes any internal data related to the ISSU migration stored in both the nodes. The current primary node remains as primary node and continues to process data traffic related to existing and new connections.

  • Force failover has happened during ISSU migration operation. If the high availability failover has happened during the ISSU migration operation, then the new primary node (say it is N1) processes traffic related to the new connections. The old primary node (new secondary node, say it is N2) processes traffic related to the old connections (existing connections before the ISSU migration operation).

    The ISSU rollback stops the ISSU migration operation and triggers a force failover. The new primary node (N2) now starts processing traffic related to the new connections. The new primary node (N2) also continues to processes traffic related to old connections (existing connections established before the ISSU migration operation). In other words, the existing connections established before the ISSU migration operation are not lost.

    The new secondary node (N1) removes all the existing connections (new connections created during the ISSU migration operation) and does not process any traffic. In other words, any existing connections that were established after the force failover of ISSU migration operation are lost forever.

Configuration steps

You can use Citrix ADC CLI or GUI to perform the ISSU rollback operation.

CLI Procedure

To perform the ISSU rollback operation by using the CLI:

At the command prompt type:

stop ns migration

GUI Procedure

To perform the ISSU rollback operation by using the GUI:

Navigate to System, click System Information tab, click Migration tab, and then click Stop Migration.

SNMP traps for In Service Software Upgrade process

The In Service Software Upgrade (ISSU) process for a high availability setup supports the following SNMP trap messages at the start and end of the ISSU migration operation.

SNMP Trap Description
migrationStarted This SNMP trap is generated and sent to the configured SNMP trap listeners when the ISSU migration operation starts.
migrationComplete This SNMP trap is generated and sent to the configured SNMP trap listeners when the ISSU migration operation completes.

The primary node (before the start of the ISSU process) always generates these two SNMP traps and sends them to the configured SNMP trap listeners.

There are no SNMP alarms associated with the ISSU SNMP traps. In other words, these traps are generated irrespective of the any SNMP alarm. You only have to configure the trap SNMP listeners.

For more information on configuring SNMP trap listeners, see SNMP traps on Citrix ADC.

In Service Software Upgrade support for high availability for performing zero downtime upgrade