Configure disaster recovery for high availability

Disaster is a sudden disruption of business functions caused by natural calamities or human caused events. Disasters affect data center operations, after which resources and the data lost at the disaster site must be fully rebuilt and restored. The loss of data or downtime in the data center is critical and collapses the business continuity.

The Citrix ADM disaster recovery (DR) feature provides full system backup and recovery capabilities for Citrix ADM deployed in high availability mode. At the time of recovery, certificates, configuration files, and a complete backup of the database is available in the recovery site.

The following table describes the terms used while configuring disaster recovery in Citrix ADM.

Terms Description
Primary site (Data center A) The primary site has Citrix ADM nodes deployed in high availability mode.
Recovery site (Data center B) The recovery site has a disaster recovery node deployed in standalone mode. This node is in read-only mode and is not operational until the primary site is down.
Disaster recovery node The recovery node is a standalone node deployed in the recovery site. This node is made operational (to the new primary) in case a disaster hits the primary site and it is non-functional.

Note: The primary site and DR site communicate with each other through ports 5454 and 22, and these ports are enabled by default. For more information on port and protocol details, see Ports.

Disaster recovery workflow

The following image shows the disaster recovery workflow, the initial setup before disaster, and the workflow after the disaster.

Initial setup before disaster

Localized image

The image shows the disaster recovery setup before disaster.

The primary site has Citrix ADM nodes deployed in the high availability mode. To learn more, see High availability deployment

The recovery site has a standalone Citrix ADM disaster recovery node deployed remotely. The disaster recovery node is in read-only mode and receives data from the primary node to create data backup. Citrix ADC instances in the recovery site are also discovered but, they do not have any traffic flowing through them. During the backup process, all data, files, and configurations are replicated on the disaster recovery node from the primary node.

Prerequisites

Before you set up the disaster recovery node, note the following the prerequisites:

  • To enable disaster recovery settings, the primary site must have Citrix ADM nodes configured in high availability mode.

  • The standalone deployment of Citrix ADM in the primary site does not support the disaster recovery feature.

  • The Citrix ADM HA pair (in primary site) and the standalone node (in DR site) must have same software version, build, and configurations.

Citrix recommends that you set CPU priority (in virtual machine properties) at the highest level to improve scheduling behavior and network latency.

The following table lists the minimum requirements to configure the Disaster Recovery node:

Component Requirement
RAM 32 GB
Virtual CPU 8 CPUs
Storage Space Citrix recommends using solid-state drive (SSD) technology for Citrix ADM deployments. The default value is 120 GB. Actual storage requirement depends on Citrix ADM sizing estimation. If your Citrix ADM storage requirement exceeds 120 GB, you have to attach an extra disk. Note You can add only one more disk. Citrix recommends you to estimate storage and attach more disk at the time of initial deployment. For more information, see How to Attach an Additional Disk to Citrix ADM.
Virtual network interfaces 1
Throughput 1 Gbps or 100 Mbps
Hypervisor Versions
Citrix Hypervisor 6.2 and 6.5
VMware ESXi 5.5 and 6.0
Microsoft Hyper-V 2012 R2
Linux KVM Ubuntu and Fedora

First time disaster recovery setup

  • Deploy Citrix ADM in high availability mode

  • Deploy and register the Citrix ADM disaster recovery node

  • Enable and disable disaster recovery settings from the user interface

Deploy Citrix ADM in high availability mode

To set up disaster recovery settings, ensure that Citrix ADM is deployed in high availability mode. For information on deploying the Citrix ADM in high availability, see High availability deployment

Note

  • Citrix ADM deployed in high availability mode must be upgraded to Citrix ADM release version 13.0.

  • Floating IP address is mandatory to register disaster recovery node with the primary node.

Deploy and register the Citrix ADM disaster recovery node using DR console

To register the Citrix ADM disaster recovery node:

  1. Download the .xva image file from the Citrix download site and import it into your hypervisor.

  2. From the Console tab, configure Citrix ADM with the initial network configurations.

    Note

    The disaster recovery node can be on a different subnet.

    localized image

  3. After the initial network configuration is complete, the system prompts for login. Log on using the following credentials – nsrecover/nsroot.

    Important

    Do not change the DR node credentials (nsrecover/nsroot) during registration. You can change the DR node credentials after you register DR node successfully.

  4. To deploy the disaster recovery node, type /mps/deployment_type.py and press enter. The Citrix ADM deployment configuration menu is displayed.

    localized image

  5. Select 2 to register disaster recovery node.

    localized image

  6. The console prompts for floating IP address of the high availability node and password.

  7. Enter the floating IP address and password to register the disaster recovery node to the primary node.

    localized image

    The disaster recovery node is now registered successfully.

    Localized image

    Note

    The disaster recovery node does not have a GUI.

  8. If you want to change the DR node password, run the following script:

    ./mps/change_freebsd_password.sh <username> <password>
    

Deploy the disaster recovery node using Citrix ADM GUI

After the disaster recovery node is registered successfully using DR console, deploy the DR node from the Citrix ADM GUI. This step enables the disaster recovery settings from the Citrix ADM primary site.

  1. Navigate to System > System Administration > Disaster Recovery Settings.

  2. On the Disaster Recovery page, select Deploy DR Node.

    Deploy DR node in the Citrix ADM GUI

  3. A confirmation dialogue box is displayed. Click Yes to continue.

    Note

    The time taken for system backup depends on the data size and the WAN link speed.

After you deploy the DR node successfully in the Citrix ADM GUI, you can monitor database state, memory, CPU, and disk usage of the DR node.

Monitor the DR node in the Citrix ADM GUI

To disable the disaster recovery settings, select Remove DR Node. A confirmation dialogue box is displayed. Click Yes to continue.

Remove DR node in the Citrix ADM GUI

To enable the DR node again, reconfigure the DR node for your high availability pair:

  1. Log on to the DR node using a hypervisor or an SSH console.

  2. Configure the DR node, by following the procedure available at Deploy and register the Citrix ADM disaster recovery node using DR console.

  3. Deploy the disaster recovery node using Citrix ADM GUI.

For more information, see the FAQs.

Important

  • It is the responsibility of the administrator to detect that a disaster has occurred on the primary site.

  • The disaster recovery workflow is manually initiated by the administrator after the primary site goes down.

  • An administrator must manually initiate the process by executing a recovery script on the disaster recovery node at the recovery site.

  • If you upgrade the HA pair in primary site, you must also manually upgrade the standalone node in the DR site.

Workflow after the disaster

When the primary site goes down after a disaster, the disaster recovery workflow must be initiated as follows:

  1. The administrator identifies that a disaster has struck the primary site and it is not operational.

  2. The administrator initiates the recovery process.

  3. The administrator must manually execute one of the following recovery scripts on the disaster recovery node based on your requirement(at the recovery site):

    • Cofigure SNMP, Syslog, and Analytics on the DR node:

       /mps/scripts/pgsql/pgsql\_restore\_remote\_backup.sh
      
      
    • Configure the DR node as a license server also:

      
       /mps/scripts/pgsql/pgsql\_restore\_remote\_backup.sh -reconfig-ls <IP-address-of-the-primary-site>
      
      
  4. Internally, Citrix ADC instances are automatically reconfigured to send the data to the disaster recovery node that has now become the new primary site.

    The following image shows that the disaster recovery workflow after the primary site is struck with a disaster.

    localized image

    Note:

    After you initiate the script at the DR site, the DR site now becomes the new primary site. You can also access the DR user interface.

Post disaster recovery

After the disaster has occurred and the administrator initiates the recovery script, the DR site now becomes the new primary site.

If you want to revert the configurations to the original site later, see Revert configurations to the original primary site.

Important

  • If you have installed Citrix ADM 12.1.49.x or earlier releases, you get a grace period of 30 days to contact Citrix to rehost the original license on the Citrix ADM (at the DR site).

  • For 12.1.50.x or later releases, the Citrix ADM license is automatically synchronized to the DR site (Not a requirement to contact Citrix for the license).

  • Pooled license for the DR site is supported from 12.1.50.x or later releases. If you have applied pooled licenses for the instances, manually reconfigure the instances to the DR site.

Revert configurations to the original primary site

Post disaster the configured disaster recovery (DR) node becomes the new primary site and the client traffic flows through this node.

Set up after disaster

For more information, see Workflow after the disaster.

When your original primary site is free from disaster and you decide to move all operations to the primary site, reconfigure the original primary site to match the configurations from the DR node.

Before you begin, ensure both primary site and DR site are active.

To revert the changes to the original primary site from the DR site, perform the following steps:

  1. Log in to the original primary site and run the following command:

    nohup /mps/sync_adm_node.py -I <DR-site-IP-address> -R <DR-node-password> -L <primary-node-password> &
    
    

    This command configures only Syslog, SNMP, and Analytics to the primary site.

    If you want to configure the primary site as a pooled license server for ADC instances, run the following command:

    
    nohup /mps/sync_adm_node.py -I <DR-site-IP-address> -R <DR-node-password> -L <primary-node-password>  -O <pooled-license> &
    
    
  2. Reconfigure the DR site. See, Deploy disaster recovery setup.

Revert to original primary site

After you successfully revert the configurations from the DR site to the original primary site, the client traffic flows through the Citrix ADM primary node.