Citrix SD-WAN Standard Edition Virtual Appliance (VPX) high availability Support for AWS

This procedure below describes how to deploy SD-WAN virtual (VPX) appliances in high-availability mode on AWS cloud.

Points to consider when deploying SD-WAN VPX high availability appliances in AWS Cloud.

  1. AWS does not support GARP (Generic Attribute Registration Protocol), VLAN or L2 related functionality, such as promiscuous mode and bridging. This is because two VMs belonging to different customers can be scheduled on the same host sharing NICs.
  2. L2 requires the switch appliance to be configured and these are not exposed to AWS users.
  3. SD-WAN appliance high availability model depends on GARP. When failover occurs, the new primary appliance sends GARPS out for VIP addresses.
  4. AWS takes a new approach for high availability failover. A new concept of ENI (Elastic Network Interface) is introduced. ENI is an entity which stands for Network Interface which has attributes like the IP address, MAC address, Security Group, and Port Rules.
  5. You can move ENIs from active or inactive Instance to another active or inactive Instance.
  6. The Instance needs to be capable to handling hot plug of interfaces.
  7. Each Instance type has limitations on number of ENIs associated and number of IPs per ENI.
  8. AWS design for high availability failover involves Instances communicating with external server to call Query API AWS servers.
  9. The AWS servers are traditional HTTP servers. A request is sent from an instance to Query API server to get or post information regarding an Instance/subnet/VPC or any other attribute on the AWS.
  10. For the cloud platform setup, the shared base MAC address configuration is ignored and has no significance.

Deploy SD-WAN standard edition VPX in high availability mode using cloud template

SD-WAN high availability solution template is published in AWS marketplace, you can subscribe and use the CloudFormation template to deploy the HA setup.

Prerequisites

Before launching the CloudFormation template, you need to have VPC, subnets, route tables created for Management, LAN, and WAN network. To create and define the subnets and route tables (if not created), refer Installing SD-WAN VPX Standard Edition AMI on AWS topic.

To deploy SD-WAN standard edition VPX in high availability mode using cloud template:

  1. Go to AWS marketplace and click Pricing tab. Select the Region from the drop-down list and specify the Fulfillment Option as High Availability Mode deployment. Click Continue to Subscribe.

    localized image

  2. Click Continue to Configuration.

    localized image

  3. Specify Fulfillment Option as CloudFormation Template and High Availability Mode Deployment from the drop-down list. Select Region and click Continue to Launch.

    localized image

  4. Choose action as Launch CloudFormation in the launch software window and click Launch.

    localized image

  5. In the Create Stack window, the predefined S3 template URL appears during the CloudFormation. Click Next.

    localized image

  6. Specify a Stack name in the Specify Details section.

    localized image

  7. Configure Virtual Private Network Configuration. Fill in the following parameter details:
    • VPC ID: Provide the virtual private cloud ID.
    • Remote SSH CIDR IP: Provide the IP address range that can SSH to the EC2 instance (port 22).
    • Remote HTTP CIDR IP: Provide the IP address range that can HTTP to the EC2 instance (port80).
    • Remote HTTPS CIDR IP: Provide the IP address range that can HTTPS to the EC2 instance (port 443).
    • Key Pair: Provide a name of an existing EC2 KeyPair to enable SSH access to the instances.

    localized image

  8. Configure Network Interfaces which should be attached to the instances created. Note that the Primary IPs are for primary instance of high availability pair and Secondary IPs are configured for secondary instance of the high availability pair.

    localized image

  9. Configure other Parameters such as Instant Type and Tenancy Type and click Next.

    localized image

    NOTE

    If any validations fail, AWS notifies you and would not let you proceed until the errors are resolved.

  10. Set Tags. These tags are AWS-specific options which are user configurable.

    localized image

  11. Configuring IAM role is not recommended. This is already created by the customized IAM role, which is done through the Cloud Formation template.

    localized image

  12. After clicking next, Review the template and acknowledge the custom IAM role which has been created by Cloud Formation template. Proceed with Create.

    localized image

  13. The new stack that you created appears on the Cloud Formation Stacks page. After successful template upload, Monitor the status of the template.

    localized image

  14. Monitor the events of all the resources created by the Cloud Formation template. If there is any failure, detailed descriptions of events are generated by AWS which helps in debugging the issue. The Events appear as follows:

    localized image

  15. After successful stack creation, the status of the template appears as Create_Complete.

    localized image

  16. Navigate from AWS console to Services > EC2 > Instances. You can see two instances SDWANPrimary and SDWANSecondary instances created, up and running with Elastic IPs associated with the instances.   localized image

  17. Select SDWANPrimary instance. You can notice all the resources rightly assigned to instance, Security groups, Elastic IP, IAM role, and four Network Interfaces. Failed to create any high availability functionality might not work as expected.

  18. Similarly select SDWANSecondary instance and verify the above resources.

You must need a secondary floating IP’s for LAN and WAN links for the high availability to work. Once the stack is created, assign new secondary private IP’s to LAN and WAN interfaces of the active EC2 instance. These secondary configured IP’s are used while configuring virtual IP addresses in VPX.

Perform the following procedure to attach the secondary LAN IP’s to active instance:

  1. Navigate to Services > EC2 > Instances.

    localized image

  2. Navigate to Services > EC2 > Network interfaces and select the LAN/WAN Elastic Network Interfaces (ENI) of the primary instance.

    localized image

  3. Assign new secondary IP.

    localized image

  4. Click Yes, Update.

    localized image

  5. Similarly create secondary private IP for WAN interface as well.

A public IP required on the WAN link to communicate with the external world. Follow the below steps to associate elastic IP to WAN ENI interface:

  1. Navigate to Addresses > Allocate new address.

    localized image

  2. Select the elastic IP created, and click Action > Associate address and associate the public to the secondary private WAN IP which we just created.

    localized image

  3. Verify final interfaces and IP’s are expected as below:

    • Primary Instance: localized image
    • Secondary Instance: localized image

    Now the instance provisioning is completed. Configuring SD-WAN high availability appliance is almost similar to configuring standalone appliance. Differences are listed below:

    • While creating LAN and WAN Virtual IP interfaces, specify the secondary private IP’s created. And for high availability virtual IP interface, specify a dummy IP in high availability network.

      localized image

    • Enable high availability and specify the high availability interface IP’s of active and secondary instance.

      localized image You can verify the high availability status.

      localized image

How to configure high availability Fail-Over for any SD-WAN instance running on AWS

Set up high availability peers with one high availability peer with three or more ENIs, and 1 high availability peer with equal number of ENIs. In both Peers, the first ENI is dedicated to Management. One high availability peer owns all Traffic ENIs. During a Failover, the traffic ENIs move from the failing instance to the new Primary instance.

For example, it can take up to or more than 20 secs to move two traffic ENIs. AWS do not have SLAs on API response and you cannot have one for high availability fail-over time.

Note

The AWS design has a limitation of instances dependent on the AWS servers to respond for attach and detach. The fail-over time is unpredictable.

Configuration steps

  1. Acquire information about your high availability Peer Instance about information on the number of ENIs associated and details of ENIs associated using REST API.
  2. Detect the condition of the failing instance.
  3. Call Detach of ENIs from failing instance using REST APIs.
  4. Ensure all ENIs associated are detached.
  5. Attach ENIs to the current Primary instance.
  6. Ensure All ENIs are attached.
  7. Trigger upper layers to detect that new ENIs are in place.

localized image

localized image

In AWS VPC, for an active SD-WAN instance, another high available SD-WAN instance running in the same VPC is released.

  1. The links configured are the same between active and stand-by SD-WAN appliances.
  2. For AWS, you can create a subnet and a dedicated link for RACP protocol to communicate between the SD-WAN appliances.
  3. In the SD-WAN GUI, configure the following:
    • Create an interface group. Name it as high availability-LINK. Add the interface used for high availability.
    • Create a Virtual IP address for Interface group.
    • In High Availability Node, Enable high availability and add control Virtual IPs which RACP protocol uses for communication. Ensure that the IP addresses are same as the configured IP addressed while creating network interfaces in AWS.
    • Perform Change Management and download the active configuration for the stand-by SD-WAN appliance.
    • After applying configuration through local change management on the stand-by SD-WAN appliance, you will see heartbeats exchanged between active and stand-by SD-WAN high availability appliances.
    • When failover occurs, you see SD-WAN appliance transitioning from stand-by to active modes and/or conversely without any configuration loss.

Note

  1. AWS supports high availability mode with features such as Elastic Load balancing and auto-scaling where the challenge is to sync configuration within the SD-WAN appliances. In this deployment, you apply the existing RACP protocol for efficient high availability.

  2. Both MCN and branch site appliances can be made available in the cloud environment.