SD-WAN Standard Edition Virtual Appliance (VPX) HA Support for AWS
Aug 09, 2017
This procedure below describes how to deploy NetScaler SD-WAN virtual (VPX) appliances in high-availability mode on Amazon AWS cloud.
Points to consider when deploying SD-WAN VPX HA appliances in AWS Cloud:
AWS does not support GARP (Generic Attribute Registration Protocol), VLAN or L2 related functionality, such as promiscuous mode and bridging. This is because two VMs belonging to different customers can be scheduled on the same host sharing NICs.
L2 requires the switch appliance to be configured and these are not exposed to AWS users.
SD-WAN appliance HA model depends on GARP. When fail-over occurs, the new primary appliance sends GARPS out for VIP addresses.
AWS takes a new approach for HA failover. A new concept of ENI (Elastic Network Interface) is introduced. ENI is an entity which stands for Network Interface which has attributes like the IP address, MAC address, Security Group, and Port Rules.
You can move ENIs from active or inactive Instance to another active/inactive Instance.
The Instance needs to be capable to handling hot plug of interfaces.
Each Instance type has limitations on number of ENIs associated and number of IPs per ENI.
When an ENI moves all the attributes of the ENI MAC address, the IP address and Firewall Rules move with ENI.
AWS design for HA fail-over involves Instances communicating with external server to call Query API AWS servers.
The AWS servers are traditional HTTP servers. A request is sent from an instance to Query API server to get or post information regarding an Instance/subnet/VPC or any other attribute on the AWS.
For the cloud platform setup, the shared base MAC address configuration is ignored and has no significance.
How to create HA solution template in AWS
To create HA solution template in AWS:
1. Go to www.aws.amazon.com and log on with AWS credentials. After successful login, navigate to the Services > Management Tools > CloudFormation.
2. On the CloudFormation Stacks page, select the Region in which you want to deploy the NetScaler SD-WAN VPX instance, and then in the Create a stack section, choose Create new stack to create a new AWS CloudFormation stack.
3. In the Select Template section, choose a template by:
- Uploading a template using Upload a template to Amazon S3 option. (or)
- Specifying Amazon s3 template URL using Specify an Amazon S3 template URL option.
In both the cases, you will provide the Template or URL.
4. In the Specify Details section, specify a Stack name.
5. Configure Virtual Private Network Configuration. Fill in the below details as suggested. You can also find the tool tips besides each field.
6. Configure Network Interfaces which should be attached to the instances created. Please note that the Primary IP’s are for primary instance of HA pair and Secondary IP’s are configured for secondary instance of the HA pair.
7. Configure Other Parameters such as Instant Type and Tenancy Type and click on Next.
If any validations fail, AWS will notify you and would not let you proceed until the errors are resolved.
8. Set Tags. These are AWS specific options which are user configurable.
9. Configuring IAM role is not recommended. This is already created by the customized IAM role, which is done through the cloud Formation Template.
10. After clicking next, Review the template and acknowledge the custom IAM role which has been created by CloudFormation template. Proceed with Create.
11. The new stack that you created appears on the CloudFormation Stacks page. After successful template upload, Monitor the status of the template.
12. Monitor the events of all the resources created by the CloudFormation template. In case of any failure, detailed description of events are generated by AWS which helps in debugging the issue. The Events appear as follows:
13. After successful stack creation, the status of the template should appear as Create_Complete.
14. Navigate from AWS console to Services > EC2 > Instances. You should see two instances SDWANPrimary and SDWANSecondary instances created, up and running with Elastic IP’s associated with the instances.
15. Select SDWANPrimary instance. You should notice all the resources rightly assigned to instance, Security groups, Elastic IP, IAM role and four Network Interfaces. Failed to create any HA functionality may not work as expected.
16. Similarly select SDWANSecondary instance and verify the above resources.
How to Configure HA Fail-Over for any SD-WAN Instance Running on AWS
Set up HA peers with one HA peer with three or more ENIs, and 1 HA peer with equal number of ENI’s. In both Peers, the first ENI is dedicated to Management. One HA peer owns all Traffic ENI’s. During a Failover, the traffic ENIs move from the failing instance to the new Primary instance.
For example; it can take upto or more than 20 secs to move two traffic ENIs. AWS do not have SLAs on API response and you cannot have one for HA failover time.
The AWS design has a limitation of instances dependent on the AWS servers to respond for attach and detach. The failover time is unpredictable.
Acquire information about your HA Peer Instance about information on the number of ENIs associated and details of ENIs associated using REST API.
Detect the condition of the failing instance.
Call Detach of ENIs from failing instance using REST APIs.
Ensure all ENIs associated are detached.
Attach ENIs to the current Primary instance.
Ensure All ENIs are attached.
Trigger upper layers to detect that new ENIs are in place.
How to Configure SD-WAN VPX-SE in a Single AWS Virtual Private Cloud (VPC) Subnet or Between Regions with Public WAN Link IP Address
In AWS VPC, for an active SD-WAN instance, another high available SD-WAN instance running in the same VPC is released.
The links configured are the same between active and stand-by SD-WAN appliances.
For AWS, you can create a new subnet and a dedicated link for RACP protocol to communicate between the SD-WAN appliances.
In the SD-WAN GUI, configure the following:
a. Create an interface group. Name it as HA-LINK. Add the interface used for HA.
b. Create a Virtual IP address for Interface group.
c. In High Availability Node, Enable HA and add control Virtual IP’s which RACP protocol uses for communication. Ensure that the IP addresses are same as the configured IP addressed while creating network interfaces in AWS.
d. Perform Change Management and download the active configuration for the stand-by SD-WAN appliance.
e. After applying configuration through local change management on the stand-by SD-WAN appliance, you will see heartbeats exchanged between active and stand-by SD-WAN HA appliances.
f. When failover occurs, you will see SD-WAN appliance transitioning from stand-by to active modes and/or vice-versa without any configuration loss.
AWS supports HA mode with features such as Elastic Load balancing and auto-scaling where the challenge is to sync configuration within the SD-WAN appliances. In this deployment you leverage the already existing RACP protocol for efficient HA.
Both MCN and branch site appliances can be made available in the cloud environment.