Citrix SD-WAN Standard Edition Virtual Appliance (VPX) high availability Support for AWS
This procedure below describes how to deploy SD-WAN virtual (VPX) appliances in high-availability mode on Amazon AWS cloud.
Points to consider when deploying SD-WAN VPX high availability appliances in AWS Cloud.
- AWS does not support GARP (Generic Attribute Registration Protocol), VLAN or L2 related functionality, such as promiscuous mode and bridging. This is because two VMs belonging to different customers can be scheduled on the same host sharing NICs.
- L2 requires the switch appliance to be configured and these are not exposed to AWS users.
- SD-WAN appliance high availability model depends on GARP. When fail-over occurs, the new primary appliance sends GARPS out for VIP addresses.
- AWS takes a new approach for high availability failover. A new concept of ENI (Elastic Network Interface) is introduced. ENI is an entity which stands for Network Interface which has attributes like the IP address, MAC address, Security Group, and Port Rules.
- You can move ENIs from active or inactive Instance to another active/inactive Instance.
- The Instance needs to be capable to handling hot plug of interfaces.
- Each Instance type has limitations on number of ENIs associated and number of IPs per ENI.
- When an ENI moves all the attributes of the ENI MAC address, the IP address and Firewall Rules move with ENI.
- AWS design for high availability fail-over involves Instances communicating with external server to call Query API AWS servers.
- The AWS servers are traditional HTTP servers. A request is sent from an instance to Query API server to get or post information regarding an Instance/subnet/VPC or any other attribute on the AWS.
- For the cloud platform setup, the shared base MAC address configuration is ignored and has no significance.
How to create high availability solution template in AWS
To create high availability solution template in AWS:
- Go to www.aws.amazon.com and log on with AWS credentials. After successful login, navigate to the Services > Management Tools > Cloud Formation.
- On the Cloud Formation Stacks page, select the Region in which you want to deploy the NetScaler SD-WAN VPX instance, and then in the Create a stack section, choose Create new stack to create a AWS Cloud Formation stack.
- In the Select Template section, choose a template by:
- Uploading a template using Upload a template to Amazon S3 option. (or)
- Specifying Amazon s3 template URL using Specify an Amazon S3 template URL option. In both the cases, you provide the Template or URL.
- In the Specify Details section, specify a Stack name.
- Configure Virtual Private Network Configuration. Fill in the below details as suggested. You can also find the tool tips besides each field.
- Configure Network Interfaces which should be attached to the instances created. Please note that the Primary IPs are for primary instance of high availability pair and Secondary IPs are configured for secondary instance of the high availability pair.
- Configure Other Parameters such as Instant Type and Tenancy Type and click on Next.
If any validations fail, AWS notifies you and would not let you proceed until the errors are resolved.
- Set Tags. These are AWS specific options which are user configurable.
- Configuring IAM role is not recommended. This is already created by the customized IAM role, which is done through the cloud Formation Template.
- After clicking next, Review the template and acknowledge the custom IAM role which has been created by Cloud Formation template. Proceed with Create.
- The new stack that you created appears on the Cloud Formation Stacks page. After successful template upload, Monitor the status of the template.
- Monitor the events of all the resources created by the Cloud Formation template. If there is any failure, detailed descriptions of events are generated by AWS which helps in debugging the issue. The Events appear as follows:
- After successful stack creation, the status of the template should appear as Create_Complete.
- Navigate from AWS console to Services > EC2 > Instances. You should see two instances SDWANPrimary and SDWANSecondary instances created, up and running with Elastic IPs associated with the instances.
- Select SDWANPrimary instance. You should notice all the resources rightly assigned to instance, Security groups, Elastic IP, IAM role, and four Network Interfaces. Failed to create any high availability functionality may not work as expected.
- Similarly select SDWANSecondary instance and verify the above resources.
How to configure high availability Fail-Over for any SD-WAN instance running on AWS
Set up high availability peers with one high availability peer with three or more ENIs, and 1 high availability peer with equal number of ENIs. In both Peers, the first ENI is dedicated to Management. One high availability peer owns all Traffic ENIs. During a Failover, the traffic ENIs move from the failing instance to the new Primary instance.
For example; it can take up to or more than 20 secs to move two traffic ENIs. AWS do not have SLAs on API response and you cannot have one for high availability failover time.
The AWS design has a limitation of instances dependent on the AWS servers to respond for attach and detach. The failover time is unpredictable.
- Acquire information about your high availability Peer Instance about information on the number of ENIs associated and details of ENIs associated using REST API.
- Detect the condition of the failing instance.
- Call Detach of ENIs from failing instance using REST APIs.
- Ensure all ENIs associated are detached.
- Attach ENIs to the current Primary instance.
- Ensure All ENIs are attached.
- Trigger upper layers to detect that new ENIs are in place.
How to configure SD-WAN VPX-SE in a single AWS Virtual Private Cloud (VPC) Subnet or between regions with Public WAN link IP address
In AWS VPC, for an active SD-WAN instance, another high available SD-WAN instance running in the same VPC is released.
- The links configured are the same between active and stand-by SD-WAN appliances.
- For AWS, you can create a subnet and a dedicated link for RACP protocol to communicate between the SD-WAN appliances.
- In the SD-WAN GUI, configure the following:
- Create an interface group. Name it as high availability-LINK. Add the interface used for high availability.
- Create a Virtual IP address for Interface group.
- In High Availability Node, Enable high availability and add control Virtual IPs which RACP protocol uses for communication. Ensure that the IP addresses are same as the configured IP addressed while creating network interfaces in AWS.
- Perform Change Management and download the active configuration for the stand-by SD-WAN appliance.
- After applying configuration through local change management on the stand-by SD-WAN appliance, you will see heartbeats exchanged between active and stand-by SD-WAN high availability appliances.
- When failover occurs, you see SD-WAN appliance transitioning from stand-by to active modes and/or conversely without any configuration loss.
AWS supports high availability mode with features such as Elastic Load balancing and auto-scaling where the challenge is to sync configuration within the SD-WAN appliances. In this deployment you apply the existing RACP protocol for efficient high availability.
Both MCN and branch site appliances can be made available in the cloud environment.