Load balancing with NetScaler

This article contains the information needed to use NetScaler to load balance two or more StoreFront servers.  

Configure a StoreFront server group and NetScaler load balancing 

Create a server certificate for the NetScaler load balancer and StoreFront servers

Create a load balancing vServer for subscription synchronization between server groups

Configure the StoreFront server group for load balancing

Citrix Service Monitor 

NetScaler Gateway and load balancing vServers on the same NetScaler appliance

Loopback options when load balancing a StoreFront server group using NetScaler

Plan your load balanced StoreFront deployment

This article provides guidance on how to deploy a StoreFront server group containing two or more StoreFront servers in all active load balanced configuration.  The article provides details of how to configure a NetScaler appliance to load balance incoming requests from Citrix Receiver/Citrix Receiver for Web between all of the StoreFront nodes in the server group and how to configure the new Storefront Monitor for use with a NetScaler or third party load balancer. 

For load balancing configuration examples, see the sections "Scenario 1" and "Scenario 2" below.

Tested with the following environment

• Four Windows Server 2012 R2 StoreFront 3.0 nodes in a single server group.
• One NetScaler 10.5 load balancer configured for Least Connection and CookieInsert "sticky" load balancing.
• One Windows 8.1 test client with Fiddler 4.0 and Citrix Receiver for Windows 4.3 installed.

Server certificate requirements for the load balanced deployment if you intend to use HTTPS

Review the section Plan gateway and server certificate usage.

Consider the following options before purchasing a certificate from a commercial certificate authority or issuing one from your enterprise CA.

• Option 1: Use a *.example.com wildcard certificate on both the NetScaler load balancing vServer and on the StoreFront server group nodes.  This simplifies the configuration and allows you to add extra StoreFront servers in the future without the need to replace the certificate.
• Option 2: Use a certificate including Subject Alternative Names (SANs) on both the NetScaler load balancing vServer and on the StoreFront server group nodes.  Extra SANs within the certificate that match all of StoreFront server fully qualified domain names (FQDNs) are optional, but recommended, as this allows greater flexibility in the StoreFront deployment.  Include a SAN for email-based discovery discoverReceiver.example.com.

For details of email-based discovery configuration, see http://blogs.citrix.com/2013/04/01/configuring-email-based-account-discovery-for-citrix-receiver/ .

Note: When exporting the private key associated with the certificate is not feasible. Use two separate certificates: one on the NetScaler load balancing vServer and a different certificate on the StoreFront server group nodes. Both certificates must include Subject Alternative Names.

Import a certificate issued from a Windows CA onto a NetScaler appliance using OpenSSL

• WinSCP is a useful third party and free tool to move files from a Windows machine to a NetScaler file system.  Copy certificates for import to the /nsconfig/ssl/ folder within the NetScaler file system. 
• You can also use OpenSSL tools on the NetScaler to extract the certificate and key from a PKCS12/PFX file to create a two separate .CER and .KEY X.509 files in PEM format that NetScaler can use.

1. Copy the PFX file into /nsconfig/ssl/ on the NetScaler appliance or VPX.
2. Open the NetScaler command line interface (CLI).
3. Type Shell to switch to exit the NetScaler CLI and switch to the FreeBSD shell.
4. Change directory using cd /nsconfig/ssl/.
5. Run openssl pkcs12 -in <imported cert file>.pfx -nokeys -out <certfilename>.cer and enter the PFX password when prompted.
6. Run openssl pkcs12 -in <imported cert file>.pfx -nocerts -out <keyfilename>.key and enter the PFX password when prompted, and then set the private key PEM passphrase to protect the .KEY file.
7. Run ls -al to check the .CER and .KEY files have been successfully created inside /nsconfig/ssl/.
8. Type Exit to return to the NetScaler CLI.

Configure the server certificate on the NetScaler after it is imported

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > SSL > SSL Certificates and click Install.
3. On the Install Certificate window, enter the certificate and private key pair names.
   • Select the .cer certificate file on the NetScaler file system under /nsconfig/ssl/.
   • Select the .key file containing the private key from the same location.

Create DNS records for the StoreFront server group load balancer

Create a DNS A and PTR record for your chosen shared FQDN.  Clients within your network use this FQDN to access the StoreFront server group using the NetScaler load balancer.

Example - storefront.example.com resolves to the load balancing vServer virtual IP (VIP).

Scenario 1: An end to end HTPPS 443 secure connection between the client and NetScaler load balancer and also between the load balancer and two or more StoreFront 3.0 servers. 

This scenario uses a modified StoreFront monitor using port 443.

Add individual StoreFront server nodes to the NetScaler load balancer

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Load Balancing > Servers > Add and add each of the four StoreFront nodes to be load balanced.
   
   Example = 4 x 2012R2 StoreFront Nodes called 2012R2-A to –D
   
   
3. Use IP based server configuration and enter the server IP address for each StoreFront node.

Define a StoreFront monitor to check the status of all StoreFront nodes in the server group

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Load Balancing > Monitors > Add and add a new monitor called StoreFront and accept all default settings.
3. From the Type drop down menu, select StoreFront.
4. Makes sure the Secure check box is checked if using HTTPS connections between your load balancing vServer and StoreFront; otherwise leave this option disabled.
5. Specify the store name under the Special Parameters tab.
6. Check the Check Backend Services check box under the Special Parameters tab.  This option enables monitoring of services running on the StoreFront server.  StoreFront services are monitored by probing a Windows service that runs on the StoreFront server, which returns the status of all running StoreFront services.

Create an HTTPS 443 service group containing all of the StoreFront servers

1.  Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.

2.  Set the SSL port and give each node a unique server ID as they are added.

3.  On the Monitors tab, select the StoreFront monitor you created earlier.

4.   On the Certificates tab, bind the server certificate you imported earlier.

5.   Bind the CA certificate used to sign the server certificate you imported earlier and any other CAs that might be part of the PKI chain of trust.

Create a load balancing vServer for user traffic

1.    Log onto the NetScaler management GUI.

2.    Select Traffic Management > Load Balancing > Virtual Servers > Add to create a new vServer.

3.    Select the load balancing method for the vServer.  Common choices for StoreFront load balancing are round robin or least connection.

4.  Bind the Service Group you created earlier to the load balancing vServer.

5.  Bind the same server and CA certificate you previously bound to the service group, to the load balancing vServer.

6.  From within the load balancing vServer menu, select Persistence on the right hand side and set the persistence method to be CookieInsert.

7.  Name the cookie. For example, NSC_SFPersistence, as this makes it easy to identify in Fiddler traces during debugging.

8.  Set backup persistence to None.

Scenario 2: HTTPS termination - HTTPS 443 communication between the client and NetScaler load balancer and HTTP 80 connections between the load balancer and the StoreFront 3.0 servers behind it.  

This scenario uses the default StoreFront monitor using port 8000.

Add individual StoreFront server servers to the NetScaler load balancer

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Load Balancing > Servers > Add and add each of the four StoreFront servers to be load balanced.
   
   Example = 4 x 2012R2 Storefront servers called 2012R2-A to -D.
   
   
3. Use IP based Server configuration and enter the server IP address for each Storefront server.

Define an HTTP 8000 StoreFront monitor to check the status of all StoreFront servers in the server group

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Monitors > Add and add a new monitor called StoreFront.
3. Add a name for the new monitor and accept all default settings.
4. Select Type from the drop down menu as StoreFront.
5. Specify the store name under the Special Parameters tab.
6. Enter 8000 into destination port, as this matches the default monitor instance that is created on each StoreFront server.
7. Tick the Check Backend Services check box under the Special Parameters tab.  This option enables monitoring of services running on the StoreFront server.  StoreFront services are monitored by probing to a Windows service that runs on the StoreFront server, which returns the status of all running StoreFront services.

Create an HTTP 80 service group containing all of the StoreFront servers

1. Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.
2. Set the HTTP port to 80 and give each server a unique server ID as you add them.
3. On the Monitors tab, select the StoreFront monitor you created earlier.

Create an HTTPS terminating load balancing vServer for user traffic

1. Select Traffic Management > Load Balancing > Virtual Servers > Add to create a new vServer.
2. Select the load balancing method vServer will use.  Common choices for StoreFront load balancing are round robin or least connection.
3. Bind the Service Group you created earlier to the load balancing vServer.
4. Bind the same server and CA certificate you previously bound to the service group, to the load balancing vServer.
   
   Note: If the client is not allowed to store the HTTP cookie, the subsequent requests do not have the HTTP cookie and Persistence is not used.
   
   
5. From within the load balancing vServer menu, select Persistence and set the persistence method to be CookieInsert.
6. Name the cookie. For example, NSC_SFPersistence, as this makes it easy to identify in Fiddler traces during debugging.
7. Set backup persistence to None.

Considerations before creating a load balancing vServer include the following:

• Option 1: Create a single vServer: To load balance only user traffic.  This is all that is needed if performing only ICA launches of published apps and desktops. (Mandatory and usually all that is required.)
• Option 2: Create a pair of vServers: One to load balance user traffic for performing ICA launches of published apps and desktops and another for load balancing subscription data synchronization operations. (Necessary only when propagating subscription data between two or more load balanced StoreFront server groups in a large multisite deployment.)

If a multisite deployment consists of two or more StoreFront server groups located in separate geographic locations, you can replicate subscription data between them using a pull strategy on a repeating schedule. StoreFront subscription replication uses TCP port 808, so using an existing load balancing vServer on HTTP port 80 or HTTPS 443 fails. To provide high availability for this service, create a second vServer on each NetScaler in your deployment to load balance TCP port 808 for each of the StoreFront server groups. When configuring the replication schedule, specify a server group address that matches the subscription synching vServer virtual IP address. Ensure the server group address is the FQDN of the load balancer for the server group at that location. 

Configure a service group for subscription synchronization

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Service Groups > Add and add a new service group.
3. Change the protocol to TCP.
4. Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.
5. On the Monitors tab, select the TCP monitor.

Create a load balancing vServer for subscription synchronization between server groups

1. Log onto the NetScaler management GUI.
2. Select Traffic Management > Service Groups > Add and add a new service group.
3. Set the load balancing method to round robin.
4. Change the protocol to TCP.
5. Enter 808 and NOT 443 as the port number.

For StoreFront server A at Location A to request and pull subscription data from server B at a different location, server A must be a member of the CitrixSubscriptionsSyncUsers local security group on server B.  The CitrixSubscriptionsSyncUsers local group contains an access control list of all remote StoreFront servers authorized to pull subscription data from a particular server.  For bidirectional subscription synchronization, server B must also be a member of the CitrixSubscriptionsSyncUsers security group on server A to pull subscription data from it.

1.   Import the same certificate and private key that was deployed on the NetScaler load balancing vServer to every StoreFront node in the server group.

2.   Create an HTTPS binding in IIS on every StoreFront node, and then bind the certificate you imported earlier to it.

3.   Install StoreFront on every node in the server group.

4.   During installation of StoreFront, set the host base URL on the primary node to be the shared FQDN used by all members of the server group.  You must use a certificate containing the load balanced FQDN as a Common Name (CN) or Subject Alternative Name (SAN).

     See the Create a server certificate for the NetScaler load balancer and StoreFront servers.

5.   When you complete the initial StoreFront configuration, join each of the nodes, one after the other, to the server group using the primary node. 

6.   Select Server Group > Add Server > Copy the Authorization Code to the joining Server.

7.   Propagate the configuration from the primary node to all other server group nodes in the group.

8.   Test the load balanced server group using a client that can contact and resolve the shared FQDN of the load balancer.

To enable external monitoring of the run-state of the Windows services on which StoreFront relies for correct operation, use the Citrix Service Monitor Windows service.  This service has no other service dependencies and can monitor and report the failure of other critical StoreFront services.  The monitor enables the relative health of a StoreFront server deployment to be determined externally by other Citrix components, such as NetScaler.  Third party software can consume the StoreFront monitor XML response to monitor the health of essential StoreFront services.

After StoreFront is deployed, a default monitor that uses HTTP and port 8000 is created. 

Note: Only a single instance of a monitor can exist within a Storefront deployment.

To make any changes to the existing default monitor, such as changing the protocol and port to HTTPS 443, use the three PowerShell cmdlets to view or reconfigure the StoreFront monitor service URL. 

Remove the default Service Monitor and replace it with one that uses HTTPS and port 443

1. Open the PowerShell Integrated Scripting Environment (ISE) on the primary StoreFront server and run the following commands to change the default monitor to HTTPS 443.

$ServiceUrl = "https://localhost:443/StorefrontMonitor"

Set-STFServiceMonitor -ServiceUrl $ServiceUrl

Get-STFServiceMonitor

2. Once completed, propagate the changes to all other servers in the StoreFront server group.

3. To perform a quick test on the new monitor, enter the following URL into the browser on the StoreFront server or any other machine with network access to the StoreFront server.  The browser should return an XML summary of the status of every Storefront service.

https://<loadbalancingFQDN>:443/StoreFrontMonitor/GetSFServicesStatus

If you have configured the NetScaler Gateway vServer and load balancing vServer on the same NetScaler appliance, internal domain users might experience issues when trying to access the StoreFront load balanced host base URL directly rather than passing through the NetScaler Gateway vServer. 

In this scenario StoreFront assumes that the end user has already authenticated at the NetScaler Gateway because StoreFront correlates the source IP address of the incoming user with the NetScaler Gateway Subnet IP Address (SNIP).  This triggers StoreFront attempt to use the AGBasic protocol to perform NetScaler Gateway silent authentication, rather than actually prompting the user to log on with their domain credentials.  To avoid this issue, omit a SNIP address as shown below so that username and password authentication is used instead of AGBasic.

Configure a Netscaler Gateway on the Storefront Server Group 

In previous versions of Storefront such as 2.6 or older, Citrix recommended that you manually modify the hosts file on each StoreFront server to map the fully qualified domain name (FQDN) of the load balancer to the loopback address or the IP address of the specific StoreFront server.  This ensures that Receiver for Web always communicates with the StoreFront services on the same server in a load balanced deployment.  This is necessary because an HTTP session is created during the explicit login process between Receiver for Web and the authentication service and Receiver for Web communicates with Storefront services using the base FQDN.  If the base FQDN were to resolve to the load balancer, the load balancer could potentially send the traffic to a different StoreFront server in the group, leading to authentication failure.  This does not bypass the load balancer except when Receiver for Web attempts to contact the Store service residing on the same server as itself. 

You can set loopback options using PowerShell.  Enabling loopback negates the need to create host file entries on every StoreFront server in the server group. 

Example Receiver for Web web.config file:

<communication attempts="2" timeout="00:01:00" loopback="On" loopbackPortUsingHttp="80">

Example PowerShell command:

& "c:\program files\Citrix\receiver storefront\scripts\ImportModules.ps1"

Set-DSLoopback -SiteId 1 -VirtualPath "/Citrix/StoreWeb" -Loopback "OnUsingHttp" -LoopbackPortUsingHttp 81

The -Loopback parameter can take three possible values.