Load balancing with Citrix ADC appliance

This article provides guidance on how to deploy a StoreFront server group containing two or more StoreFront servers in all active load balanced configuration. The article provides details of how to configure a Citrix ADC appliance to load balance incoming requests from Citrix Workspace app, and Citrix Receiver for Web between the StoreFront nodes in the server group. This article also shows you how to configure StoreFront Monitor for use with a Citrix ADC appliance.

The examples in this section have been tested in the following environment:

  • Four Windows Server 2012 R2 StoreFront 3.x nodes in a single server group.
  • One Citrix ADC appliance 12.1 load balancer configured for Least Connection and CookieInsert “sticky” load balancing.
  • One Windows 10 test client with Citrix Workspace app installed.

Server certificate requirements for the load balanced deployment if you intend to use HTTPS

Review the section Plan gateway and server certificate usage.

Consider the following options before purchasing a certificate from a commercial certificate authority or issuing one from your enterprise certificate authority.

  • Option 1: Use a *.example.com wildcard certificate on both the Citrix ADC appliance load balancing vServer and on the StoreFront server group nodes. This simplifies the configuration and allows you to add extra StoreFront servers in the future without the need to replace the certificate.
  • Option 2: Use a certificate including Subject Alternative Names (SANs) on both the Citrix ADC appliance load balancing vServer and on the StoreFront server group nodes. Extra SANs within the certificate that match all of StoreFront server fully qualified domain names (FQDNs) are optional, but recommended, as this allows greater flexibility in the StoreFront deployment. Include a SAN for email-based discovery discoverReceiver.example.com.

For details of email-based discovery configuration, see http://blogs.citrix.com/2013/04/01/configuring-email-based-account-discovery-for-citrix-receiver/.

Note:

When exporting the private key associated with the certificate is not feasible. Use two separate certificates: one on the Citrix ADC appliance load balancing vServer and a different certificate on the StoreFront server group nodes. Both certificates must include Subject Alternative Names.

localized image

localized image

Create a server certificate for the Citrix ADC appliance load balancer and all StoreFront servers

Import a certificate issued from a Windows Certificate Authority onto a Citrix ADC appliance

  • WinSCP is a useful third party and free tool to move files from a Windows machine to a Citrix ADC appliance file system. Copy certificates for import to the /nsconfig/ssl/ folder within the Citrix ADC appliance file system.
  • You can also use OpenSSL tools on the Citrix ADC appliance to extract the certificate and key from a PKCS12/PFX file to create a two separate .CER and .KEY X.509 files in PEM format that Citrix ADC can use.
  1. Copy the PFX file into /nsconfig/ssl/ on the Citrix ADC appliance or VPX.
  2. Open the Citrix ADC appliance command line interface (CLI).
  3. Type Shell to exit the Citrix ADC appliance CLI and switch to the FreeBSD shell.
  4. Change directory using cd /nsconfig/ssl/.
  5. Run openssl pkcs12 -in <imported cert file>.pfx -nokeys -out <certfilename>.cer and enter the PFX password when prompted.
  6. Run openssl pkcs12 -in <imported cert file>.pfx -nocerts -out <keyfilename>.key and enter the PFX password when prompted, and then set the private key PEM passphrase to protect the .KEY file.
  7. Run ls -al to check the .CER and .KEY files have been successfully created inside /nsconfig/ssl/.
  8. Type Exit to return to the Citrix ADC appliance CLI.

Configure the server certificate on the Citrix ADC appliance after it is imported

  1. Log on to the Citrix ADC appliance management GUI.
  2. Select Traffic Management > SSL > SSL Certificates and click Install.
  3. On the Install Certificate window, enter the certificate and private key pair names.
    • Select the .cer certificate file on the Citrix ADC appliance file system under /nsconfig/ssl/.
    • Select the .key file containing the private key from the same location.

localized image

Create DNS records for the StoreFront server group load balancer

Create a DNS A and PTR record for your chosen shared FQDN. Clients within your network use this FQDN to access the StoreFront server group using the Citrix ADC appliance load balancer.

Example: storefront.example.com resolves to the load balancing vServer virtual IP (VIP).

Scenario 1: End-to-end HTTPS 443 secure connection between the client and the Citrix ADC appliance load balancer, and between the load balancer and multiple StoreFront 3.x servers

This scenario uses a modified StoreFront monitor using port 443.

Add individual StoreFront server nodes to the Citrix ADC appliance load balancer

  1. Log onto the Citrix ADC appliance management GUI.
  2. Select Traffic Management > Load Balancing > Servers > Add and add each of the four StoreFront nodes to be load balanced.

    Example = 4 x 2012R2 StoreFront Nodes named 2012R2-A to -D.

  3. Use IP-based server configuration and enter the server IP address for each StoreFront node.

localized image

Define a StoreFront monitor to check the status of all StoreFront nodes in the server group

  1. Log onto the Citrix ADC management GUI.
  2. Select Traffic Management > Load Balancing > Monitors > Add and add a new monitor called StoreFront and accept all default settings.
  3. From the Type drop down menu, select StoreFront.
  4. If you are using HTTPS connections between your load balancing vServer and StoreFront, ensure that the Secure option is selected; otherwise leave this option unselected.
  5. In the Special Parameters tab, type the Store Name.
  6. In the Special Parameters tab, select the Check Backend Services option. This option enables monitoring of services running on the StoreFront server. StoreFront services are monitored by probing a Windows service that runs on the StoreFront server, which returns the status of the following services:
    • W3SVC (IIS)
    • WAS (Windows Process Activation Service)
    • CitrixCredentialWallet
    • CitrixDefaultDomainService

    localized image

Create an HTTPS 443 service group containing all of the StoreFront servers

  1. Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.

  2. Set the TLS port and give each node a unique server ID as you add it.

    localized image

  3. On the Monitors tab, select the StoreFront monitor you created earlier.

    localized image

  4. On the Certificates tab, bind the server certificate you imported earlier.

  5. Bind the CA certificate used to sign the server certificate you imported earlier, and any other CAs that might be part of the PKI chain of trust.

    localized image

Create a load balancing vServer for user traffic

  1. Log onto the Citrix ADC appliance management GUI.

  2. Select Traffic Management > Load Balancing > Virtual Servers > Add to create a new vServer.

  3. Select the load balancing method for the vServer. Common choices for StoreFront load balancing are round robin or least connection.

    localized image

  4. Bind the Service Group you created earlier to the load balancing vServer.

  5. Bind the same server and CA certificate you previously bound to the service group, to the load balancing vServer.

  6. From within the load balancing vServer menu, select Persistence on the right hand side and set the persistence method to be COOKIEINSERT.

  7. Name the cookie. For example, NSC_SFPersistence, as this makes it easy to identify in Fiddler traces during debugging.

  8. Set backup persistence to NONE.

localized image

Scenario 2: HTTPS termination—HTTPS 443 communication between the client and the Citrix ADC load balancer, and HTTP 80 connections between the load balancer and the StoreFront 3.x servers behind it

This scenario uses the default StoreFront monitor using port 8000.

Add individual StoreFront server servers to the Citrix ADC load balancer

  1. Log onto the Citrix ADC management GUI.
  2. Select Traffic Management > Load Balancing > Servers > Add and add each of the four StoreFront servers to be load balanced. Example = 4 x 2012R2 Storefront servers called 2012R2-A to -D. 3.
  3. Use IP based Server configuration and enter the server IP address for each Storefront server.

Define an HTTP 8000 StoreFront monitor to check the status of all StoreFront servers in the server group

  1. Log onto the Citrix ADC management GUI.
  2. Select Traffic Management > Monitors > Add and add a new monitor called StoreFront.
  3. Add a name for the new monitor and accept all default settings.
  4. In the Type list select StoreFront.
  5. In the Special Parameters tab, type the Store Name.
  6. Type 8000 into Destination Port. This matches the default monitor instance that is created on each StoreFront server.
  7. In the Special Parameters tab, select the Check Backend Services option. This option enables monitoring of services running on the StoreFront server. StoreFront services are monitored by probing to a Windows service that runs on the StoreFront server, which returns the status of all running StoreFront services.

Create an HTTP 80 service group containing all of the StoreFront servers

  1. Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.
  2. Set the HTTP port to 80 and give each server a unique server ID as you add it.
  3. On the Monitors tab, select the StoreFront monitor you created earlier.

Create an HTTPS terminating load balancing vServer for user traffic

  1. Select Traffic Management > Load Balancing > Virtual Servers > Add to create a new vServer.
  2. Select the load balancing method vServer will use. Common choices for StoreFront load balancing are round robin or least connection.

  3. Bind the Service Group you created earlier to the load balancing vServer.

  4. Bind the same server and CA certificate you previously bound to the service group, to the load balancing vServer.

    Note:

    If the client is not allowed to store the HTTP cookie, the subsequent requests do not have the HTTP cookie and Persistence is not used.

  5. From within the load balancing vServer menu, select Persistence on the right hand side and set the persistence method to be COOKIEINSERT.

  6. Name the cookie. For example, NSC_SFPersistence, as this makes it easy to identify in Fiddler traces during debugging.
  7. Set backup persistence to NONE.

Create a load balancing vServer for subscription synchronization between server groups

Considerations before creating a load balancing vServer include the following:

  • Option 1: Create a single vServer: to load balance only user traffic. This is all that is needed if performing only ICA launches of published apps and desktops. (Mandatory and usually all that is required.)
  • Option 2: Create a pair of vServers: one to load balance user traffic for performing ICA launches of published apps and desktops and another for load balancing subscription data synchronization operations. (Necessary only when propagating subscription data between two or more load balanced StoreFront server groups in a large multi-site deployment.)

If a multisite deployment consists of two or more StoreFront server groups located in separate geographic locations, you can replicate subscription data between them using a pull strategy on a repeating schedule. StoreFront subscription replication uses TCP port 808, so using an existing load balancing vServer on HTTP port 80 or HTTPS 443 fails. To provide high availability for this service, create a second vServer on each Citrix ADC appliance in your deployment to load balance TCP port 808 for each of the StoreFront server groups. When configuring the replication schedule, specify a server group address that matches the subscription synching vServer virtual IP address. Ensure the server group address is the FQDN of the load balancer for the server group at that location.

Configure a service group for subscription synchronization

  1. Log onto the Citrix ADC appliance management GUI.
  2. Select Traffic Management > Service Groups > Add and add a new service group.
  3. Change the protocol to TCP.
  4. Within your Service Group, select the Members option on the right hand side and add all of the StoreFront server nodes you defined previously in the Servers section.
  5. On the Monitors tab, select the TCP monitor.

localized image

Create a load balancing vServer for subscription synchronization between server groups

  1. Log onto the Citrix ADC appliance management GUI.
  2. Select Traffic Management > Service Groups > Add and add a new service group.
  3. Set the load balancing method to round robin.
  4. Change the protocol to TCP.
  5. Enter 808 and NOT 443 as the port number.

localized image

Membership within CitrixSubscriptionsSyncUsers

For StoreFront server A at Location A to request and pull subscription data from server B at a different location, server A must be a member of the CitrixSubscriptionsSyncUsers local security group on server B. The CitrixSubscriptionsSyncUsers local group contains an access control list of all remote StoreFront servers authorized to pull subscription data from a particular server. For bidirectional subscription synchronization, server B must also be a member of the CitrixSubscriptionsSyncUsers security group on server A to pull subscription data from it.

localized image

Scenario 1: Configure the StoreFront server group using HTTPS between Citrix ADC and StoreFront

  1. Import the same certificate and private key that was deployed on the Citrix ADC appliance load balancing vServer to every StoreFront node in the server group.

  2. Create an HTTPS binding in IIS on every StoreFront node, and then bind the certificate you imported earlier to it.

    localized image

  3. If you are using HTTPS between the Citrix ADC load balancer and StoreFront, you must use a certificate containing the load balanced FQDN as a Common Name (CN) or as a Subject Alternative Name (SAN).

    See Create a server certificate for the Citrix ADC appliance load balancer and StoreFront servers.

Scenario 2: Configure the StoreFront server group using HTTP between Citrix ADC and StoreFront

  1. Remove the HTTPS binding in IIS from every StoreFront node if this already exists.
  2. Ensure that the HTTP binding is present in IIS, and that it is set to use port 80.
  3. Configure the loopback settings within Receiver for Web as OnUsingHTTP and port 80. This step is essential to ensure that client detection between native Citrix Workspace App and Receiver for Web succeeds.

    Advanced Settings.

Steps common to both Scenarios 1 and 2

  1. Install StoreFront on every node in the server group.

  2. During installation of StoreFront, set the host base URL on the primary node to be the shared FQDN used by all members of the server group. This should always be https://storefrontlb.domain.com for both scenario 1 and 2 and must match the FQDN of the Citrix ADC load balancing vServer.

    See the Create a server certificate for the Citrix ADC appliance load balancer and StoreFront servers.

  3. After you have completed the initial StoreFront configuration, join each of the nodes, one after the other, to the server group using the primary node.

  4. Select Server Group > Add Server > Copy the Authorization Code to the joining Server.

    localized image

  5. Propagate the configuration from the primary node to all other server group nodes in the group.

  6. Test the load balanced server group using a client that can contact and resolve the shared FQDN of the load balancer.

Citrix Service Monitor

To enable external monitoring of the run-state of the Windows services on which StoreFront relies for correct operation, use the Citrix Service Monitor Windows service. This service has no other service dependencies and can monitor and report the failure of other critical StoreFront services. The monitor enables the relative health of a StoreFront server deployment to be determined externally by other Citrix components, such as a Citrix ADC appliance. Third party software can consume the StoreFront monitor XML response to monitor the health of essential StoreFront services.

After StoreFront is deployed, a default monitor that uses HTTP and port 8000 is created.

Note:

Only one instance of a monitor can exist within a Storefront deployment.

To make any changes to the existing default monitor, such as changing the protocol and port to HTTPS 443, use PowerShell cmdlets to view or reconfigure the StoreFront monitor service URL.

Remove the default Service Monitor and replace it with one that uses HTTPS and port 443

  1. Open the PowerShell Integrated Scripting Environment (ISE) on the primary StoreFront server and run the following commands to change the default monitor to HTTPS 443.

    $ServiceUrl = "https://localhost:443/StorefrontMonitor"
    Set-STFServiceMonitor -ServiceUrl $ServiceUrl
    Get-STFServiceMonitor
    
  2. Once completed, propagate the changes to all other servers in the StoreFront server group.

  3. To perform a quick test on the new monitor, enter the following URL into the browser on the StoreFront server or any other machine with network access to the StoreFront server. The browser should return an XML summary of the status of every Storefront service.

    https://<loadbalancingFQDN>:443/StoreFrontMonitor/GetSFServicesStatus

    localized image

Citrix Gateway and load balancing vServers on the same Citrix ADC appliance

If you have configured the Citrix Gateway virtual server and load balancing vServer on the same Citrix ADC appliance, internal domain users might experience issues when trying to access the StoreFront load balanced host base URL directly rather than passing through the Citrix Gateway virtual server.

In this scenario StoreFront assumes that the end user has already authenticated at the Citrix Gateway because StoreFront correlates the source IP address of the incoming user with the Citrix Gateway Subnet IP Address (SNIP). This triggers StoreFront to attempt to use the AGBasic protocol to perform Citrix Gateway silent authentication, rather than actually prompting the user to log on with their domain credentials. To avoid this issue, omit a SNIP address as shown below or enter a VIP so that username and password authentication is used instead of the AGBasic logon protocol.

Configure a Citrix Gateway on the Storefront Server Group

Gateway General Settings.

Enter the Citrix Gateway VIP and into the VServer IP address field. Do NOT use the SNIP for the Citrix Gateway if the load balancing vServer resides on the same Citrix ADC appliance.

Gateway Authentication Settings.

Loopback options when load balancing a StoreFront server group using a Citrix ADC appliance

You can set loopback options using PowerShell.

Example Receiver for Web web.config file

<communication attempts="2" timeout="00:01:00" loopback="On" loopbackPortUsingHttp="80">

Example PowerShell command

& "c:\program files\Citrix\receiver storefront\scripts\ImportModules.ps1"
Set-DSLoopback -SiteId 1 -VirtualPath "/Citrix/StoreWeb" -Loopback "OnUsingHttp" -LoopbackPortUsingHttp 81

The -Loopback parameter can take three possible values:

Value Context
On — Changes the host of the URL to 127.0.0.1. The schema and port (if specified) are not changed. Cannot be used if TLS-terminating load balancer is used.
OnUsingHttp — Changes the host to 127.0.0.1 and schema to HTTP and modifies the port the value configured for loopbackPortUsingHttp attribute. Use only when the load balancer is TLS terminating. Communication between the load balancer and StoreFront servers is with HTTP. You can explicitly configure the HTTP port using the -loopbackPortUsingHttp attribute.
Off — The URL in the request is not modified in any way. Use for trouble shooting. Tools like Fiddler cannot capture the traffic between Receiver for Web and StoreFront Services if loopback is set to On.