Product Documentation

Scaling XenMobile

Dec 22, 2015

Understanding the scale of your XenMobile infrastructure plays a significant role in how you decide to deploy and configure XenMobile. This article offers answers to common questions on determining the requirements for small to large scale enterprise deployments.

Performance and Scalability Guidelines

The data in this article are intended as guidelines for determining performance and scalability of a XenMobile infrastructure. The two key factors for determining how to configure your server and database are scalability (maximum users/devices) and logon rate.
  • Scalability is defined as the maximum number of concurrent users executing a defined workload. For more information on the flows used to load the XenMobile infrastructure, see Workloads.
  • Logon Rate is defined as the on-boarding of new users and the authentication of existing users.
    • On-boarding rate is the maximum number of devices that can be enrolled on the environment for the first time. Called First Time Use or FTU in this article, this data point is important when orchestrating a rollout strategy.
    • Existing user rate is the maximum number of users who authenticate to the environment, who have already enrolled and connected with their device. These tests included creating sessions for already enrolled users and the execution of WorxMail and WorxWeb apps.

The following table displays scalability guidelines based on the test results for the corresponding XenMobile environment.

Table 1. XenMobile Enterprise with Enrollment
Scalability Up to 100,000 devices
Logon Rates On-boarding (FTU) Up to 2,777 devices per hour
Existing users Up to 16,667 devices per hour
Configuration NetScaler Gateway MPX 20500
XenMobile Enterprise Edition XenMobile Server 10-node cluster
Database Microsoft SQL Server external database

System Configuration and Test Results

This section describes hardware configuration used and the results of running the on-boarding (FTU) workload and the Existing User workload scalability tests.

The following table defines the hardware and configuration recommendations for XenMobile when scaling from 1,000 to 100,000 devices. These guidelines are based on the test results and their associated workloads. The recommendations account for the acceptable margin of error as defined in Exit Criteria.

Analysis of the test results led to these conclusions:

  • Logon rate is an important factor in determining the scalability of a system. In addition to the initial logon, logon rates are dependent upon the authentication time-out values configured in your environment. For instance, if you set the authentication time-out value too low, users must perform more frequent logon requests. Therefore, you need to clearly understand how time-out settings affect your environment.
  • An external database (SQL Server) with 128 GB of RAM, 300 GB of disk space, and 24 virtual CPUs was used for the tests and is recommended for production environments.
  • To achieve maximum scalability, CPU and RAM resources were increased on XenMobile.
  • The 10-node cluster configuration was the largest configuration validated. Scaling beyond 10 nodes requires an additional XenMobile implementation.

Table 2. XenMobile Enterprise with Enrollment Scalability Results

Number of devices

1,000

10,000

30,000

60,000

100,000

Logon Rate

On-boarding (FTU)

125

1,250

2,500

2,500

2,777

Existing users

1,000

2,500

7,500

15,000

16,667

Configuration

Reference environment

VPX-XenMobile Standalone

MPX-XenMobile Standalone

MPX-XenMobile Cluster (3)

MPX-XenMobile Cluster (6)

MPX-XenMobile Cluster (10)

NetScaler Gateway

VPX with 2 GB RAM

2 virtual CPUs

MPX-10500

MPX-20500

XenMobile - mode

Standalone

Standalone

Cluster

XenMobile - cluster

N/A

N/A

3

6

10

XenMobile - virtual appliance

8 GB RAM and 4 virtual CPUs

16 GB RAM and 4 virtual CPUs

Database

External

The preceding table shows the recommended on-boarding and existing user logon rates based on the XenMobile configuration, NetScaler Gateway appliance, cluster settings, and database. Use the data in this table to construct an optimal enrollment schedule for new deployments and returning user/device rates for existing deployments. The Configuration section relates enrollment and logon performance data to the appropriate hardware recommendations.

Figure 1. XenMobile Enterprise with Enrollment — Logon Rate per Hour 

localized image

Note: You will experience the following if you exceed the recommended rates or hardware recommendations when sizing your system.

  • Enrollment or logon latency (round-trip time)
    • Total average latency: > 1.5 seconds
    • Average latency for a NetScaler Gateway logon: > 440 ms
    • Average latency for a Worx Store request: > 3 seconds
  • Physical performance degradation, such as CPU and memory exhaustion, was observed on the infrastructure components when scalability limits were reached.
    • Invalid responses on the NetScaler Gateway and XenMobile appliances.
    • Slow XenMobile console response time.

Figure 2. On-boarding (FTU) Logons and Error Percentage with Enrollment

localized image

The error percentage in the preceding figure includes the overall error experienced considering requests corresponding to every operation and is not limited to logons. The error percentage is within the acceptable limit for each test run as defined in Exit Criteria.

The following figure shows the reference architecture for a small scale deployment. It is a standalone architecture that supports up to 10,000 devices.

 

localized image

The following figure shows the reference architecture for an enterprise deployment. It is a clustered architecture with SSL offload for MAM over HTTP that supports 10,000 or more devices.

localized image

Test Methodology

The tests were run against XenMobile Enterprise to establish benchmarks. In an effort to target both small and large scale deployments, 1,000 to 100,000 devices were used in the measurements.

Workloads were created to simulate real-world use cases. These workloads were run for each test to study the effect on enrollment and logon rates. The objective of the tests was to obtain an optimal logon rate that was within the acceptable margin of error as outlined in Exit Criteria. Logon rates are a critical factor in determining the hardware configuration recommendations for the infrastructure components.

The On-boarding (FTU) workload logon requests included auto discovery, authentication, and device registration operations. Application subscription, installation, and launch operations were uniformly distributed throughout the test period. This provided the best real-world simulation of user actions. At the conclusion of the test, the session was logged out. The Existing User workload logon requests included only authentication requests.

Workloads

User workloads are defined as follows:
Table 3. User Workload Definitions
User sessions/devices Includes NetScaler Gateway logons, enumerations, device registration, and so on for each session.
Worx Store launches Users launch Worx Store multiple times and each time they subscribe to or install more than one app regardless of whether it is a mobile app (web/SaaS/MDX) or a Windows app (HDX).
Web/SaaS app SSO per device Accounts for the launch sequence of web/SaaS apps up to the point where XenMobile completes the SSO and returns the actual app URL. Traffic was not sent to actual apps.
MDX app downloads per device Counts of the number of MDX app downloads (this can happen across Worx Store launches). For iOS, this also includes the automation of app installation from Apple ITMS, which leverages the new token/tms service APIs on NetScaler Gateway.
 

Notes and Assumptions

In order to scale XenMobile beyond 30,000 devices, you should tune the following server parameters:

Config File - /opt/sas/sw/tomcat/inst1/webapps/ROOT/WEB-INF/classes/push_services.xml

  • <property name="heartbeatFrequency" value="24" />

 Config File - /opt/sas/sw/tomcat/inst1/webapps/ROOT/WEB-INF/classes/ew-config.properties

  • ios.mdm.apns.connectionPoolSize=15
  • hibernate.c3p0.max_size=1000

You should make these changes on all XenMobile nodes and then restart the server.

The following scenarios are not covered as part of the scalability tests. These scenarios would be considered for future enhancements in scale tests :

  • Policy Push to device is not tested.
  • Android Connected Devices not tested.
  • Package deployment is not tested.
  • Windows platform is not tested.

Each XenMobile supports a maximum of 10,000 connections simultaneously.

Tests were run in ideal conditions on LAN to ignore network latency issues. In a real world scenario, the scalability also depends on the user bandwidth available, especially for app downloads.

 
On-Boarding (FTU) Workload
The On-boarding (FTU) workload is defined as the first time a user accesses the XenMobile environment. Operations included in this workload were:
  • Auto discovery
  • Enrollment
  • Authentication
  • Device registration
  • Application delivery (web, SaaS, and mobile MDX apps)
    • App subscription (including images and icon downloads)
    • Installation of the subscribed MDX apps
  • App launch (web, SaaS, and mobile MDX apps)
  • Minimal WorxMail and WorxWeb connections (VPN tunnels) — two connections
  • Installation of required apps through XenMobile
The workload parameters included:
  • 1 device registration per device
  • 1 enumeration per device
  • 14 apps enumerated per device
  • 4 Worx Store launches per device
  • 4 web/SaaS app SSOs per device
  • 1 MDX app downloaded per device
  • 2 required app downloads
Existing Users Workload
The following table shows the Existing Users workload. This workload simulated a user using WorxMail and WorxWeb apps. This simulation was used to measure the NetScaler Gateway port's scalability within the XenMobile configuration. For the WorxWeb app, users were accessing internal web sites, which do not trigger XenMobile SSO. Operations in this mode included:
  • Authentication (NetScaler Gateway and XenMobile)
  • WorxMail and WorxWeb connections (VPN tunnels) — four connections
WorxApps Connection Profiles

The following table shows the workload parameters for existing users.

Table 4. WorxApps Connection Profiles
Device connection Connection type Data sent per session1 Data received per session1
WorxMail Connection #1 Type 12 4.1 MB 4.1 MB
WorxMail Connection #2 Type 1 6.3 MB 12.5 MB
WorxWeb Connection #1 Type 23 5.2 MB 15.7 MB
WorxWeb Connection #2 Type 2 4.1 MB 3.4 MB
Total bytes transferred per session1 ~19.7 MB ~ 40.7 MB

1. Per session: 8 hours.

2. Type 1: Asymmetric send and receive with long lived connections (that is, WorxMail with a dedicated Microsoft Exchange mailbox connection).

3. Type 2: Asymmetric send and receive with connections that close and reopen after delays (that is, WorxWeb connections).

Note: Modifications to the connection details affect analysis results. For example, if the number of connections per user is increased, then the number of NetScaler Gateway sessions supported may be reduced.
WorxMail and WorxWeb Profiles

The following tables show the WorxMail and WorxWeb profile details.

Table 5. WorxMail Profile for a Medium Workload
Messages sent per day 20
Messages received per day 80
Messages read per day 80
Messages deleted per day 20
Average message size (KB) 200
Table 6. WorxWeb Profile for Medium Workload
Number of web apps launched 10
Number of web pages opened manually 10
Average number of request–response pairs per web app 100
Average size of request (bytes) 300
Average size of response (bytes) 1000
Configuration and Parameters
The following configurations were used when running the scalability tests:
  • NetScaler Gateway and load balancing (LB) virtual servers coexisted on the same NetScaler Gateway appliance.
  • A 2048-bit key was used on NetScaler Gateway for SSL transactions.

Exit Criteria

Logon rates are the foundation of this analysis. They provide the guidelines for the infrastructure components and their respective configurations. It is important to note that the logon rates take into account a margin of error that consists of the following:

  • Invalid responses
    • A response with status code 401/404 instead of 200 is considered invalid.
  • Request time-outs
    • A response is expected within 120 seconds.
  • Connection errors
    • A connection reset occurs.
    • An abrupt connection termination occurs.

The logon rate is acceptable if the overall error rate is less than 1 percent of the total requests that are sent from a given device. The error rate includes errors corresponding to each individual workload operation, as well as the physical performance of the infrastructure component, such as CPU and memory exhaustion.

Software and Hardware Details

The following table lists the XenMobile infrastructure software used for these tests.

Table 8. XenMobile Infrastructure Components

Component Version
NetScaler Gateway 10.5-57.4.nc
XenMobile 10.1.0.63030
External database MS SQL Server 2014 R2

(128 GB RAM, 300 GB disk space, 24 virtual CPUs)

The scalability tests were run on a XenServer platform as outlined in the following table.

Table 9. XenServer Hardware

Vendor Genuine Intel
Model Intel Xeon CPU — E5645 @ 2.40 GHz (CPUs = 24)

This includes the infrastructure core services (for example, Active Directory, Windows Domain Name Service (DNS), Certificate Authority, Microsoft Exchange, and so on), as well as the XenMobile components (XenMobile virtual appliance and the NetScaler Gateway VPX virtual appliance, where applicable).

For additional product information and technical questions concerning this article or the products mentioned herein, see Citrix.com, search the XenMobile documentation site for the latest product documentation, or contact your local Citrix representative.