Citrix Virtual Apps and Desktops on Azure

Introduction

This guide will assist with the Architecture and deployment model of Citrix Virtual Apps and Desktops services on Microsoft Azure.

The combination of Citrix Cloud and Microsoft Azure makes it possible to spin up new Citrix virtual resources with greater agility and elasticity, adjusting usage as requirements change. Virtual Machines on Azure support all of the control and workload components required for a Citrix Virtual Apps and Desktops service deployment. Citrix Cloud and Microsoft Azure have common control plane integrations that establish identity, governance, and security for global operations.

This document also provides guidance on prerequisites, architecture design considerations, and deployment guidance for customer environments. The document will highlight the design decisions and deployment considerations across the following five key architectural principles:

  • Operations - Operations includes a wide variety of topics such as image management, service monitoring, business continuity, support, etc. A variety of tools are available to assist with automation of operations including Azure PowerShell, Azure CLI, ARM Templates, and Azure API.

  • Identity - One of the cornerstones of the entire picture of Azure is the identity of a person and their role-based access (RBAC). Azure identity is managed through Azure Active Directory (Azure AD) and Azure AD Domain Services. The customer must decide which way to go for its identity integration.

  • Governance - The key to governance is establishing the policies, processes, and procedures associated with the planning, architecture, acquisition, deployment, and operational management of Azure resources.

  • Security - Azure provides a wide array of configurable security options and the ability to control them so that customers can customize security to meet the unique requirements of their organization’s deployments. This section helps to understand how Azure security capabilities can help you fulfill these requirements.

  • Connectivity - Connecting Azure virtual networks with customer’s local/cloud network is referred to as hybrid networking. This section explains the options for network connectivity and network service routing.

Planning

The three most common scenarios for delivering Citrix Apps and Desktops through Azure are:

  • Greenfield deployment with Citrix Cloud delivering resource locations in Azure. This scenario is delivered via Citrix Virtual Apps and Desktops service and used when customers prefer to go to a subscription model and outsource control plane infrastructure to Citrix.
  • Extending an on-premises deployment into Azure. In this scenario, the customer has a current on-premises control layer and would like to add Azure as a Citrix resource location for new deployments or migration.
  • Lift and shift. With this scenario, customers deploy their Citrix Management infrastructure into Azure and treat Azure as a site, using Citrix NetScaler and StoreFront to aggregate resources from multiple sites.

This document focuses on the Citrix Cloud deployment model. Customers can plan and adopt these services based on their organization needs:

Citrix Virtual Apps and Desktops Service

Citrix Cloud Virtual Apps and Desktops services simplifies the delivery and management of Citrix technologies, helping customers to extend existing on-premises software deployments or move one hundred percent to the cloud. Deliver secure access to Windows, Linux and Web apps and Windows and Linux virtual desktops. Manage apps and desktops centrally across multiple resource locations while maintaining a great end user experience.

Citrix Virtual Desktops Essentials Service

Citrix and Microsoft customers have more options to deploy Windows 10 Enterprise within their organization. Citrix Virtual Desktops Essentials Service accelerates Windows 10 Enterprise migration for customers who prefer Microsoft Azure cloud solutions. This enables customers who have licensed Windows 10 Enterprise (Current Branch for Business) on a per-user basis the option to deliver a high-performance Windows 10 Enterprise virtual desktop experience from Azure.

Citrix Virtual Apps Essentials Service

Citrix Virtual Apps Essentials, the new application virtualization service, combines the power and flexibility of the Citrix Cloud platform with the simple, prescriptive, and easy-to-consume vision of Microsoft Azure RemoteApp. Citrix Virtual Apps Essentials provides superior performance and flexibility by moving the backend infrastructure to the cloud, simplifying app delivery without sacrificing management or end user experience.

Conceptual Reference Architecture

This conceptual architecture provides common guidelines for deployment of a Citrix Cloud resource location in Azure which will be discussed in the following sections.

Azure-RA-Image-1

Diagram-1: Citrix Cloud Conceptual Reference Architecture

Refer to the whitepaper on scalability and economics of delivering Citrix Virtual Apps and Desktops services on Microsoft Azure

Operations

In the operations subject area, this guide dives deeper into planning for the workspace environment requirements and hierarchy for foundational services. At the top layer, is found the subscription, resource group, and regional design considerations. Followed by common questions for VM storage, user profile storage, and master image management/provisioning. Also provided is guidance on Reserved instance optimization with Smart Scale and planning for Business Continuity/Disaster Recovery.

Naming Conventions

The naming of resources in Microsoft Azure is important because:

  • Most resources cannot be renamed after creation
  • Specific resource types have different naming requirements
  • Consistent naming conventions make resources easier to locate and can indicate the role of a resource

The key to success with naming conventions is establishing and following them across your applications and organizations.

When naming Azure subscriptions, verbose names make understanding the context and purpose of each subscription clear. Following a naming convention can improve clarity when working in an environment with many subscriptions.

A recommended pattern for naming subscriptions is:

Variable Example Description
[System] CTX (Citrix) CORE (Azure) Three letter identifier for the product, application, or service that the resource supports.
[Role] XAW (XenApp Workers) VDA (Virtual Desktop Agent) CC (Cloud Connector) CVA (Citrix Virtual Apps) Three letter identifier for a subsystem of the service.
[Environment] D, T, P (dev, test, or prod) Identifies the environment for the resource
## 01, 02 For resources that have more than one named instance (web servers, etc.).
[Location] wu (West US), eu (East US), scu (South Central US) Identifies the Azure region into which the resource is deployed

When naming resources in Azure use common prefixes or suffixes to identify the type and context of the resource. While all the information about type, metadata, context, is available programmatically, applying common affixes simplifies visual identification. When incorporating affixes into your naming convention, it is important to clearly specify whether the affix is at the beginning of the name (prefix) or at the end (suffix).

A well-defined naming scheme should identify the system, role, environment, instance count, and location of an Azure resource. Naming can be enforced using Azure Policy.

Service Scope Suggested Pattern Example
Subscriptions Global [System][Environment]##[Location]-sub WSCD01scu-sub
Resource Groups Global [System]-[Role]-[Environment]##-[Location]-rg CTX-Apps-P01-CUS-rg
Virtual Network Resource Group [System][Environment]##[Location]-vnet 10.0.0.0/16
Subnet Parent VNET [Descriptive Context] DMZ - 10.0.1.0/24 Infrastructure - 10.0.2.0/24
Storage Account Resource Group [System][Role][Environment]##[Location] Note: Must be lower case alphanumeric ctxinfd01scu
Container Storage Account [Descriptive Context] vhds
Virtual Machine Resource Group [System][Role][Environment]##[Location] Note: Must be 15 characters or less. CTXSTFD01scu
Network Interface Resource Group [vmname]-nic# CTXSTFD01scu-nic1
Public IPs Resource Group [vmname]-pip CTXSTFD01scu-pip
Virtual Network Gateway Virtual Network [System][Environment]##[Location]-vng WSCD01scu-vng
Local Network Gateway Resource Group [System][Environment]##[Location]-lng WSCD01scu-lng
Availability Sets Resource Group [System][Role]-as CTXSTF-as
Load Balancer Resource Group [System][Role]-lb CTXNSG-lb
Workspaces Subscription [System][Environment]-analytics CTXP-analytics
Tags Resource [Descriptive Context] Finance
Key Vault Subscription [System][Environment]-vault CTXP-vault

Subscriptions

Selecting a subscription model is a complex decision that involves understanding the growth of the customer’s Azure footprint within and outside the Citrix deployment. Even if the Citrix deployment is small, the customer might still have a large amount of other resources that are reading/writing heavily against the Azure API, which could have a negative impact on the Citrix environment. The reverse is also true, where a large number of Citrix resources could consume an inordinate number of the available API calls, reducing availability for other resources within the subscription.

Single Subscription workspace model

In a single subscription model, all core infrastructure and Citrix infrastructure are located in the same subscription. This is the configuration recommended for deployments that will require up to 1,000 Citrix VDAs.

Azure-RA-Image-2

Diagram-2: Azure Single Subscription workspace model

Multi-Subscription workspace model

In this model, core infrastructure and Citrix infrastructure are in separate subscriptions to manage the scalability in large deployments. Often enterprise deployments with multi-region infrastructure design are broken into multiple subscriptions to prevent reaching Azure subscription limits.

Azure-RA-Image-3

Diagram-3: Azure Multi-Subscription workspace model

The following questions provide guidance to help customer’s understand the Azure subscription options and plan their resources.

Component Requirement
Will the Azure subscription contain only Citrix resources? Determine if the Azure subscription will be used for dedicated Citrix resources or if the Citrix resources will be shared with other systems.
Single or Multiple subscription deployment? Typically, multiple subscription deployments are for larger deployments where single subscription limitations are an issue and more granular security controls are necessary.
What Azure Limits are likely to be reached? How many resources will be in a resource Group? Resource Groups has limits and Machine Creation Services (MCS) requires either 2 or 3 disks per VM resource. Review Azure subscription limits while planning the solution.
What permissions are necessary for the CVAD service principle on the Azure subscription? Citrix Virtual Apps and Desktops requires to create new resource groups and resources within the subscription. For example, when the service principle cannot be granted full access to a subscription, then it will need to be granted Contributor access to a pre-created resource group.
Will Development and Test environments be created in separate subscriptions from Production? Isolating Development and Test subscriptions from Production enables the application and change of global Azure services in an isolated environment and silos resource utilization. This practice has benefits for security, compliance, and subscription performance. Creating separate subscriptions for these environments does add complexity to image management. These trade-offs should be considered based on the customer’s needs.

Azure Regions

An Azure region is a set of datacenters deployed within a latency-defined perimeter and connected through a dedicated regional low-latency network. Azure gives customers the flexibility to deploy applications where they need to. Azure is generally available in 42 regions around the world, with plans announced for 12 additional regions as of Nov 2018.

A geography is a discrete market, typically containing two or more Azure regions, that preserves data residency and compliance boundaries. Geographies allow customers with specific data-residency and compliance needs to keep their data and applications close.

Availability Zones are physically separate locations within an Azure region. Each Availability Zone is made up of one or more datacenters equipped with independent power, cooling and networking. Availability Zones allow customers to run mission-critical applications with high availability and low-latency replication. To ensure resiliency, there’s a minimum of three separate zones in all enabled regions.

Consider these factors when choosing your region.

Component Requirement
Compliance and data residency Do customers have specific compliance or data-residency requirements? Microsoft may copy customer data between Regions within a given Geo for data redundancy or other operational purposes. For example, Azure Globally-Redundant Storage (GRS) replicates Blob and Table data between two regions within the same Geo for enhanced data durability in case of a major datacenter disaster. Certain Azure services do not enable the customer to specify the region where the service will be deployed. These services may store customer data in any of Microsoft’s datacenters unless specified. Review AzureDatacenterMap website for latest updates.
Service availability Review service availability within the tentative regions. Service Availability by region will help customer to determine which services are available within a region. While an Azure Service may be supported in a given region, not all Service features are available in sovereign clouds, such as Azure Government, Germany, and China.
Determine the target Azure region(s) for the Citrix deployment. Review proximity of Azure region to users and customer datacenters.
Are multiple Azure regions required? Multiple Azure regions are typically considered for the following high-level reasons: - Proximity to application data or end users - Geographic Redundancy for Business Continuity and Disaster Recovery - Azure Feature or Service availability

Availability Sets

An Availability Set is a logical grouping capability that can be used in Azure to ensure that the VM resources placed within an Availability Set are isolated from each other when they are deployed within an Azure datacenter. Azure ensures that the VMs placed within an Availability Set run across multiple physical servers, compute racks, storage units, and network switches. If a hardware or Azure software failure occurs, only a subset of your VMs are impacted, and the overall application stays up and continues to be available to customers. Availability Sets are an essential capability when customers want to build reliable cloud solutions.

Each component of a Citrix deployment should be in its own Availability Set to maximize overall availability for Citrix. For example, a separate Availability Set should be used for Cloud Connectors, another for Citrix Application Delivery Controllers (ADC), StoreFront, etc.

Once availability sets are optimized, the next step is to build resiliency around VM downtime within the availability sets. That minimizes/eliminates service downtime when VM’s are restarted or redeployed by Microsoft. This can be expanded to planned maintenance events as well. There are two features that could make use of which could increase the reliability of the overall service.

Please note that these two features do not protect against unplanned maintenance/crashes.

  • Azure Planned Maintenance
  • Azure Scheduled Events

Azure Planned Maintenance

Azure periodically performs updates to improve the reliability, performance, and security of the host infrastructure in Azure. If maintenance requires a reboot, Microsoft will send a notice. Using Azure Planned Maintenance, it is possible to capture these notices and proactively take action on them on the customer’s schedule, instead of on Microsoft’s schedule.

Make use of planned maintenance feature by sending email notifications to the service owner of each tier (for manual intervention) and build runbooks to automate the service protection.

Azure Scheduled Events

Azure Scheduled Events is an Azure Metadata Service that gives notices programmatically to applications to alert of immediate maintenance. It provides information about upcoming maintenance events (e.g. reboot) so the application administrator can prepare for and limit disruption. While it might sound like planned maintenance, it is not. The key difference is that these events are fired for planned maintenance and sometimes non-planned maintenance. For example, if Azure is doing host healing activities and needs to move VMs on a short notice.

These events are consumed programmatically, and will give the following advance notice:

  • Freeze – 15 Minutes
  • Reboot – 15 Minutes
  • Redeploy – 10 Minutes

Disaster Recovery (DR)

Azure can provide a highly cost-effective DR solution for Citrix customers looking to gain immediate value from cloud adoption today. The deployment model topology determines the DR solution implementation.

Extending the Architecture

Under this topology, the management infrastructure remains on-premises, but workloads are deployed to Azure. If the on-premises datacenter is not reachable, existing connected users will remain connected, but new connections will not be possible because the management infrastructure is unavailable.

To protect the management infrastructure, pre-configure Azure Site Recovery to recover management infrastructure into Azure. This is a manual process and once recovered, your environment can be made operational. This option is not seamless and cannot recover components such as NetScaler VPX, however for organizations with more a more flexible recovery time objective (RTO) it can reduce the operational costs.

Hosting Architecture

When deploying this topology, the Citrix Management infrastructure is deployed into Azure and treated as a separate site. This provides functional isolation from on-premise deployment in the event of a site failure. Use Citrix NetScaler and StoreFront to aggregate resources and provide users a near instant failover between Production and Disaster Recovery resources.

The presence of the Citrix Infrastructure in Azure means that no manual processes need to be invoked and no systems need to be restored before users can access their core workspace.

Cloud Services Architecture

When using Citrix Cloud, Azure becomes just another resource location. This topology provides the simplest deployment as the management components are hosted by Citrix as a Service, and Disaster Recovery workloads can be achieved without deploying duplicate infrastructure to support it. The user experience during failover in the event of a disaster can be virtually seamless.

The items in the following table will help the customer with their DR planning:

Component Requirement
What are the RTO and RPO requirements of the Citrix environment? RTO - Targeted duration of time and a service level within which a business process must be restored after a disaster. RPO - The interval of time that might pass during a disruption before the quantity of data lost during that period exceeds the Business Continuity Plan’s maximum allowable threshold or “tolerance.”
What is the desired outcome when a service disruption occurs in the entire region where your Azure virtual machine application is deployed? These options should be reviewed in alignment with the customer’s RTO and RPO for DR. Disaster Recovery of a Citrix environment in Azure can be addressed with the following: *Azure Site Recovery *Passive Secondary Site *Active Site Azure Site Recovery only supports Server OS (Citrix infrastructure and Server VDAs). Client OS is not supported (for example persistent desktops created using ARM Templates). Additionally, Machine Catalogues created by MCS (Server or Client VDA) must be recreated using a Recovery Task.

Resource Groups

Resource Groups (RG) in Azure is a collection of assets in logical groups for easy or even automatic provisioning, monitoring, and access control, and for more effective management of their costs. The benefit of using RGs in Azure is grouping related resources that belong to an application together, as they share a unified lifecycle from creation to usage and finally, de-provisioning.

The key to having a successful design of resource groups is understanding the lifecycle of the resources that are included in them.

One or more Resource Groups can be applied to a Machine Catalog during initial creation. These Resource Groups cannot be shared across Machine Catalogues. Resource Groups are limited to 240. Citrix MCS VMs due to the 800 Resource Count limitation per type of resource within a Resource Group.

Resource Groups are tied to Machine Catalogues at creation time and cannot be added or changed later. To add additional Resource Groups to a Machine Catalog, the Machine Catalog must be removed and recreated.

Image Management

Image management is the process of creating, upgrading, and assigning an image that is consistently applied across development, test, and production environments. The following should be considered when developing an image management process:

On-Demand Provisioning

The customer will need to determine if MCS be used to manage the Azure non-persistent machines or create their own Azure Resource Manager (ARM) templates. When a customer uses MCS to create machine catalogues, the Azure on-demand provisioning feature reduces storage costs, provides faster catalog creation and faster virtual machine (VM) power operations. With Azure on-demand provisioning, VMs are created only when Citrix Virtual Apps and Desktops initiates a power-on action, after the provisioning completes. A VM is visible in the Azure portal only when it is running, while in Citrix Studio, all VMs are visible, regardless of power status. Machines created via ARM templates or MCS can be power managed by Citrix using a Azure host connection in Citrix Studio.

Storage Account Containers

The customer will need to decide the organizational structure for the storing source (or golden) images from which to create the virtual machines using Citrix Machine Creation Services (MCS). Citrix MCS images can be sourced from snapshots, managed or unmanaged disks and can reside on standard or premium storage. Unmanaged disks are accessed through general purpose storage accounts and are stored as VHDs within Azure Blob storage containers. Containers are folders which can be used to separate Production, Test, and Development images.

Image Replication

The customer will need to determine the appropriate process for replicating images across regions and how Citrix App Layering technology might be leveraged within the overall image management strategy. PowerShell scripts may be used in combination with Azure Automation to schedule image replication. More information on Citrix App Layering can be found here, but keep in mind that Elastic Layering requires an SMB File share that does not reside on Azure Files. See the File Servers section for supported SMB share technologies that work with Elastic Layering.

File Server Technologies

Azure offers several file server technologies that can be leveraged to store Citrix user data, roaming profile information or function as targets for Citrix Layering shares. These options include the following:

  • Standalone File Server
  • File Servers using Storage Replica
  • Scale Out File Server (SOFS) with Storage Spaces Direct (S2D)
  • Distributed File System – Replication (DFS-R)
  • Third-party storage appliances from Azure marketplace (such as NetApp, etc.)

The customer should select file server technologies that best meet their business requirements. The table below outlines some benefits and considerations for each of the different file serving technologies.

Options Benefits Considerations
Standalone File Server Well known and tested. Compatible with existing backup/restore products Single point of failure No data redundancy Outage for monthly patching, usually measured in minutes
File Servers using Storage Replica Block Level Replication SMB 3.0 Storage Agnostic (SAN, Cloud, Local, etc.) Offers Synchronous and Asynchronous Replication Recommended when multi-region access is required Manual failover needed Uses 2x disk space Manual failover still has downtime, usually measured in minutes DNS dependency
SOFS on Storage Spaces Direct Highly-available Multi-node and Multi-disk HA Scale up or scale out SMB 3.0 and 3.1 Transparent failover during planned and unplanned maintenance activities Recommended for user profile storage within Azure Uses 2-3x disk space 3rd party backup software support can be limited by the vendor Does not support multi-region deployment
Distributed File System – Replication Proven technology for file-based replication Supports PowerShell Domain-based Cannot be deployed in an active-active configuration
Third-party storage applications Deduplication technologies Better use of storage space Additional cost Proprietary management tools

The recommended file server virtual machine types are generally DS1, DS2, DS3, DS4, or DS5, with the appropriate selection depending on customer use requirements. For best performance, ensure premium disk support is selected. Additional guidance can be found on Microsoft Azure documentation.

Infrastructure Cost Management

Two technologies are available that can be used to reduce the costs of the Citrix environment in Azure, reserved instances and Citrix Smart Scale.

Reserved Instances

Azure Reserved VM Instances (RIs) significantly reduce costs—up to 72 percent compared to pay-as-you-go prices—with one-year or three-year terms on Windows and Linux virtual machines (VMs). When customers combine the cost savings gained from Azure RIs with the added value of the Azure Hybrid Benefit, they can save up to 80 percent. The 80% is calculated based on a three-year Azure Reserved Instance commitment of a Windows Server when compared to the normal pay-as-you-go rate.

While Azure Reserved Instances require making upfront commitments on compute capacity, they also provide flexibility to exchange or cancel reserved instances at any time. A reservation only covers the virtual machine compute costs; it does not reduce any of the additional software, networking, or storage charges. This is good for Citrix infrastructure and minimum capacity needed for a use case (on and off hours).

Citrix Smart Scale

The Smart Scale feature in Smart Tools enables proactive scaling and power management of registered machines in a Virtual Apps and Desktops Site, based on a schedule you define or the level of demand for user sessions. Smart Scale is configured on a per-delivery group basis. Customers can use Smart Scale with Delivery Groups that reference only Machine Catalogues containing power-managed machines. These machines can be Server OS machines or Desktop OS machines.

To use Smart Scale with a Site, customers must install the Smart Tools Agent on one or more Delivery Controllers or Cloud Connectors within the Virtual Apps and Desktops Site. When managing VDI desktop machines, disable the Virtual Apps and Desktops’ built-in power management functions so they don’t interfere with Smart Scale’s scaling actions.

Machine Type Schedule-based Load-based Load and schedule-based
Server OS machines hosting published applications or hosted shared desktops (Server VDI) Supported Supported Supported
Desktop OS machines hosting static persistent (dedicated) VDI desktops Supported During periods when machines are powered off (for example, after working hours), users can trigger machines to power on through Citrix Receiver. You can set Smart Scale’s Power Off Delay so Smart Tools does not automatically power machines off before the user can establish a session. Not currently supported Not currently supported
Desktop OS - machines hosting - random non-persistent VDI desktops (pooled VDI desktops) Supported Supported Use the Session Count scaling metric and set the maximum number of sessions to 1. Supported Use the Session Count scaling metric and set the minimum number of machines to 1.

Smart Scale supports power managing Sites using Virtual Apps and Desktops, the Virtual Apps and Desktops service, Virtual Apps Essentials, and Virtual Desktops Essentials.

Power management for a single Site using one of these services is supported as follows:

  • Up to 2,000 VDAs or VDIs per Site can be power managed.
  • Up to 120 Delivery Groups can be power managed.
  • Up to 1,000 VDAs or VDIs per Delivery Group can be power managed.

Azure-RA-Image-4

Diagram-4: Citrix SmartScale Flow

Optimizing End-User Experience

Optimizing the end-user experience includes balancing the end-user’s perception of responsiveness with the business needs of staying within a budget. This section discusses the design concepts and decisions around providing an environment that is correctly sized for the business and the end-user.

Defining User Workspace

When planning for a user assessment, the Citrix VDI Handbook and Best Practices documentation provides invaluable guidance. Review the following high-level questions to better understand existing use cases and the resources needed for their end users.

Topic Question
Number of Users How many users are expected within the environment? Did the assessment phase determine the appropriate VDI Model? (Virtual Apps or Virtual Desktops)
Use Cases What types of applications will be consumed by the end-users? What are the VDA requirements for the applications? How will the applications be delivered best? (Virtual Apps vs Virtual Desktops)
User Group working hours When will users be accessing the environment? What are the peak hours? What is the expected consumption throughout the day? (The consumption of users during specific hours helps identify workspace requirements for scale automation and Azure reserved Instance purchasing.)
Location Where are end users located? Should workspaces be deployed across multiple regions or only in a single region?
User and Application Data Where is the user and application data stored? Will data be contained solely in Azure, only on-premises, or a mix of both? What is the maximum tolerable latency for accessing the user data?

Azure VM Instance Types

Each Citrix component will leverage an associated virtual machine type in Azure. Each VM series available is mapped to a specific category of workloads (general purpose, compute-optimized, etc.) with various sizes controlling the resources allocated to the VM (CPU, Memory, IOPS, network, etc.).

Most Citrix deployments leverage the D-Series and F-Series instance types. The D-Series are commonly used for the Citrix infrastructure components and sometimes for the user workloads when they require additional memory beyond what is found in the F-Series instance types. F-Series instance types are the most common in the field for user workloads because of their faster processors which bring with them the perception of responsiveness.

Why D-Series or F-Series? From a Citrix perspective, most infrastructure components (Cloud Connectors, StoreFront, NetScaler, etc.) leverage CPU to run core processes. These VM types have a balanced CPU to Memory ratio, are hosted on uniform hardware (unlike A-Series) for more consistent performance and support premium storage. Certainly, customers can and should adjust their instance types to meet their needs and their budget.

The size and number of components within customer’s infrastructure will always depends on customer’s requirements, scale, and workloads. However, with Azure we have the ability to scale dynamically and on-demand! For cost-conscious customers, starting smaller and scaling up is the best approach. Azure VMs require a reboot when changing size so plan these events within scheduled maintenance windows only and under established change control policies.

How about Scale-up or Scale-out?

The following high-level questions should be reviewed to better understand customer’s use case and the resources needed for their end users. This will also help them to plan their workload well in advance.

Scaling up is best when the cost per user per hour needs to be the lowest and a larger impact can be tolerated should the instance fail. Scaling out is preferred when the impact of a single instance failure needs to be minimized. The table below provides some example instance types for different Citrix components.

Component Recommended Instance Type
Delivery Controllers Cloud Connectors Standard DS2_v2 or DS2_v3 with Premium SSD storage
Scale Up Server OS User Workloads Standard_F16s_v2 VMs with Virtual App were identified to have the lowest $/user/hr cost compared to other instances. Standard_DS5_v2 VMs were also cost competitive compared to other instances
Scale Out Server OS User Workloads Standard_F4_v2 and Standard_F8_v2 instances support a lower user count however provide more flexibility of power management operations due to smaller user container sizes. This allows machines to be more effectively deallocated to save costs on Pay-as-You-Go instances. Additionally, the failure domains are smaller when scaling out.
Desktop OS User Workloads Standard_F2_v2 has the lowest dual-core cost and performs well with Windows 10.

The latest instance type study was done to provide great insight in this area and we highly recommend the read. In all cases, customers should evaluate the instance types with their workloads.

When it comes to graphics-intensive workloads, consider the NV_v2-series virtual machines, which are powered by NVIDIA Tesla M60 GPUs and NVIDIA GRID technology with Intel Broadwell CPUs. These virtual machines are targeted for GPU accelerated graphics applications and virtual desktops where customers want to visualize their data, simulate results to view, work on CAD, or render and stream content. Additionally, these virtual machines can run single precision workloads such as encoding and rendering. NV_v2 virtual machines support Premium Storage and come with twice the system memory (RAM) when compared with its predecessor NV-series.

Storage

Just like any other computer, virtual machines in Azure use disks as a place to store an operating system, applications, and data. All Azure virtual machines have at least two disks – a Windows operating system disk and a temporary disk. The operating system disk is created from an image, and both the operating system disk and the image are stored within Azure as virtual hard disks (VHDs). Virtual machines may also have additional disks attached as data disks, also stored as VHDs.

Azure Disks are designed to deliver enterprise-grade durability. Three performance tiers for storage exist that can be selected when creating disks: Premium SSD Disks, Standard SSD, and Standard HDD Storage, and the disks may be either managed or unmanaged. Managed disks are the default and are not subject to the storage account limitations like the unmanaged disks.

Managed Disks are recommended over unmanaged disk by Microsoft. Unmanaged disks should be considered by exception only. Standard Storage (HDD and SSD) include transaction costs (storage I/O) that must be considered but have lower costs per disk. Premium Storage has no transaction costs but have higher per disk costs and offers an improved user experience.

The disks offer no SLA unless an Availability Set is used. Availability Sets are not supported with Citrix MCS but should be included with Citrix Cloud Connector, NetScaler, and Storefront.

Microsoft Operations Management Suite (OMS)

OMS Workspaces are best defined by customer, workload type, business groups, or department. For example Server VDAs, Client VDAs, StoreFront servers would all have their own OMS Workspace. To simplify management, create as few workspaces as possible. Two key management technologies are available within Microsoft’s Operations Management Suite.

Management Solutions

Management solutions typically collect information into Log Analytics and provide log searches and views to analyze collected data. They may also leverage other services such as Azure Automation to perform actions related to the application or service.

A Solution is a set of services, configurations, event IDs, capacity, etc. that can be captured and monitored for a given workload. Multiple Solutions can be applied to an OMS Workspace. Determine what data points are critical to the customer for the workload within scope of the OMS Workspace. Solutions are pre-built or custom modules available to OMS and assigned to a given Workspace.

Workspaces Log Analytics

Log Analytics workspace, is a unique Log Analytics environment with its own data repository, data sources, and solutions. The purpose being to manage and access the log data that comes from Azure, looking for anomalies in the log data.

Identity

The section focuses on Identity controls, workspace user planning, and the end-user experience. The primary design consideration is managing identities within both Azure and Citrix Cloud tenants.

Microsoft Azure Active Directory (Azure AD) is an identity and access management cloud solution that provides directory services, identity governance, and application access management. A single Azure AD directory is automatically associated with an Azure subscription when it is created.

Every Azure subscription has a trust relationship with an Azure AD directory to authenticate users, services, and devices. Multiple subscriptions can trust the same Azure AD directory, but a subscription will only trust a single Azure AD directory.

Microsoft’s identity solutions span on-premises and cloud-based capabilities, creating a single user identity for authentication and authorization to all resources, regardless of location. This concept is known as Hybrid Identity. There are different design and configuration options for hybrid identity using Microsoft solutions, and in some case, it might be difficult to determine which combination will best meet the needs of an organization.

Common Identity Design Considerations

In most cases extending the customers Active Directory Site to Azure and will utilize the use of Active directory replication to provide identity and authentication with the Citrix workspaces. A common step is to use AD Connect to replicate user to Azure Active directory which provides you with the Subscription based activation required for Windows 10.

It is recommended to extend local Active Directory Domain Services to the Azure Virtual Network Subnet for full features and extensibility. Azure Role-Based Access Control (RBAC) helps provide fine-grained access management for Azure resources. Too many permissions can expose and account to attackers. Too few permissions means that employees can’t get their work done efficiently. Using RBAC, administrator can give employees the exact permissions they need.

Authentication

Domain Services (either AD DS or Azure AD DS) are required for core Citrix functionality. RBAC is an authorization system built on Azure Resource Manager that provides fine-grained access management of resources in Azure. RBAC allows you to granularly control the level of access that users have. For example, you can limit a user to only manage virtual networks and another user to manage all resources in a resource group. Azure includes several built-in roles that you can use.

Azure AD Authentication is supported for the Workspace Experience Service and Citrix ADC/StoreFront authentication. For full SSON with Azure AD, Citrix Federated Authentication Service (FAS) or Azure AD DS (for core Domain Services) must be used.

Citrix FAS is not yet supported for the Workspace Experience Service therefore StoreFront is required as part of the deployment architecture. Azure MFA server (Cloud Service, Azure MFA Server, Azure MFA NPS Extension) can enable the usage of Azure MFA without requiring a SAML policy and the use of Citrix FAS for full SSON. However this is an IaaS VM and must be managed by the customer.

Active Directory and Azure Active Directory Outcomes

  • Azure Active Directory Provisioned Tenant
  • List of desired Organizational roles for Azure RBAC with mapping to Built-In or Custom Azure Roles
  • List of desired Admin access levels (Account, Subscription, Resource Group etc.)
  • Procedure to grant access/role to new user(s) for Azure
  • Procedure to assign JIT (just in time) elevation for users for specific tasks

Below is an example architecture of namespace layout and authentication flow.

Azure-RA-Image-5

Diagram-5: Architecture of namespace layout and authentication flow

Citrix Cloud Administration + Azure AD

By default, Citrix Cloud uses the Citrix Identity provider to manage the identity information for all users who access the Citrix Cloud. Customers can change this to use Azure Active Directory (AD) instead. By using Azure AD with Citrix Cloud, Customers can:

  • Leverage their own Active Directory, so they can control auditing, password policies, and easily disable accounts when needed.
  • Configure multi-factor authentication for a higher level of security against the possibility of stolen sign-in credentials.
  • Use a branded sign-in page, so your users know they’re signing in at the right place.
  • Use federation to an identity provider of your choice including ADFS, Okta, and Ping, among others.

Citrix Cloud includes an Azure AD app that allows Citrix Cloud to connect with Azure AD without the need for you to be logged in to an active Azure AD session. Citrix Cloud Administrator Login allows Azure AD identities to be used in the customers Citrix Cloud tenant.

  • Determine if Citrix Cloud administrators will use their Citrix Identity or Azure AD to access Citrix Cloud URL will follow the format “https://citrix.cloud.com/go/{Customer Determined}”
  • Identify the Authentication URL for Azure AD authentication into Citrix Cloud

Governance

Azure Governance is a collection of concepts and services that are designed to enable management of your various Azure resources at scale. These services provide the ability to organize and structure your subscriptions in a logical way, to create, deploy and re-usable Azure native packages of resources. This subject is focused on establishing the policies, processes, and procedures associated with the planning, architecture, acquisition, deployment, operation and management of Azure resources.

Citrix Cloud Administrator Login

Determine if Citrix Cloud administrators will use their Citrix Identity, Active Directory Identity, or Azure AD to access Citrix Cloud. Azure AD integration enables multi-factor authentication into Citrix Cloud for administrators. Identify the Authentication URL for Azure AD authentication into Citrix Cloud. URL will follow the format https://citrix.cloud.com/go/{Customer Determined}

RBAC permissions & delegation

Using Azure AD customers can implement their governance policies using Role Based Access Control (RBAC) of Azure resources. One of the primary tools for application of these permissions is the concept of a Resource Group. Think of a Resource Group as a bundle of Azure resources that share the same lifecycle and administrative ownership.

In the context of a Citrix environment these should be organized in a way that will allow for proper delegation between teams and promote the concept of least privilege. A good example is when a Citrix Cloud deployment uses a Citrix ADC VPX provisioned from the Azure marketplace for external access. Although a core piece of Citrix infrastructure, the Citrix ADCs could have a separate update cycle, set of admins, etc. This would call for separating the Citrix ADCs from the other Citrix components into separate Resource Groups so the Azure RBAC permissions can be applied through the administrative zones of tenant, subscription, and resources.

MCS Service Principal

To access resources that are secured by an Azure AD tenant, the entity that requires access must be represented by a security principal. This is true for both users (user principal) and applications (service principal). The security principal defines the access policy and permissions for the user/application in the Azure AD tenant. This enables core features such as authentication of the user/application during sign-in, and authorization during resource access.

Determine the permissions allocated to the Service Principal used by the Citrix MCS service.

Subscription scope service principals have Contributor rights to the applicable subscription used by the Citrix environment. Narrow Scope service principals have granular RBAC applied to the Resource Groups containing the network, master images, and VDAs. Narrow Scope Service Principals are recommended to limit the permissions only to the permissions required by the service. This adheres to the security concept of “least privilege”.

Tagging

Customer apply tags to their Azure resources giving metadata to logically organize them into a taxonomy. Each tag consists of a name and a value pair. For example, they can apply the name “Environment” and the value “Production” to all the resources in production.

Customer can retrieve all the resources in your subscription with that tag name and value. Tags enable them to retrieve related resources from different resource groups. This approach is helpful when admin need to organize resources for billing or management.

There is a limit of 15 tags per Resource. Citrix MCS creates 2 tags per VM therefore a customer is limited to 13 tags for MCS machines. MCS non-persistent machines are deleted during reboot. This will remove Azure VM specific characteristics such as tags, boot diagnostics, etc. If tags are required it is recommended to create an Azure Append policy and apply it to the applicable MCS Resource Groups.

Azure Policy

Azure policies can control aspects such as tagging, permitted SKUs, encryption, Azure region, and naming convention. There are default policies available as well as the capability to enforce custom policies. Azure policies can be applied at the subscription or Resource Group level. Multiple policies can be defined. Policies applied at the Resource Group level take precedence over Subscription Level policy.

Identify aspects of Azure that should be controlled and standardized across the Citrix environment. Hard quota will force the policy and not permit exceptions. Soft quota will audit for policy enforcement and notify if the policy is not met. Refer Azure documentation for more detailed information to define the policies.

Azure-RA-Image-6

Diagram-6: Azure Governance Access Policy and RBAC

Security

Security is integrated into every aspect of Azure. Azure offers unique security advantages derived from global security intelligence, sophisticated customer-facing controls, and a secure hardened infrastructure. This powerful combination helps protect applications and data, support compliance efforts, and provide cost-effective security for organizations of all sizes.

IAAS - Azure Security Center Monitoring

Azure Security Center analyzes the security state of Azure resources. When Security Center identifies potential security vulnerabilities, it creates recommendations that guide customer through the process of configuring the needed controls. Recommendations apply to Azure resource types: virtual machines (VMs) and computers, applications, networking, SQL, and Identity and Access. There are few best practices that has to followed:

  • Control VM access and Secure privileged access.
  • Provisioning antimalware to help identify and remove malicious software.
  • Integrate your antimalware solution with Security Center to monitor the status of your protection.
  • Keep your VMs current & ensure at deployment that images you built include the most recent round of Windows and security updates.
  • Periodically redeploy your VMs to force a fresh version of the OS.
  • Configuring network security groups and rules to control traffic to virtual machines.
  • Provisioning web application firewalls to help defend against attacks that target your web applications.
  • Addressing OS configurations that do not match the recommended baselines.

Network Design

Network security could be defined as the process of protecting resources from unauthorized access or attack by applying controls to network traffic. The goal is to ensure that only legitimate traffic is allowed. Azure includes a robust networking infrastructure to support your application and service connectivity requirements. Network connectivity is possible between resources located in Azure, between on-premises and Azure hosted resources, and to and from the internet and Azure.

Virtual Network (VNet) Segmentation

Azure virtual networks are similar to a LAN on your on-premises network. The idea behind an Azure virtual network is that you create a single private IP address space–based network on which customers can place all their Azure virtual machines. The best practice is to segment the larger address space into subnets and create network access controls between subnets. Routing between subnets happens automatically, and you don’t need to manually configure routing tables.

Use a Network Security Group (NSG). NSGs are simple, stateful packet inspection devices that use the 5-tuple (the source IP, source port, destination IP, destination port, and layer 4 protocol) approach to create allow/deny rules for network traffic. This allow or deny traffic to and from a single IP address, to and from multiple IP addresses, or to and from entire subnets.

Customers can create custom, or user-defined, routes called User-defined Routes (UDRs) in Azure to override Azure’s default system routes, or to add additional routes to a subnet’s route table. In Azure, admins can create a route table, then associate the route table to zero or more virtual network subnets. Each subnet can have zero or one route table associated to it.

NSGs and UDRs are applied at the subnet-level within a Virtual Network. When designing a Citrix Virtual Network in Azure it is recommended to design the virtual network with this in mind, creating subnets for similar components, allowing for the granular application of NSGs and UDRs as needed. An example of this would be segmenting Citrix infrastructure into its own subnet, with a corresponding subnet for each use case.

Identify the ports and protocols required for Citrix and the supporting technologies. Review to verify these ports are allowed within the Network Security Groups used in the environment. Network Security Groups can limit inbound and outbound communications to a defined set of IP, Virtual Networks, Service Tags, or Application Security Groups.

Azure-RA-Image-7

Diagram-7: Azure Security Center and Network Security using NSG and ASG

Connectivity

Connecting Azure virtual networks with customers local / cloud network is referred to as hybrid networking. This section explains the options for network connectivity and network service routing. Customers can connect their on-premises computers and networks to a virtual network using any combination of the following options:

  • Point-to-site virtual private network (VPN): Established between a virtual network and a single computer in customer network. Each computer that wants to establish connectivity with a virtual network must configure its connection. This connection type is great for just getting started with Azure, or for developers, because it requires little or no changes to customer existing network. The communication between your computer and a virtual network is sent through an encrypted tunnel over the internet.
  • Site-to-site VPN: Established between on-premises VPN device and an Azure VPN Gateway that is deployed in a virtual network. This connection type enables any on-premises resource that customer authorize to access a virtual network. The communication between on-premises VPN device and an Azure VPN gateway is sent through an encrypted tunnel over the internet.
  • Azure ExpressRoute: Established between customer’s network and Azure, through an ExpressRoute partner. This connection is private. Traffic does not go over the internet.

The primary considerations for Azure to Customer connectivity are bandwidth, latency, security, and cost. Site to Site VPNs have lower bandwidth limits than Express Route and are dependent on the performance of the edge router used by the customer. SLAs are available on the VPN Gateway SKUs. Site to Site VPNs use IPSEC over the internet.

Express Routes are dedicated private connections and not over the internet. This results in lower latency when using Express Route. Additionally, Express Route can scale up to 10 Gbps. Express Route is configured using a certified partner. Configuration time by these providers should be considered during project planning. Express Route costs have a Microsoft component and a Express Route provider component.

Typically these connections are shared across multiple services (database replication, domain traffic, application traffic, etc.) In a hybrid cloud deployment there may be scenarios where internal users will require their ICA traffic to go through this connection to get to their Citrix apps in Azure, therefore monitoring its bandwidth is critical.

With NetScaler and traditional StoreFront optimal gateway routing may also be used to direct a user’s connection to a NetScaler using an office’s ISP rather than the Express Route or VPN to Azure.

User Defined Routes (UDRs)

Typically customers will use a UDR to route Azure traffic to a firewall appliance within Azure or a specific virtual network. For example, North/South traffic from a VDA to the internet. If large amounts of traffic are routed to 3rd party firewall appliances within Azure this can create a resource bottleneck or availability risk if these appliances are not sized or configured appropriately. NSGs can be used to supplement third-party firewalls and should be utilized as much as possible where appropriate. Consider Azure Network Watcher if traffic introspection is required.

Virtual network peering

Virtual network peering enables to seamlessly connect two Azure virtual networks. Once peered, the virtual networks appear as one, for connectivity purposes. The traffic between virtual machines in the peered virtual networks is routed through the Microsoft backbone infrastructure, much like traffic is routed between virtual machines in the same virtual network, through private IP addresses only.

Azure supports:

  • VNet peering - connecting VNets within the same Azure region
  • Global VNet peering - connecting VNets across Azure regions

Customers deploying workloads on multiple VNets should consider to use the VNet peering to enable the communication between VMs between VNets.

Azure-RA-Image-8

Diagram-8: Datacenter Connectivity and Routes

SD-WAN

Software-defined WAN (SD-WAN) technology makes it possible to deliver a great user experience, even over challenging connections. It’s a natural fit for virtual apps and desktops delivery.

  • Aggregate all available bandwidth into an active/active connection, providing more bandwidth.
  • Use the unique HDX Quality of Experience technology to optimize performance and tune network policies.
  • Ensure always-on connections for virtual apps and desktop users with the highest possible-quality experience—even for rich media and high-definition video.

Customers using VPN might leverage SD-WAN to add redundancy to the Azure and Customer Datacenter connectivity or to provide application specific routing. Citrix SD-WAN automatically redirecting traffic across any available connections. In fact, the experience is so seamless, users won’t even realize any change has occurred. Their primary access IP address will remain unchanged, allowing users to access their apps and data using the same methods and devices.

Citrix ADC

Citrix ADC on Microsoft Azure ensures organizations have access to secure and optimized applications and assets deployed in the cloud and provides the flexibility to establish a networking foundation that adjusts to the changing needs of an environment. In the event of a data center failure, Citrix ADC automatically redirects user traffic to a secondary site, with no interruptions for users. Load balancing and global server load balancing across several data centers further ensures optimum server health, capacities, and utilization.

Discuss with customer and define the following use case for each Resource Location:

Access Method Considerations
Internal only A Citrix ADC is not required if only internal access is needed.
External access via Citrix ADC Gateway Service. The Citrix Cloud ADC Gateway Service provides ICA proxy (secure remote connectivity only).
External access via Citrix ADC VPX deployed in Azure Resource Location A customer will need to consider a Citrix ADC VPX appliance in Azure if they require the following: 1. Multi-factor authentication with full SSON 2. Endpoint scanning 3. Advanced authentication or pre-authentication policies 4. Citrix Smart Access policies Note: These requirements will prompt the need for authentication to occur at the Citrix ADC rather than the Workspace Experience service. StoreFront is required if authentication is managed by a Citrix ADC Gateway vServer.

Citrix ADC - Deployment Model

Active-Active deployments leverage standalone Citrix ADC nodes that can be scaled out using the Azure Load Balancer. Active-Passive pairs facilitate stateful failover of ICA traffic in the event of a node failure however they are limited to the capacity of a single VPX. Active-Passive nodes also require Azure Load Balancer.

Citrix ADC is limited to 500 Mbps per Azure NIC. Multiple NICs are recommended to isolate the SNIP, NSIP, and VIP traffic to maximize the throughput available for Citrix ADC Gateway or other services.

References

Operations

Identity

Governance

Security

Azure Monitor

Connectivity

Contributors

Author: Loay Shbeilat