Reference Architecture: Citrix DaaS - AWS

Audience and Objective

This document is intended to help Citrix partners and customers understand the most critical design decisions necessary to successfully deploy Citrix virtualization technologies on Amazon's public cloud. It is not meant to be a "How-To" reference - Citrix considers those Deployment Guides, and they are now delivered and maintained separately from Reference Architectures. In this document, we use the Citrix Architectural Design Framework to organize and present the leading practices, recommendations, and design patterns which are used by Citrix and select Citrix consulting partners.

Overview and Executive Summary

Citrix virtualization and networking technologies have been successfully serving the needs of enterprises large and small for nearly three decades. There are many ways in which these technologies can be licensed, deployed, integrated, and managed. This flexibility allows Citrix technologies to serve various different use cases, business types, integration requirements, and operational models. This paper is written for the Citrix customer or partner who's considering or planning to deploy these technologies on AWS' public cloud infrastructure.

For both existing customers looking to modernize their infrastructure and new customers looking to deploy Citrix virtualization and networking technologies, there are several key high and low level decisions which must be made along the way to help facilitate a successful deployment. To help customers and partners understand these decision points, we have introduced and defined some specific terminology to set the appropriate context, then used this context to highlight the critical decisions and implications to consider as you plan your deployment.

We start by defining two primary technology adoption models: Customer Managed and Cloud Services. We then break the components of a Citrix virtualization system down into multiple subsystems, and categorize them by adoption model:

We take a stand for cloud services, recommending that most organizations use or plan to use cloud services where feasible. We don't offer this stand blindly - while we do believe cloud services, in the end, offer overwhelmingly positive benefits for our customers, we recognize that not all organizations/deployments are able to use cloud services for all subsystems today. Sometimes, use case requirements (with technical attributes or shortcomings in a currently available/specific cloud service) need adopting a combination of cloud services and a customer managed component: we focus on these in this paper. In other cases, non-technical items (politics, budgetary/contractual considerations, cloud readiness deficiencies, regulatory/compliance considerations, and such) may discourage the usage of cloud services. In these instances, we recommend working with your Citrix partner/sales/engineering team to help overcome them. Through the rest of this paper, we go to great lengths to identify key capabilities, features, or attributes that help customers decide which adoption model to use for which subsystem and when.

Next, we define and examine three common deployment scenarios, highlighting which adoption model is used for each subsystem:

• Greenfield/Cloud only - uses cloud services for all Citrix virtualization system subsystems, plus AWS public cloud services.
• Hybrid (not to be confused with a 'hybrid cloud') - the most common deployment model, the hybrid model uses DaaS for session brokering and administration, with both customer managed and cloud service options for the remaining subsystems.
• Lift and Shift - as the name states, this model uses existing, customer managed CVAD, StoreFront, and ADC/Gateway and either migrates these components to AWS as is, or installs them into AWS as part of a workload migration to AWS public cloud services. While this is a valid deployment model for certain specific use cases, it comes with substantial caveats.

Finally, we use the well documented Citrix Architectural Design Framework to organize and present the key design decisions to be considered when deploying Citrix virtualization technology on AWS. We keep our focus on "what's different about Citrix on AWS" for clarity, providing links to other resources for more detailed information as needed.

We ultimately recommend that most customers use the Hybrid deployment model from day one, using the CVAD service for session brokering and administration. This provides the customer with the key capabilities necessary to cost-effectively run a Citrix virtualization system on AWS, substantially reduces the cost and complexity, provides access to the latest features and capabilities available, and simplifies the migration to and usage of other cloud services in the future. Either cloud services OR customer managed components can be used for the remaining subsystems (depending upon the customers' specific needs), though we recommend customers are clear as to why they're using customer managed components and have a plan to move to cloud services in the future once the cloud services meet their specific needs.

For more insights into leading practices for Citrix on AWS, readers can reference the following Cloud Guidepost articles:

• Leading practices for Citrix Cloud on AWS - Part 1
• Leading practices for Citrix Cloud on AWS - Part 2
• Leading practices for Citrix Cloud on AWS - Part 3

Key Concepts and Deployment Scenarios

In this section, we describe some key concepts and deployment scenarios before we dive into specific considerations for each layer of the Citrix Architectural Design Framework.

Technology Adoption Models

Citrix DaaS technology can be 'consumed' or implemented many different ways. The most common methods can be described generally as Customer Managed and Cloud Services. A third model is also worth mentioning - Partner Managed. We describe this model here for clarity, but from an architectural design perspective, the first two are the most relevant.

Why are we discussing technology adoption models in a reference architecture? The choice of adoption or 'consumption' model has a substantial impact upon the system design, capabilities, cost, and delineation of management responsibilities. This section will define and explore these models, and provide some general guidance to help customers make informed choices as they design a Citrix virtualization system.

Customer Managed

For many years, businesses wanting to consume technology had no choice but to buy, install, configure, and maintain the entire technology stack required to build a Citrix virtualization system. Citrix's virtualization software was only available as installable binaries. Customers who bought Citrix's virtualization software would download these binaries (often in the form of a downloadable ISO disk image or self-extracting executables) then install, configure, and maintain the software themselves. The Citrix software (and networking hardware) was most commonly installed into/on infrastructure the customer owned and maintained, in data centers they also owned and maintained.

Conceptually speaking, a Citrix virtualization system is made up of various different Citrix components, many of which we describe in detail in this paper. They also require different layers of third-party components which must be provided before you can do anything with the Citrix bits. Ultimately, they all come together to create a functional Citrix virtualization system. For the sake of clarity, this document refers to this technology adoption model as "Customer Managed". We use this term to describe various different components in a Citrix virtualization system, including the requisite third party components in the layers underneath, next to, and 'above' the Citrix components. This model can also be called "Self-Managed."

Today, customers have compelling alternatives to a customer managed adoption model, yet some still adopt elements of their technology stack using this model for various reasons. While this model provides customers with the most control over each component, it comes at a cost: the customer takes on the responsibility to manage and maintain the component, including securing, operating, patching, upgrading, and maintaining high availability. This model is also commonly deployed for 'air gapped' systems (those without any Internet access, and hence are limited in their ability to use cloud services, which are commonly and securely accessed over public networks).

Here's an example of the architecture of a Citrix virtualization system that's using 100% customer managed components deployed on AWS using basic AWS IaaS services such as Elastic Compute Cloud (EC2) and Virtual Private Cloud (VPC) networking. We are discussing some of the details of this architecture in later sections of this document, though you can immediately notice the similarities to the much simpler greenfield/cloud only deployment model:

Cloud Services

Over the last 15 years, technological advancements across many different fields have given rise to hyper scale public clouds, sophisticated cloud services, microservice architectures, DevOPS/Agile delivery frameworks, subscription licensing models, and 'evergreen' software and systems. These advancements have revolutionized the way technology is acquired, adopted, delivered, and maintained across almost every industry in the world.

Today, many of the components or layers that comprise a Citrix virtualization system are available "as a Service." In contrast to the Customer Managed adoption model (where customers buy technology as a corporate asset and build/maintain systems themselves), customers "subscribe" to various services, and the service provider takes on the responsibility for delivering and managing these services. These services are often consumed over public networks (that is, the Internet) leading some to call this "Cloud Service" or "Web Service" adoption model. In this paper we're going to refer to this type of adoption model as "Cloud Managed Services," or simply the "Cloud Service" model.

Citrix offers many of its traditional products 'as a Service', using its platform partners' latest technological advancements to simplify and streamline adoption, accelerate the pace of innovation, improve quality, and deliver more incremental value to their customers over time. Citrix calls this service delivery platform "Citrix Cloud," and it represents the current and future state of the art from Citrix.

Here's an example of the architecture of a system that's using 100% cloud service components for a Citrix virtualization system on AWS. We are discussing the details of this design in a later section of this document:

Partner Managed

While many organizations choose to build their own Citrix virtualization system, some customers seek to get out of the business of managing IT so they can focus resources and attention on serving their own customers and markets. To serve these customers, Citrix works with integration partners who use Citrix technologies to provide a 'finished goods' service to their customers.

Defining and differentiating the different integration partners/types and offerings available are outside of the scope of this document. However, Citrix partners face the same choices customers face when designing a Citrix virtualization system. The Citrix partner can choose to use one or more services from Citrix Cloud, or they can choose to build and manage some components of the system for their customers' specific needs. As such, the guidance provided in this document is relevant to both the Citrix partner and their customers, just for different reasons.

Citrix Virtualization System Components

To understand the implications of the design decisions we detail later in this document, we're going to put the components of a Citrix virtualization system into higher level 'buckets' we'll then use to guide your decision-making process. Every Citrix virtualization system, regardless of how it is deployed and licensed, needs these components available to function and provide the best, most secure user experience. It is not uncommon for customers to mix and match self-managed components and cloud services, especially if they've got complex use case requirements, third party integration requirements, or extreme control or availability needs.

The following table highlights these key components for clarity. Details and recommendations on when/where you'd use one option vs another is covered later in this document:

Leading Practices and Recommendations

Whether you manage the Citrix virtualization system yourself or you use Citrix or an authorized partner to do it, consider using cloud services wherever possible. For use cases/environments where the cloud service doesn't meet your needs, customer managed components can be used. That said - Citrix encourages customers to be clear on why they are deploying self-managed components, and be prepared to migrate to cloud services once the cloud service meets their specific needs. The cloud services provided by Citrix through Citrix Cloud are evolving rapidly. Over time you can expect them to provide all the functionality required to serve all but the most complex use cases. Citrix Cloud services ultimately minimize the amount of infrastructure the customer is responsible for managing and maintaining. Citrix Cloud also provides highly available, pre-integrated services, and ensures customers always have access to the latest, most secure, and feature-rich services.

Common Deployment Models for Citrix Virtualization on AWS

As a cloud provider with the most functionality, largest community of customers, unmatched experience, and maturity, AWS sees a wide range of customers from various industries moving systems and infrastructure to their clouds. Over time they've seen common deployment scenarios/migration patterns develop. In this section, we explore these patterns/scenarios, discuss when and where you may want to consider using them to bring a Citrix Virtual Apps and Desktops workload to AWS, and provide some recommendations for which patterns to consider for common migration scenarios.

The three most common scenarios for delivering Citrix Apps and Desktops on AWS are:

• Greenfield/Cloud Only deployment, using Citrix Cloud services with "resource locations" on Amazon EC2 (Amazon Elastic Compute Cloud) service. This scenario is commonly used when customers prefer to go to a subscription model and outsource control plane infrastructure and management responsibility to Citrix, or they're looking to experience/evaluate the capabilities provided by Citrix Cloud services.
• Hybrid deployment/workload migration to AWS, using Citrix Cloud services for session brokering and administration, Workspace UI or StoreFront for content aggregation/session presentation/session launching, and can also use customer managed Citrix ADC/Gateways for HDX session proxying, complex authentication scenarios, or both.
• Lift and shift. With this scenario, customers essentially move or redeploy their self-managed Citrix infrastructure into AWS, treating the deployment on AWS just like their existing customer managed deployment. With this scenario, customers use Citrix ADC/Gateway and Citrix StoreFront to aggregate resources from on-premises and AWS hosted sites. This facilitates the migration of workloads to AWS, though customers can keep their on-premises workloads around and simply add another site in AWS. The new site can be used for new workloads or to support disaster recovery (DR) and failover use cases. This model is characterized by the use of customer managed components for session brokering and administration, UI services, authentication, and HDX session proxy.

This section defines these scenarios in more detail, including architectural overviews of how each scenario is commonly designed. You find that the leading practice is to use Citrix Cloud services, and as such this document will focus on the Citrix Cloud brokered deployment models ("Greenfield" and "Hybrid").

Greenfield Deployment

The most common example of the green field deployment model is trial or proof of concept deployments of Citrix virtualization technology on the AWS cloud. Since you're essentially starting from scratch, the power of 'infrastructure as code' can be experienced since you're not trying to integrate with a bunch of existing 'stuff'. It is also a fantastic opportunity to 'play with' various cloud services and evaluate their suitability to your or your customers' needs.

A green field deployment is also the quickest and easiest Citrix virtualization system you can build. You can simply tear it down when the system is no longer needed, and you stop paying for the resources it consumed. All you need for this type of deployment is an active AWS account and either a trial or paid subscription to Citrix Cloud and Citrix DaaS. Armed with these two resources, you can use AWS' QuickStart CloudFormation templates to build a reference deployment. Citrix and AWS have collaborated on the Citrix DaaS on AWS quick start template, which helps you to either build an entire Citrix virtualization system from scratch, or create a Citrix Cloud "Resource Location" in an existing VPC with an existing Active Directory. When deploying the entire Citrix virtualization system from scratch, the resulting system on AWS is built closely matching the following reference architecture diagrams:

It is worth noting that this deployment model (actually, all three deployment models) use AWS Availability Zones to provide a highly available design. See Availability Zones later in this document for more context.

As mentioned previously, this is a great place to start when learning about AWS and Citrix Cloud services. Many of the design patterns depicted in the preceding diagram are used for hybrid and even lift and shift deployment types, so learning these design patterns suits a Citrix on AWS architect well, regardless of the deployment model.

To summarize, the green field deployment model uses all cloud services, at least as a starting point:

We mentioned earlier that the green field deployment model is often used as a starting point for proof of concept and technology trial systems. If you start with this model and then drop in StoreFront or Citrix ADC/Gateway VPXs in, you're ostensibly creating our next type of deployment model: hybrid.

Hybrid Deployment

With the hybrid deployment model, customers may choose to install/configure/manage some of the Citrix virtualization system components themselves, but not the session brokering and administration subsystem. In a hybrid deployment model, this subsystem is provided as a cloud service called "Citrix DaaS", and it is delivered as a subscription from Citrix Cloud.

The hybrid deployment model is the most common deployment seen today, and is the model Citrix recommends for most customers. Here are some of the primary reasons why we take this position:

• Simplicity - With Citrix Cloud services, simplicity is a foundational design tenet. When multiple cloud services are used, they come pre-configured to work together, and when configuration is necessary, workflows and options are dramatically simplified.
• Infrastructure and licensing cost savings - customer managed Citrix virtualization services often require extra hardware and software to support them, and these have costs associated with them. One good example is Microsoft SQL Server: customer managed brokering and administration services require databases, and if you're going to build/manage your own, you must provide them. An alternative is using the AWS Relational Database Service (Amazon RDS) for SQL Server.
• Autoscaling - Citrix's managed brokering service (DaaS) includes the Citrix Autoscale feature, which provides built-in VDA capacity and cost management functionality. This feature can save customers a substantial amount of money on infrastructure when they're only paying for what they use. When running a Citrix virtualization workload on AWS, this often means the difference between paying for committed use discounts or paying for VM usage as you go. The cost savings can be dramatic for many use cases, and the Citrix Autoscale feature helps ensure you're only consuming what you need.
• Management savings - With cloud services, Citrix shoulders the responsibility for keeping the services highly available, performant, secure, and up to date. You still build and manage your VDAs regardless, but don't underestimate the value of delegating these responsibilities. Cloud services help free up IT resources, allowing them to focus on providing unique value to their businesses instead of these critical but tedious (and often time-consuming) tasks.
• "Free" upgrades and continuous innovation - with customer managed infrastructure, the onus is on the customer to upgrade and patch the components in their care. With Cloud services, most of those work efforts go away. The service providers (Citrix or AWS for example) tend to be constantly innovating, and they bring those innovations to the customers who consume the services, often without requiring any work on behalf of the customer.
• Access to more features, functionality, and services - modern service delivery platforms (such as Citrix Cloud and AWS EC2) give technology providers a powerful, cost-effective way to bring new features, capabilities, and services to market that would not otherwise be possible. Vendors such as Citrix are committed to meeting the customer wherever they are at in their digital transformation journey, but sometimes the only way to cost-effectively deliver new capabilities is to deliver them as a cloud service.
• Flexibility - with DaaS as the foundation of this deployment model, customers can mix and match customer managed or cloud service components of the Citrix virtualization system. This allows the system to meet various different use cases and support complex enterprise requirements for a Citrix virtualization system. We explore these choices in depth in a later section of this paper.

To summarize, the hybrid deployment model uses the following:

Given the options a customer can choose in the hybrid deployment model, and the flexibility provided by customer managed components, there isn't one, succinct architecture that fits all customers. There are, however, some common design patterns that can also be mixed/matched to suit the customers' needs. The foundational pattern, however, is the pattern for a Citrix Cloud "Resource Location" on AWS. It is also the pattern built by the Citrix DaaS on AWS QuickStart template, and it looks similar to the following architectural diagram:

It is worth noting that this deployment model also uses AWS Availability Zones to provide a highly available design. See Availability Zones later in this document for more context.

It is also worth noting that the hybrid deployment model (a DaaS resource location on AWS) can be combined with a hybrid cloud model, connecting customer managed data centers/resources to AWS using AWS Direct Connect, AWS VPN, or other networking tools. With this model, the customers' existing Active Directory is often extended into AWS, and customers create more Citrix Cloud 'Resource Locations' which deliver apps, desktops, and resources from the customer managed data center. The resulting conceptual architecture looks something like the following diagram:

Lift and Shift

Referring to our definition of the Citrix virtualization system components, when we're talking about a lift and shift deployment scenario, the key component is the session brokering and administration subsystem and associated infrastructure. If you're using self-managed brokering infrastructure (you're deploying Delivery Controllers instead of Cloud Connectors) then for the purposes of this paper you're lifting and shifting.

Lift and Shift - why

Despite Citrix guidance against this model, some customers still choose to go with this model and deploy/manage the Citrix virtualization system components themselves. Per CTX270373, the use of public clouds including AWS is only supported with LTSR product versions. For customers who do choose the lift and shift (self-managed) deployment model, we often find that non-technical reasons are behind it. Politics, time pressures, fear of the unknown, perceived skills deficits, loss of control, and license acquisition fall into this category. There are, however, a few technical reasons why this model is appealing. These include:

• System Isolation - some use cases, such as air-gapped systems with no Internet access, often make the lift and shift model appealing. Since cloud services require outbound Internet access to function, in a strictly air-gapped deployment, cloud services won't function. This mainly applies to the Cloud Connectors (primary component of managed session brokering services) as they need outbound Internet connectivity to communicate with and utilize the Citrix Cloud services. Some customers can consider utilizing a secure, outbound proxy for Cloud Connectors (while keeping all other infrastructure strictly air-gapped). This is often a suitable concession which allows the managed brokering services to be utilized, but even this may not be an option for some customers and use cases.
• Configuration flexibility - one person's 'complex' is another person's 'flexible', and flexibility has been a strong suite of customer managed Citrix virtualization infrastructure for more than two decades. Over the years the technology has gained a ton of features that support some very niche use cases and third-party integrations. The Citrix Cloud services focus on simplicity and pre-integration. In doing so, some of these niche features and integrations are not available. Therefore, some edge cases are still best served by a customer managed stack. That said, given the rapid pace of innovation coming to the Citrix Cloud services, these edge cases are becoming increasingly rare.
• Control - some organizations, cultures, and business models demand as much control as possible. With customer managed Citrix virtualization components, customers can completely own their destiny. This control comes at a cost (infrastructure, complexity, personnel, and such) but "control at all cost" is a thing for some customers.

To summarize, the lift and shift deployment model uses the following:

In its simplest form, a lift and shift deployment of Citrix virtualization technology onto AWS resembles a traditional customer managed deployment on-premises. It uses a CVAD 'site' deployed into an AWS region and uses basic AWS IaaS services such as EC2 virtual machines and VPC networking at a minimum. As mentioned previously, it requires the customer to build/configure/maintain all system components, plus supporting services such as SQL databases. The following diagram depicts this deployment model:

It is worth noting that this deployment model also uses AWS Availability Zones to provide a highly available design. See Availability Zones later in this document for more context.

A lift and shift deployment model is often combined with a hybrid cloud infrastructure model, using AWS Direct Connect, AWS VPN, or similar networking technology to connect a customer managed data center and resources to AWS. Customers can optionally adopt some of AWS' more advanced cloud services (to provide a measure of simplification with the transition), and they may also choose to host some services (such as SQL databases, Citrix licensing, Citrix StoreFront, and Citrix ADC/Gateway) either on AWS, in a customer managed data center, or both depending upon their existing investments, use case requirements, and such. A conceptual architecture of this deployment model (using AWS RDS for SQL Server or on-premises SQL server) is shown in the following diagram. Only one active instance of Citrix Licensing is needed, but we've shown multiple to depict available options:

Lift and Shift - why not

By now you've gathered that the Citrix leading practice/recommendation is to NOT do a full lift and shift. You can be wondering why, or where this is coming from. Referring to our breakdown of Citrix virtualization system components, the session brokering and administration subsystem is the most critical component you want to consider NOT lifting and shifting. We strongly recommend customers consider using Citrix's cloud services for session brokering and administration (deploy Cloud Connectors only, vs. deploying Delivery Controllers + SQL databases + Director servers + Citrix Licensing servers). Here are some of the primary reasons why we take this position (and they might sound familiar):

• Simplicity - While customer-managed session brokering services provide the ultimate in control and configuration flexibility, it comes at the cost of complexity and ongoing maintenance requirements. With Citrix Cloud services, simplicity is a foundational design tenet. When multiple cloud services are used, they come pre-configured to work together, and when configuration is necessary, workflows and options are dramatically simplified.
• Infrastructure and licensing cost savings - customer managed Citrix virtualization services often require extra hardware and software to support them, and these have costs associated with them. One good example is Microsoft SQL Server: customer managed brokering services require databases, and if you're going to build/manage your own, you must provide them.
• Speaking of infrastructure cost savings - this brings up a critical differentiator between the two session brokering options: Autoscaling. Citrix's managed brokering service (DaaS) includes the Citrix Autoscale feature, which provides built-in VDA capacity and cost management functionality. This feature can save customers a substantial amount of money on infrastructure when they're only paying for what they use. When running a Citrix virtualization workload on AWS, this often means the difference between paying for committed use discounts or paying for VM usage as you go. The cost savings can be dramatic for many use cases, and the Citrix Autoscale feature helps ensure you're only consuming what you need. Important note: This feature is only available to Citrix Cloud service (Citrix DaaS) customers - it is not available to customer managed brokering infrastructure (Citrix Virtual Apps and Desktops LTSR or CR releases). - Management savings - With cloud services, Citrix (and AWS in this case) shoulders the responsibility for keeping the services highly available, performant, secure, and up to date. You still build and manage your VDAs regardless, but don't underestimate the value of delegating these responsibilities. Cloud services help free up IT resources, allowing them to focus on providing unique value to their businesses instead of these critical but tedious and often time-consuming tasks.
• "Free" upgrades and continuous innovation - with customer managed infrastructure, the onus is on the customer to upgrade and patch the components in their care. With cloud services, most of those work efforts go away. The service providers (Citrix and AWS in this case) tend to be constantly innovating, and they bring these innovations to the customers who consume the services, often without requiring any work on behalf of the customer.
• Access to more features, functionality, and services - modern service delivery platforms (such as Citrix Cloud and Amazon EC2) give technology providers a powerful, cost-effective way to bring new features, capabilities, and services to market that wouldn't otherwise be possible. Vendors such as Citrix are committed to meeting the customer wherever they're at in their digital transformation journey, but sometimes the only way to cost-effectively deliver new capabilities is to deliver them as a cloud service.

Lift and Shift - more resources

Before Citrix Cloud services were born, customers were already successfully deploying Citrix virtualization technologies on AWS. In those days Citrix called the Virtual Apps and Desktops products XenApp and XenDesktop. Extensive work went into creating and publishing reference architectures and deployment guides for this deployment scenario. A good portion of the detail in these aging resources still applies to deployments who must go down this road today.

For customers who MUST go down this route, the following published resources provide you with useful background detail you can use to help you be successful. We recommend reviewing these materials before you continue on with this document, as we are highlighting important design decisions that have changed since these works were completed:

• Using AWS Directory Service and Amazon RDS with Citrix Virtual Apps and Desktops (blog)
• Deploying Citrix Virtual Apps and Desktop with AWS Directory Service and Amazon RDS Version 1.0 (deployment guide)

Design Decisions

This section explores key design decisions to consider as you're architecting your Citrix virtualization system on AWS. We walk down through each layer of the Citrix Architectural Design Framework, exploring key areas for you to consider.

About the Citrix Architectural Design Framework

Citrix's Virtual Apps and Desktops solution (the product family name collectively referring to Citrix's virtualization technologies) enables organizations to create, control and manage virtual machines, deliver applications and desktops, and implement granular security policies. The Citrix Virtual Apps and Desktops solution provides a unified framework for developing a complete digital workspace offering. This offering enables Citrix users to access applications and desktops independent of their device's operating system and interface.

The Citrix architectural design framework is based on a unified and standardized layer model. It provides a consistent and easily accessible framework for understanding the technical architecture for most of the common Virtual Apps and Desktops deployment scenarios. These layers are depicted in the following conceptual diagram:

• User Layer - This layer defines user groups and locations of the Citrix environment.
• Access layer - This layer defines how users access the resources.
• Control layer - This layer defines the components that control the Citrix solution.
• Resource layer - This layer defines provisioning of Citrix workloads and how resources are assigned to the given users.
• Platform layer - This layer defines the physical elements where the hypervisor components and cloud service provider framework run to host the Citrix workloads.
• Operations Layer - This layer defines the tools that support the delivery of the core solutions.

User Layer Considerations

In the Citrix Architectural Design Framework, the User layer describes the user groups, their locations, specific requirements, and more. The user layer appropriately sets the overall direction for each user group's environment. This layer incorporates the assessment criteria for business priorities and user group requirements to define effective strategies for endpoints and Citrix Workspace App. These design decisions affect the flexibility and functionality for each user group.

When designing and deploying a Citrix virtualization system on any platform, the decisions and strategies adopted after careful assessment set the foundation for many other decisions that customers ought to consider as they work their way down through the other layers in the Citrix Architectural Design Framework. As such, this is a critical layer to understand thoroughly and get right.

Access Layer Considerations

In the Citrix Architectural Design Framework, the Access layer defines how users access AWS resources. The design of your access layer is critical to the functionality delivered by any Citrix virtualization system. It controls how users authenticate to the system. It also controls how users view and launch virtualized applications and desktops, plus what type of applications and content are available to them. Also, the Access layer controls how and when sessions are securely proxied or directly connected.

In the context of the Citrix virtualization system components we defined earlier, the access layer contains the following components and choices:

The following table contains critical decision points when determining which access layer component to deploy, but the choice is not binary. Citrix supports various different access methods that can be customized to suit your needs.

UI Service and Authentication Considerations

Consider the following when choosing how you want to provide UI services for your Citrix virtualization system on AWS:

HDX Session Proxy Considerations

Consider the following when choosing how you want to provide HDX session proxy functionality for your Citrix virtualization system on AWS:

Summary, Recommendations, and Leading Practices

Now that we've reviewed some of the attributes/features/capabilities that help drive your customer managed vs. cloud service decisions for the Access Layer subsystems, let's examine the top level decisions in the context of the deployment models we defined earlier.

Access Layer: Greenfield/Cloud Only Deployment

Since the green field or cloud only deployment model use cloud services across the board, the AWS specific implications on the design of your Citrix virtualization system are simple: there aren't any. It's not necessary build or configure anything on AWS since everything required for both UI and HDX proxy services is provided for you, configured and ready to go 'out of the box'.

The Access Layer of a Citrix deployment is a key requirement for delivering virtual apps and desktops to users. If an access point is unreachable or fails, users cannot access their resources. Network design and implementation can be complicated, but with Citrix Gateway Service and Citrix Workspace, redundancy, failover, maintenance, and global presence are all part of the package - with no networking knowledge required.Using the Citrix Gateway Service and Citrix Workspace can reduce your infrastructure footprint substantially. By moving the access layer to a cloud services model, users can securely access network resources from anywhere in the world. This approach requires the least deployment and maintenance efforts, so it is a great option if you want to get up-and-running quickly, have a limited IT staff, or if infrastructure is not your focus. With everything pre-configured, this deployment model is the least customizable, but for deploying a simple, secure, fully functional, globally accessible system, using Citrix Workspace and Gateway Service for your access layer is the way to go.

Access Layer: Hybrid Deployment

With the hybrid deployment model, you're going to be building/managing some of the Citrix virtualization system components, otherwise it is a green field or cloud only deployment by definition. With the hybrid model, you're possibly deploying Citrix ADC/Gateway VPXs on AWS or even on-premises, and depending upon your requirements, you might also be deploying Citrix StoreFront on AWS or on-premises. Customers who have made significant investments in their on-premises Gateway and identity solutions can benefit from the ability to use Citrix Gateway as the identity provider for Workspace.

This deployment model is common for security-focused deployments, deployments with current on-prem infrastructure (ADC or StoreFront), and for DR/failover sites for existing customer managed data centers. One of the key considerations for this model is keeping your users, resources, and access points as close together as possible. Choose AWS regions near the on-prem resource locations in which to deploy your Access Layer. Where possible, keep your ADCs and StoreFront servers as close as possible to each other. This is where things can get tricky. Consider the Citrix Virtual Apps and Desktops launch sequence when designing your hybrid deployment, noting especially that all traffic is routed through the Citrix ADC.

With Citrix ADC/Gateway and StoreFront as EC2-based instances in AWS, there is also much more potential for customization. In addition to the multiple StoreFront stores, multifactor authentication, and various industry-leading ADC features, hybrid deployments can also use native AWS services such as the Relational Database Service (RDS) and AWS Directory Services. Hybrid deployments lend well to a more gradual cloud transition and leave room for adjustments to the architecture along the way, as opposed to lift, and shift methods.

The hybrid approach does require a higher level of expertise and increased lead time to deploy than the greenfield/cloud only model, but can serve as a solid transition state between a traditional customer managed/on-prem deployment and cloud only state.

Access Layer: Lift and Shift Deployment

With the legacy lift and shift deployment model, you're deploying both Citrix ADC/Gateway VPXs and Citrix StoreFront on AWS, or potentially reusing existing on-premises deployments of these technologies for the same purpose. This type of deployment tends to have the least lead time for customers with existing on-prem Citrix virtualization environments, and is also the easiest transition from an Operations and Maintenance perspective. Staff with experience managing an on-prem environment has a shorter ramp-up time with the lift and shift deployment model, as the Citrix infrastructure remains largely unchanged. For the access layer specifically, this method is straightforward and allows for many customizations. The lift and shift is a great first step for existing deployments going into the cloud or for new or air-gapped AWS regions, but may be a hindrance to adopting a cloud-forward architecture in the future.

Citrix ADC/Gateway VPX on AWS

Deploying the Citrix ADC/Gateway on AWS is different than deploying it on-premises, though in the end you're managing them yourself. Fortunately deploying Citrix ADC/Gateway on AWS is thoroughly documented, so we recommend reviewing the following resources before you solidify your design and begin implementation:

• Citrix ADC VPX on AWS in Citrix Docs: Provides a comprehensive overview of Citrix ADC on AWS, including supported VPX models, AWS regions, EC2 instance types, and extra resource references.
• Citrix ADC and Amazon Web Services Validated Reference Design in Citrix Docs/Advanced Concepts - includes more details and deployment guidance.

While there are potential variants for a Citrix ADC/Gateway VPX architecture on AWS, the following diagram (from the Citrix ADC for Web Applications Quick Start Deployment Guide) depicts a multi-AZ Citrix HA pair deployment as deployed by the Quick Start template (with default subnets/CIDR blocks):

As discussed in Citrix ADC VPX on AWS on Citrix Docs, there are two primary deployment options available. They are:

• Standalone: Individual instances of Citrix ADC/Gateway can be deployed and managed as separate entities. This is commonly used for smaller scale or POC deployments where high availability is not a requirement.

• High Availability: This is the most commonly deployed model for production environments: pairs of Citrix ADC/Gateway VPX instances can be deployed using native Citrix HA mode on AWS. With older firmware versions, the pair is deployed in the same AWS Availability Zone. Starting with Citrix ADC 12.1 firmware, highly available pairs of VPX appliances can be deployed across Availability Zones (AZ). How high availability on AWS works explains the difference between deploying a pair of ADCs within the same AZ and across AZs. We dig into this option more deeply later in this section.

While Citrix ADC VPX generally supports single, dual, or multiple NIC deployment types, Citrix recommends using at least three subnets for each ADC when deployed on AWS, with a network interface in each subnet for optimum throughput and data separation. When deployed to support Citrix Virtual Apps and Desktops, the NSIP is typically attached to the "Private Citrix Infrastructure Subnet," the SNIP is attached to the "Private Citrix VDA Subnet," and the Citrix Gateway VIP to the "Public Subnet." The following simplified conceptual diagram depicts this configuration. It shows a single VPX instance in a single AZ - this design pattern would be duplicated (likely in a second AZ) for a High Availability configuration:

ADC High Availability across Availability Zones

As mentioned earlier, this is the most common deployment model for Citrix virtualization systems. This model uses a pair of Citrix ADC VPXs deployed across Availability Zones by either using Citrix ADC's native HA (active/passive) feature or a combination of Citrix ADC's native Global Service Load Balancing (GSLB) and IPSet features. The latter option (which became feasible in early 2020) allows for an active/active configuration across AZs, and functions by allowing the ADC to act as an authoritative DNS source. This new option/architecture is expected to be popular for public cloud deployments, so we focus on that here.

The Domain Based Services for cloud load balancers allow for AutoDiscovery of dynamic cloud services. By deploying Citrix ADCs across multiple AZs in an active-active configuration, you can use cloud resources in different AZs to optimize High Availability/Disaster Recovery. Each AZ can contain cloud resources in the familiar Pod Infrastructure, to allow for easily managed updates, patching, and scalability for expansion. For detailed information about setting up GSLB between AWS AZs, see Citrix Documentation.

In the preceding diagram, we can see that each ADC has a different Gateway virtual IP (VIP). This is characteristic of an Independent Network Configuration (INC). When VPXs in an HA pair reside in different Availability Zones, the secondary ADC must have an INC, as they cannot share mapped IP addresses, virtual LANs, or network routes. The NSIP is different for each ADC in this configuration, while SNIPs and Load Balancing VIPs utilize a special Citrix ADC feature called IPset, or Multi-IP virtual servers, which can be used for clients in different subnets to connect to the same set of servers. With IPset, you can associate a private IP to each of the primary and secondary instances. A public IP can then be mapped to the primary ADC in the pair. In the case of failover, the public IP mapping changes dynamically to the new primary. For GSLB deployments in AWS, the service IP can be part of the IPset for both IPv4 and IPv6 traffic.

For more information on adding a remote node to an ADC to create an INC-based HA pair, see Citrix docs.

Citrix StoreFront on AWS

Deploying Citrix StoreFront on AWS is not much different than deploying it on-premises, and in the end, you're also managing all the components of StoreFront yourself too. See Plan your StoreFront deployment for general considerations which apply to all deployments including StoreFront on AWS. The main difference is that you typically deploy multiple StoreFront instances in a StoreFront server group across multiple AWS availability zones. It is important to note that the features enabled with this design are dependent upon latency between AZs. Per Plan your StoreFront Deployment/Scalability, StoreFront server group deployments are only supported where links between servers in a server group have latency of less than 40 ms (with subscriptions disabled) or less than 3 ms (with subscriptions enabled). Make sure you measure latencies between instances in all AZs you plan to host StoreFront and enable/disable subscriptions accordingly.

We already called this out in the UI Service and Authentication Considerations table earlier in this document, but it is worth calling out again: for Citrix DaaS environments with extensive resiliency requirements, Citrix strongly recommends a StoreFront implementation to fully benefit from the Local Host Cache feature (available in both CVAD and DaaS session brokering infrastructure types). For CVAD, this provides resiliency if there is a database outage. For DaaS, this architecture provides resiliency in case Cloud Connectors cannot reach Citrix Cloud. In either case, disconnected users will still be able to connect to new and existing sessions during an outage scenario. For more details, limitations, and implications of Local Host cache activation, see Local Host Cache (DaaS) and Local Host Cache (CVAD).

While we're on the topic of resilience, Citrix also recommends that your StoreFront implementation span multiple AZs (if the AWS region includes multiple AZs), but remember to take the ADC design into account. Citrix ADC is often used in front of StoreFront instances to provide load balancing and extra service resiliency.

By utilizing Citrix Zones, StoreFront redundancy can be built in by spreading satellite zones across two or more AZs in a VPC with a single site. Using Zones is a great way to have resources as close to the users as possible and highly available. Satellite Zones contain StoreFront servers, Delivery Controllers, and app/desktop resources, leaving the Primary Zone with the full infrastructure setup, including the license server and SQL. This allows for scalability of StoreFront web UI and Zone creation/destruction can be orchestrated. Keeping the Zones smaller will allow for optimal east-west scalability and reduce the impact in the case of an outage.

StoreFront on AWS is fully customizable, including Featured App Groups, splash page, coloring and logo, and apps and desktops can be arranged in the best way for your specific needs. StoreFront on AWS also requires knowledgeable administration and engineering to be kept up, but can provide a powerful web UI, especially when integrated with the Citrix ADC.

Resource Layer Considerations

The Resource Layer design focuses on personalization, applications, and image design. The Resource Layer is where users interact with desktops and applications. When deploying a Citrix virtualization system on AWS, the key things to keep in mind (aside from all the 'normal' stuff we won't cover here) are:

• CIFS storage and data replication - Regardless of the tooling you use for managing user personalization settings (the users' Windows profile and redirected folders) you've got to have Windows file shares to store them on. If you've got VDAs in multiple regions (and users can access apps/desktops in more than one) then you've also got to deal with data replication. Many applications also use Windows file shares, so CIFS storage and data replication are important for these too.
• Image design - Citrix App Layering and Citrix Provisioning Services (PVS) do not currently support Amazon EC2 - customers hosting a resource location in AWS use Machine Creation Services for VDA fleet creation, management, and updating.

CIFS Storage and Data Replication

Most Citrix virtualization systems on AWS require at least basic access to a Windows compatible file share to persist user settings, user data, and application data. When these shares are not available, the user experience and application functionality suffer, so it is important to ensure that whatever solution you choose to provide Windows compatible file shares is highly available and data is regularly backed up.

For multi-site deployments, reliable and performant data replication may also be necessary to meet availability, RPO, and RTO needs. This is especially true for environments where users may connect to desktops/apps in 2 or more regions, and application data/user settings must be available in the region where the apps/desktops run. The following section describes some solutions to consider for providing CIFS storage and data replication services on AWS.

While non-Windows solutions for providing Windows file shares exist, most of these solutions cannot deliver the indexing capabilities required for search functionality inside a Windows desktop or applications such as Microsoft Outlook running on Windows. As such, most customers turn to Windows-based file server solutions, at least for storing user profiles and persistent application data. Fortunately, both customer managed and cloud service options are available for use when Citrix virtualization systems are run on AWS.

Customer Managed: Windows File Servers on Amazon EC2

The first solution many customers consider for providing Windows compatible file services on AWS is building their own Windows file servers on EC2 to serve each resource location on AWS. Since Windows file servers are needed by various different types of applications and workloads, many IT shops may gravitate towards building and managing their own since this is something they know how to do. At the most basic level, the customer spins up one or more Windows EC2 instances, attach extra Amazon Elastic Block Store (EBS) volume, join the instance's to their Active Directory, and get busy configuring and setting up Windows File Services.

This option, as you might imagine, provides customers with the most control and flexibility. While this is very appealing to certain types of customers and certain verticals, it also comes at a cost: the responsibility to size, scale, build, manage, patch, secure, and maintain everything from the Windows OS up. Customers electing to go this route ought to also ensure these file servers are highly available. This is often accomplished using file servers in multiple Availability Zones, and using Windows DFS-N/DFS-R, though it's easy to end up in an unsupported configuration (per Microsoft) if you're not careful.

Note: Customers considering this option ought to review Microsoft's support statement regarding using DFS-R and DFS-N for roaming profile shares and folder redirection shares. One more point to consider since the Citrix virtualization system will be running on AWS: a new deployment or migration event may provide an excellent opportunity to evaluate using a cloud service for Windows file services instead of building your own. Fortunately, Amazon has some cloud service options worth considering. We touch on some of these now.

Cloud Service: Amazon FSx for Windows File Server

Amazon's FSx for Windows File Server is a cloud service which customers can consume on AWS. FSx for Windows File Server provides a fully managed, native Windows file system, and SSD-based storage with consistent submillisecond performance. Since FSx is built on Windows Server, it delivers a fully native, Windows compatible file system that provides storage and protection for Citrix virtualization systems on AWS. FSx for Windows File Server is also Citrix Ready Verified, meaning this AWS supported solution has been validated by Citrix to be compatible with Citrix Virtual Apps and Desktops. While it is not officially supported by Citrix, the service IS fundamentally native Microsoft Windows file server - it is just managed by AWS instead of the customer. For more information, see Amazon FSX for Windows File Server on Citrix Ready.

For IT teams, this is an excellent option that removes many of the more mundane or low-value tasks around deploying and managing storage. Most importantly, using FSx offloads security, data protection/backup, compliance, software updating/patching tasks, and the monitoring of storage infrastructure to make sure it meets required service levels. IT teams can treat the entire FSx file service as a single operational platform instead of managing a Windows operating system file server, storage, networking, and such. Also, FSx supports all the common management tools it already uses, such as Active Directory (AD) integration, Windows DFS Namespaces, DFS Replication, and others.

Each FSx managed file system you create essentially becomes a highly available and durable file server in a specific Availability Zone. For servicing a Citrix virtualization system, customers ought to ensure these "file systems" are highly available. This can be accomplished by provisioning FSx managed file systems in multiple availability zones, and using Windows DFS-N/DFS-R to create highly available Windows file shares, though it's easy to end up in an unsupported configuration (per Microsoft) if you're not careful.

Note: Since FSx is a Windows file server, customers considering this option ought to review Microsoft's support statement regarding using DFS-R and DFS-N for roaming profile shares and folder redirection shares.

More Cloud Service Options

Besides Amazon's first party managed Windows file service, AWS supports many more expansive and feature rich options, some of which integrate with traditional on-premises storage technologies. While these other options are outside the scope of this document, there are many options to choose from. A good place to start exploring options is on the AWS Marketplace. These types of solutions can be especially relevant for more complex, multi-region use cases where reliable, and resilient data replication is needed.

CIFS Storage and Data Replication: Summary and Conclusions

Customers can manage their own highly available DFS file share, benefit from this as an AWS service (FSx) to save on management effort, or use third party storage appliance solutions to extend on an on-premises environment. Citrix recommends that customers analyze the pros and cons of each to determine a solution that is right for them.

Image Design and Management

In a Citrix virtualization system on AWS, applications and desktops are delivered via EC2 instances called "VDAs" (named after Citrix's Virtual Delivery Agent software, which is installed into Windows or Linux instances containing the applications being delivered by the Citrix virtualization system). A group of identical VDAs are provisioned and maintained in "Machine Catalogs," a management construct defined and maintained through the session brokering and management subsystem (both DaaS and CVAD). The creation, sizing, and management of these instances is key, as many systems have large numbers of VDAs and the software stack in a VDA changes frequently as hotfixes, service packs, and software updates are applied. We discuss some of the higher-level considerations in this section.

VDA Provisioning and Image Management

On AWS EC2, Citrix virtualization systems use Citrix's Machine Creation Services (MCS) provisioning technology for VDA deployment and image management. MCS utilizes an IAM service account on EC2 to orchestrate the mastering process (turning a snapshot of a template VM's system disk into a generalized AMI), the cloning process (creating and managing a fleet of VDA instances based on the AMI created from the snapshot of the template VM), autoscaling Delivery Groups, updating deployed images, and more. We discuss MCS on AWS in much more detail in the Control Layer Considerations sections of this document.

Note: customers already using MCS for their on-premises environments can notice some differences between the options available to them when provisioning machines in AWS. MCS managed VDA instances on EC2 have two disks attached: the system disk (a read/write copy of the template image AMI created during the mastering process) and a 1GB personality disk. Depending on the machine catalog type and hosting connection options configured, the system disk (and sometimes the VM instance) will be deleted at shutdown and recreated at 'power on' (for pooled or shared catalogs) or they are retained (for persistent catalog types). See CTX234562 for more information.

Delivery and Persistence Models

Choosing the right delivery models is critical and has broad implications beyond just cost. Citrix virtualization technology supports three main delivery models, which can be mixed and matched and used in combination to support many different use cases. The three delivery models are:

• Hosted shared: The hosted shared model most commonly utilizes a Windows Server OS with the RDSH role installed, though Linux instances can provide the same functionality for compatible apps. With this model, a single VDA instance can support multiple simultaneous users, each running either a full desktop or connecting to one or more published applications. When using Windows Server OS/RDSH with the Desktop Experience and related components installed, desktops and apps look and feel like they're running on a Windows desktop OS. Since every user on a given instance shares OS instance, administrators typically pre-install and configure the mix of applications on hosted shared instances, and users do not have local administrator rights to the OS. Hosted shared instances can also run on shared infrastructure, and can be consumed using both on-demand and reserved instance pricing models. Administrators usually deploy a fleet of instances to support the hosted shared model, and both customer managed and cloud service types of Citrix brokering subsystems provide sophisticated load balancing capabilities to ensure every user experiences adequate performance. Hosted shared instances can also use GPU backed instance types on AWS to increase performance for graphically intensive workloads that can benefit from a GPU, though the GPU vendor can require extra licenses. Both Windows Server OS and RDS CAL licenses can be 'rented' under Microsoft's SPLA licensing model, though customers can avoid these additional costs by using Linux as the OS. This model is, hands down, the most cost effective to run on AWS.
• Server VDI: The "Server VDI" (Virtual Desktop Infrastructure) model also uses a Windows Server OS, and with the Desktop Experience and related components installed, it looks and feels to the user just like a Windows Desktop OS. The RDSH role is not installed with this model, so one instance supports one user at a time, and users are sometimes provided with elevated rights to the Server OS so they can install their own applications. Like hosted shared instances, server VDI instances can also run on shared infrastructure, can be consumed using both on-demand and reserved instance pricing models, can use GPU backed instance types, and Microsoft OS and RDS CALs can be 'rented' under Microsoft's SPLA licensing model. Given the tools available today, 99+% of Windows applications can be installed and ran on the Windows Server OS, and though sometimes software vendors don't explicitly support their applications on Windows Server, most Windows apps run as well on Windows Server as they do on a Windows Desktop OS. It's also worth noting that server VDI instances can also use GPU backed instance types on AWS to increase performance for graphically intensive workloads that can benefit from a GPU, though the GPU vendor may require another licenses. This is the second most cost-effective delivery model to run on AWS.
• Client VDI: The client VDI delivery model typically uses a Windows desktop OS such as Windows 10 or Windows 7, although a supported Linux OS version can be used as well. Client VDI is a 1:1 model, meaning each unique user requires their own OS instance. Customers who are new to Citrix virtualization technology often come into these types of projects asking for client VDI, even though more cost-effective models are available. Their vernacular can also have been influenced by other virtualization vendors whose technology stacks don't support hosted shared or server VDI deployment models. The client VDI model, while looking 'simpler' on the surface, gets much more complicated the deeper you get into it, though most of the complexity can be avoided by using Linux as the OS. Most of this complication is driven by Microsoft's licensing requirements for the Windows Desktop OS which, unlike Windows Server, is not available via Microsoft's SPLA licensing program. As such, customers must bring their own licensing for these products. Also - Windows desktop based client VDI instances cannot run on shared infrastructure. This means that client VDI instances must run in either AWS dedicated instances or on AWS dedicated hosts. This substantially increases the cost and complexity of managing the infrastructure required, reduces flexibility and cost control options, and gets expensive quickly. As you might expect, client VDI instances can also use GPU backed instance types on AWS to increase performance for graphically intensive workloads that can benefit from a GPU, though the GPU vendor can require more licenses. Client VDI is the most expensive delivery model to run on AWS. For both VDI models, another important consideration is the persistence model. VDI instances can be randomly assigned to users with no persistence (pooled) or users can have assigned machines that persist and are personalized (dedicated). Pooled instances can be easier to manage over time since all instances in a given pool are identical. Citrix's MCS can update the system disks attached to pooled instances with a few clicks, and capacity/cost management is more effective since an idle pool of instances can serve many users. Pooled instances are a bit less flexible than dedicated since end-user changes to pooled instances don't usually persist between reboots, though technologies such as Citrix App Layering's User Layer or Personalization Layer released in CVAD 1912 can be used to minimize the impact on the user experience. Dedicated instances can also be tougher to manage from a cost perspective too - since it is often tough to predict when a user will log on, the user must either wait while their instance is started, or administrators must keep them running during time windows where each user is expected to log on.

While we've mentioned it previously, we're going to mention it again here for clarity: various flavors of Linux can be used in a Citrix virtualization system, as long as one or more applications run on Linux. Citrix's virtualization technology supports both hosted shared and VDI delivery models, persistent and pooled models, and GPU backed instance types. The user and administrator experiences are different than with Windows based instances, but Linux based VDAs are often much less expensive to run since they don't require Microsoft licenses.

Finally, let's revisit the consideration of GPU acceleration. All three delivery models (for both Linux and Windows) can use NVIDIA accelerated GPU instances on AWS. G-series instances can be used for graphics accelerated use cases, but are not yet commercially viable for general purpose usage. Note that Citrix doesn't support the AWS Elastic GPU today, but since Elastic GPU only works for OpenGL, its impact on typical graphics workloads in the enterprise is minimal.

So - which delivery models do you use? It is worth noting that you can mix and match delivery models in the same system to meet the needs of different user groups or use cases. The most cost-effective delivery model from an infrastructure perspective is hosted-shared. The combination of server OS with multi-user concurrency is highly efficient, and the number of users per virtual machine can be sized accordingly depending on user type (for example - task worker vs. knowledge worker vs. power user) to ensure a great experience. For situations where users and apps require extra capabilities that aren't satisfied by hosted-shared, VDI is the way to go. Server VDI ought to be evaluated first: it is substantially more cost-effective to run than Windows 10 VDI for Windows workloads, and Server VDI can deliver a desktop that looks and feels similar to Windows 10. Also, Server VDI doesn't have the Microsoft EULA requirement to use dedicated instances/hosts - Client VDI (deploying Windows 10 or sometimes Windows 7) does. For Windows based workloads on AWS, Client VDI ought to be considered as a last resort, and deployed only when hosted shared and Server VDI delivery models are not possible.

To help with the decision making process, the following decision tree compares

AWS Instance Billing Model

Once you've decided which delivery model to use (hosted-shared, server VDI, or client VDI), the next step is to plan for an hourly on-demand billing model or a reserved billing model. Ideally, as many VDAs as possible are to be paid for by the hour with the on-demand billing model, and use the Citrix Autoscale feature to control costs. By using Citrix Autoscale (a feature exclusive to the DaaS cloud service brokering subsystem) VMs are powered on as needed with anticipation for peak hours. During off peak hours, however, VMs are shut down, so it's important to consolidate loads with the hosted-shared model and for all models ensure that users save their work and ideally log off gracefully from their sessions. Reserved instance capacity can be used for infrastructure components like the Cloud Connectors (which remain on 24/7) and a predetermined number of VDAs that will always remain on (for example, 10% of peak). Besides providing significant discount compared to On-Demand pricing, Reserve Instances also provide a capacity reservation when used in a specific Availability Zone.

VDA Instance Sizing and Cost Management

When running a fleet of VDAs on AWS, choosing the right instance type for your different workloads (VDAs) is a key decision, with substantial performance, manageability, and cost considerations. Choose too small of an instance and performance can suffer. Choose too large of an instance, and you're paying for resources you're not using. Choosing the right instance type ends up being a balancing act, and often requires fine-tuning for each specific workload.

Which AWS EC2 instance type to choose for your VDAs depends heavily upon the specific workload and delivery type. However, as a general guideline, "M" series instances are often most suitable for hosted-shared whereas "T" series instances are suitable for VDI. "M" series has balanced CPU and RAM designed for the mostly predictable resource consumption across multiple sessions on a host. "T" series are "burstable" in nature designed for the mostly unpredictable characteristics of VDI (for example - one minute a user is idling and the next they are running a macro calculation). For extra details on instance type selection and pricing, readers can refer to the Citrix on AWS cost estimation presentation (in sources section).

For more information regarding instance selection (especially as it applies to the hosted shared delivery model) sees Citrix Scalability in a Cloud World – 2018 Edition. This article, while slightly dated, discusses leading practices regarding instance selection based on performance, manageability, cost, reserved vs. on demand pricing models, and LoginVSI scalability testing. These concepts and considerations are still valid today, even though instance choices and pricing have likely changed since its initial publication.

Note: Some newer AWS instance types will not show up by default in the Machine Catalog creation wizard in Studio (either CVAD or DaaS). The UI is populated with instance types from a static XML file which resides on Delivery Controllers (CVAD) or Cloud Connectors (DaaS). This XML can be modified to include newer instance types, but this file is overwritten with default values during upgrades (both Citrix initiated Cloud Connector updates or customer-initiated Delivery Controller upgrades). See CTX139707 for more details on how to update the list of available AWS instance types. During this round of testing (a point in time reference) the M5.2Xlarge instance type (8vCPU, 32 GB RAM) turned out to be the winner in terms of $/user/hour (with an industry standard sample workload). Your numbers - given your specific workload characteristics and available AWS pricing - can vary, but the process and tooling can be used to approximate your monthly IaaS costing more accurately. Regardless of how you determine the instance types you start with, it is important to monitor usage over time and adjust as needed to keep the balance between resource availability, consumption, and cost. Customers ought to consider using services such as Citrix Analytics for Performance - the information such services provide can play a key role in keeping performance up and costs down.

Application Design

An extra consideration includes application design. As customers plan to migrate workloads to a cloud platform such as AWS, they must ensure that app performance is not impacted. A rule of thumb which has applied for over 20 years is that the data ought to reside as near as possible to the workload. This means more complex applications architectures ought to respect this rule. An example of this includes apps with a front-end and back-end (database). To avoid adding latency which will impact application performance, both the front-end and back-end are to be migrated. An alternative would be a hybrid approach using a mix of on-premises (for complex apps) and cloud hosted workloads (for simple applications). It is important to always consult with application vendors for compatibility. The linked Tech Zone decision matrix compares the different

One final word on application design, the Enterprise Layer Manager appliance does not currently run on AWS, and does not currently support exporting layered images into an immediately consumable disk format for use on AWS. If App Layering support on AWS is critical for your migration or deployment, email aws@citrix.com with information about your project. You’ll be added to the list to be an early adopter candidate for future releases, and your voice will be heard. For more information on Citrix App Layering, refer to the product documentation and App Layering reference architecture.

Control Layer Considerations

In the Citrix Architectural Design Framework, the Control layer defines the components that control the Citrix solution. This includes components like Active Directory (forest/domain, OU, and user group structure, group policies, and such), Microsoft SQL database usage, Citrix licensing, session brokering and administration, load management, and VDA provisioning/image management. As with previous sections of this document, here we focus on the considerations which are most important for Citrix virtualization systems on AWS, and provide links to existing documentation/guidance on others.

One of the most impactful decisions you are making for control layer components is the session brokering and administration choice. This decision is critical, with substantial implications on cost, complexity, availability, and ongoing maintenance efforts. We start by reviewing the deployment models we introduced earlier in this document, then dig more deeply into the AWS specific considerations.

Control Layer: Greenfield/Cloud Only Deployment

The green field or cloud only deployment model uses cloud services across the board. n). The AWS specific implications on the design of your Citrix virtualization system are minimal, but we walk you through them anyway. Since Citrix Cloud provides most of the infrastructure and administrative components as a service, you won't have to worry about SQL databases, Citrix License Servers, Citrix Director servers and more.

Control Layer: Hybrid Deployment

Remember that with the hybrid deployment model, you're going to be building/managing some of the Citrix virtualization system components, otherwise it is a green field or cloud only deployment by definition. The interesting thing here is that, in the context of the Control layer, they're almost identical.

Control Layer: Lift and Shift Deployment

With the legacy lift and shift deployment model, you're deploying all key control layer components (including Active Directory and all Citrix session brokering/management components) on AWS. If you have to go down the "lift and shift" path, it is both a blessing and a curse. It is a blessing in that most of these considerations have been thoroughly documented in various published works that are already available. It is a curse in that you'll have a lot more work to do both up front and over time to build, manage, secure, and maintain these components.

If you're a "lift and shift"er, you want to review and reference the following before you continue: collectively, they cover most of the design decisions you must consider to be successful deploying Citrix on AWS using the lift and shift deployment model:

• Using AWS Directory Service and Amazon RDS with Citrix Virtual Apps and Desktops (blog)
• Deploying Citrix Virtual Apps and Desktop with AWS Directory Service and Amazon RDS – Version 1.0 (deployment guide)

Active Directory Considerations

All deployment models for Citrix virtualization systems on AWS require Microsoft Active Directory. For a compelling user experience, functional Active Directory services must be available in every AWS region where you've got VDAs deployed. The structure and complexity of your Active Directory implementation must be carefully considered, but fortunately Citrix virtualization can flexibly integrate with various different AD designs and servicing models.

When deploying Active Directory on AWS, customers can build/maintain their own Active Directory Domain Controllers using Windows Server instances, use AWS Directory Service for Microsoft Active Directory, or a combination of the two. Active Directory trusts can also be used to connect two or more AD forests/domains depending upon the customer's needs.

For customers looking to minimize the administrative overhead required to build and maintain functional Active Directory services, the AWS Directory Service for Microsoft Active Directory (also known as AWS Managed Microsoft AD) is an option worth considering. This service provides you with a fully functional Active Directory forest/domain without the overhead of building and maintaining Windows Server VM instances. AWS Managed Microsoft AD is built on highly available, AWS-managed infrastructure. Each directory is deployed across multiple Availability Zones, and monitoring automatically detects and replaces domain controllers that fail. In addition, data replication and automated daily snapshots are configured for you. You do not have to install software, and AWS handles all patching and software updates. With AWS Managed Microsoft AD, you can use native Microsoft administrative tools, manage Windows machines and users with Microsoft Group Policy, join EC2 instances and AWS RDS for SQL Server instances to it, and even setup Active Directory trusts with existing AD instances to support various complex Enterprise scenarios.

Customers who choose to use the AWS Managed Microsoft AD service with Citrix virtualization technologies can expect these technologies to work with this AWS service, though there are a few important considerations to consider before doing so. For starters - you won't have Domain Administrator, Enterprise Administrator, or other 'super user' type access to the AD instance. You do, however, have full control of your own container at the root of the directory where you can create users, computers, groups, OU's, and group policies.

A few other things you CAN NOT do:

• Create AD objects in any of the default containers (such as /Computers): they're read-only. This brings up a common mistake some customers make when using Citrix's MCS provisioning technology: you must choose to create the machine accounts for your MCS managed VDAs in a container/OU that's writeable - if you don't choose such a location, MCS won't be able to create the machine accounts.
• Install and configure some AD integrated features such as Certificate Services. As such, this impacts customers who will be using Citrix's Federated Authentication Services ("FAS") technology (which requires AD integrated Certificate Services): these customers must build and manage their own Active Directory on AWS using EC2 Windows Server instances.
• Have local Server Administrator equivalence by default. In an 'out of the box' Active Directory installation, the Domain Administrators group is added to the local Server Administrators group by default. If you're using the AWS Managed Microsoft AD service, you must create your own server administrators' group, add your own users to it, create and apply a group policy to add your group to the built-in Server Administrators group on member servers/workstations.

While trust relationships, site/service configuration, replication, and other AD related topics will not be covered in this paper, Citrix has provided extensive documentation on these topics applicable to all three deployment models.

Note: AWS Directory Service for Microsoft Active Directory is a "Citrix Ready Verified" offering. While not officially supported by Citrix, the service IS fundamentally native Microsoft Active Directory - it is just managed by AWS instead of the customer. This AWS service does have some limitations imposed upon it to deliver it as a service at scale, and the currently known/most impactful limitations for a Citrix environment are listed here. For more information on Active Directory requirements for green field and hybrid deployments (environments using Citrix Cloud and the CVAD Service for session brokering and administration) see Citrix Cloud Connector Technical Details. Besides covering supported Active Directory functional levels, this article also covers deployments scenarios for Cloud Connectors in Active Directory.

For more information on Active Directory requirements for lift and shift deployments (environments using customer managed session brokering and administration via Citrix Virtual Apps and Desktops LTSR or CR versions) see CVAD System Requirements, Active Directory Functional Levels.

Session Brokering and Administration Considerations

As you've probably already gathered by now, the choice of how you provide session brokering and administration services is critical, and has broad reaching implications on overall cost, manageability, maintenance, and available capabilities for your Citrix virtualization system. As we've already discussed, Citrix recommends the use of the Citrix Cloud service (DaaS) for this critical component, but for certain requirements and scenarios, deploying a customer managed session brokering and administration subsystem (via CVAD LTSR or CR releases) can be necessary or recommended. The following table highlights some of these requirements and scenarios for your consideration:

Cloud Connectors, Delivery Controllers, and Resource Locations

Since both green field and hybrid models use cloud services (DaaS) for session brokering and administration, you deploy Cloud Connectors to create a resource location in each region where you plan to host VDAs. When you create a resource location in a region, you build a highly available configuration by deploying n+1 Cloud Connector instances and spreading the Cloud Connectors across Availability Zones in that region. Cloud Connectors are typically placed in separate private subnets from the VDAs to simplify security policy application, and the Cloud Connector instances must have outbound Internet access to facilitate connecting to Citrix Cloud. Placing them in a separate subnet from VDAs allows administrators to apply different routing policies to the two different resource types.

The same general concepts apply when we're talking about Delivery Controllers (CVAD), though we use the term zone vs. resource location in the customer managed brokering subsystem. Also note that Cloud Connector instances on EC2 are great candidates for reserved pricing since they are running anytime the system needs to be up. See this article for more information about sizing Cloud Connector instances.

Citrix DaaS Site Design Considerations

Resource Locations and Zones

Using Citrix zones (not to be confused with Availability Zones) can help users in remote regions connect to resources without necessarily forcing their connections to traverse large segments of the WAN. In a Citrix DaaS environment, each resource location is considered a zone. When you create a resource location and install a Cloud Connector, a zone is automatically created for you. Each zone can have a different set of resources, based on your unique needs and environment. For more information on zones, see the following link.

Machine Catalogs, Delivery Groups, and Resource Locations

Citrix administrators ought to ensure that VDAs are also spread across Availability Zones (AZ). An AWS Availability Zone (AZ) is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region - a physical location around the world where AWS cluster data centers. A virtual private cloud (VPC) is a virtual network which spans Availability Zones in the Region. Subnets are a required subcomponent of a VPC, and virtual network interfaces are each attached to a single subnet. Each subnet must reside entirely within one Availability Zone and cannot span zones. By launching VDAs in separate Availability Zones, you can protect your applications from the failure of a single location. See What is an Amazon VPC? for more information. To ensure VDAs are spread between AZs you can create a Machine Catalog per AZ (using one Host Connection per AZ) which then can map to a single Delivery Group.

Provisioning in AWS: Machine Creation Services

Starting with the release of CVAD 1811, role-based authentication can be used when creating a host connection for MCS provisioning in AWS. An IAM role or IAM user account associated with a Delivery Controller or Cloud Connector on an EC2 instance can be used in the place of a user's secret key and API key, enabling increased security, delegated administrative rights, and PKI-based environments with temporary credentials and session tokens. To configure a host connection using role-based authentication, first create an IAM role with the permissions described in CTX140429. Associate this role with an EC2 instance with a CVAD 1811+ Delivery Controller or a Cloud Connector. On versions of CVAD earlier than 1811, admins must provide the API Key (Access Key) and Secret Key of an IAM user to create a host connection.

After creating the host connection, create a machine catalog as described here using an AMI created from the master VDA image in AWS. For more details about MCS in AWS, see the following articles: Citrix MCS on AWS Deep Dive 1 and How MCS works after pooled VMs are created in AWS.

Another item that ought to be considered when deploying VDAs in AWS using MCS is EBS Volume initialization (also known as pre-warming or hydration). For volumes that were restored from snapshots, the storage blocks must be pulled down from Amazon S3 and written to the volume before you can access them. This preliminary action takes time and can cause a significant increase in the latency of I/O operations the first time each block is accessed. Volume performance is achieved after all blocks have been downloaded and written to the volume. See Initializing Amazon EBS Volumes on Windows for AWS recommended steps to Initialize Amazon EBS Volumes on Windows instances and see Initializing Amazon EBS Volumes on Linux for Linux instances.

See Infrastructure (or Platform) Layer Considerations for details on VPC design as it relates to MCS.

Troubleshooting Machine Creation Services

This section lists some common issues and associated recommendations/resolution links.

• Some newer AWS instance types will not show up by default in the Machine Catalog creation wizard in Studio (either CVAD or DaaS). The UI is populated with instance types from a static XML file which resides on Delivery Controllers (CVAD) or Cloud Connectors (DaaS). This XML can be modified to include newer instance types, but this file is overwritten with default values during upgrades (both Citrix initiated Cloud Connector updates or customer-initiated Delivery Controller upgrades). See Updating AWS Instance Types for XenDesktop for more details on how to update the list of available AWS instance types.
• MCS Provisioning fails on AWS EC2 when using Dedicated instances
• Unable to create AWS hosting connection on Citrix Cloud DDC when proxy is configured on connector server
• When Creating Machines with MCS and AWS an Error "XDDS:2367399e" Occurs
• Configure the Volume Worker instance to use the Machine Catalog VPC and not the default VPC
• How to Ensure Region Compatibility When Using XenApp and XenDesktop MCS in AWS
• Troubleshooting tips from the field for Machine Creation Services in AWS

Infrastructure (or Platform) Layer Considerations

In the Citrix Architectural Design Framework, the Infrastructure (or Platform) layer defines the physical elements where the Citrix workloads run. In this document, that of course refers to AWS. AWS provides many cloud services (165+) and is both the oldest and largest of the Hyperscale Cloud providers in existence today. It was also the first public cloud supported by Citrix virtualization technology, and is a compelling option for new or existing Citrix customers looking to move existing or run new Citrix virtualization workloads in 'the Cloud'.

Infrastructure as Code and the AWS Object Model

To understand how Citrix virtualization technologies are integrated with and run on top of AWS, it is useful to start with a basic understanding of the object model behind some of their key/relevant services. This also allows us to describe the AWS platform in terms that are familiar to most IT professionals. To facilitate this understanding, we refer to the following diagram which represents the design pattern for a DaaS resource location on AWS:

This design pattern is the foundation of most Citrix virtualization system architectures on AWS. It is also not just one massive pattern - it is built on various different, well maintained and documented design patterns for Enterprise IT on AWS. These patterns are represented, documented, and reproduced using AWS CloudFormation templates. AWS provides a library of Quick Start templates which can be run as-is, layered together ('nested') with other templates, and even duplicated and customized for your own specific needs. This highlights a couple of the other major advantages of public cloud infrastructure: infrastructure as code, and the 'pay as you go' nature of many cloud services. We dig more deeply into infrastructure as code in the Citrix virtualization world shortly, but we emphasize the point with a quick touchpoint that will likely resonate for the expected readers of this paper: for many enterprise IT architects, having access to such a vast library of services, design patterns, and technology tools at your fingertips is awesome. Combined with the ability to pay for resources as you consume them and simply remove them when you're done? This is a powerful way to learn about or evaluate new stuff, and it makes the ROI for at scale investments much easier to understand and communicate.

Back to the AWS object model for a moment: the top level object in diagram 14 is the AWS Region. You can think of AWS regions as clusters of well-connected but strategically separated data centers called Availability Zones. Each region will typically include 2 or more Availability Zones, which consist of one or more physical buildings with redundant power, networking, and connectivity. As of the time of this writing, AWS has 23 regions globally, which consist of 69 availability zones, but it is important to note that they're constantly investing in new regions and AZs. These numbers, while staggering to most of us, are likely already outdated by the time you read this. This highlights one of the other benefits of moving to public cloud infrastructure on AWS: you continue to benefit from the investments they're making (on a scale well beyond the reach of most IT organizations or even governments) over time. This continuous evolution/improvement, while daunting for change-averse IT organizations and business cultures, provides a broad reaching set of empowering benefits for Enterprise IT as it adapts to this 'new' model.

AWS region adoption choices are often based on proximity, services available, cost, compliance, or SLA. While choosing one or more right regions for your Citrix virtualization system is beyond the scope of this document, consider at least the following when making your choices:

• If you have one or more existing, customer-managed data centers you are connecting to AWS, consider one or more regions which provide the lowest latency network connectivity to your data centers and major offices.
• All regions can not have the AWS services or instance types you're looking for. AWS deploys new services or instance types initially to a few main regions, then expands to the rest over time. Also, newer regions cannot have older instance types - do your research before you build whenever possible.
• CVAD sites and DaaS resource locations are bound to a specific region. High availability for individual components of a site/resource location (such as cloud connectors, StoreFront servers, and ADC/Gateway VPX instances) is accomplished by placing resources in multiple availability zones in a given region.
• Don't go overboard spreading your infrastructure across regions: while it is easy to do on AWS, consider cost and complexity relative to the payoff you expect before you scale any system. You do end up paying for network traffic and storage traffic as well sometimes. The costs can be trivial for traffic while it is local to a region, but goes up when the traffic traverses regions or the Internet.

Stepping up a layer in Diagram 14 now, let's look at some of the networking constructs in this design pattern. The primary network construct on AWS is the VPC or "Virtual Private Cloud." VPCs are a regional construct (they span AZs) - you have at least one VPC in each region you deploy Citrix virtualization tech into. VPCs have a CIDR block of IP addresses defined, which must be unique if your network design routes traffic between multiple VPCs. VPCs are further broken down into subnets, and subnets are tied to an AZ (that is they do NOT span AZs in a region).

Subnets also have different attributes and objects associated with them, including routing policies and security policies. This is why the design patterns highlighted in this document (and other Citrix documentation) recommend putting VDAs in separate subnets from Cloud Connectors - so you can assign different routing and security policies to VDAs and Cloud Connectors.

Outbound Internet access from any subnet in a VPC (a regional construct) can be handled many different ways, but a common method is using NAT Gateways to provide Internet connectivity to private subnets. Public subnets are often served by Internet Gateways, which facilitate the routing of inbound connections to services you make accessible from the Internet.

Subnets are also commonly labeled as 'public' and 'private'. A public subnet is a subnet with Internet routable IP addresses assigned (in addition to the private IP addresses) and is associated with a route table that has a route to an Internet Gateway (IGW) for both inbound and outbound Internet traffic. A private subnet is a subnet with only private IP addresses assigned, and is associated with a route table that has a route for outbound internet access through a NAT Gateway or NAT Instances which reside in a public subnet. In a Citrix virtualization system, the Gateway virtual server (VIP) usually resides in a public subnet since it accepts inbound connections from client devices over the Internet and is used to securely proxy Citrix virtualization traffic into private subnets in a VPC.

There are many ways to build networks on AWS, with many innovative features and techniques available that you can't get elsewhere. We're not going to introduce you to them all here, but two tools/techniques worth looking into are VPC peering and transit gateways. These two constructs help introduce you to routing traffic between VPCs simply (VPC peering) or in a more Enterprise ready, hybrid cloud friendly model (transit gateways).

There's much more we can dig into here, and for the curious and motivated, there's a mountain of public domain knowledge available at your fingertips to learn more. For now, let's bring this back around to design patterns underneath all the diagrams you've seen in this paper.

One of the compelling attributes of the AWS platform is that it has been built on publicly consumable APIs. Why is this compelling? For one thing, this means that much any type of infrastructure component you can run on AWS can be reproducibly built from code. When combined with a powerful and comprehensive deployment service such as AWS CloudFormation, customers have a powerful framework for learning about, customizing, deploying, and managing IT systems. The concept of Infrastructure as Code can be new or perplexing for many traditional Enterprise focused technologists, but it can be transformational once adopted and practiced.

As we mentioned earlier, AWS provides a library of CloudFormation based Quick Start templates which can be run as-is, layered together ('nested') with other templates, and even duplicated and customized for your own specific needs. This library of templates is managed and maintained by AWS, in cooperation with technology partners such as Citrix, and these templates are often open-sourced (meaning they can be duplicated and modified as needed). As of the time of writing, the following Quick Start templates are available for Citrix technologies on AWS:

• Citrix ADC for Web Applications - deploys highly available Citrix ADC VPX instances on AWS. While the use case focus differs slightly, this design pattern is functional and relevant for Citrix Gateway deployments with CVAD/DaaS also.

Summary - Understanding Design Patterns for Citrix on AWS

Confused yet? If so, don't be alarmed: this can well be the start of your Citrix on AWS public cloud journey, and we've merely skimmed the surface of many deep topics here. Hopefully, however, we've successfully illustrated the following salient points:

• Infrastructure as Code is a powerful concept that can revolutionize the way complete systems are designed, built, and maintained.
• When deploying systems on AWS' public cloud, different components of any given solution can be represented by code, and built on-demand using AWS CloudFormation and other technologies.
• These components are represented by stack templates when using AWS CloudFormation, and templates can be copied and modified, as needed, to achieve the desired results.
• Templates can be nested, building complete systems (such as a fully functioning DaaS resource location on AWS) from the individual design patterns (templates).
• The Citrix DaaS on AWS Quick Start template is built upon three AWS managed/maintained foundation templates, which are well documented. Start with the following links to learn more about each:
• By using templates and performing trial builds, an Enterprise technologist can learn about, evaluate, and design systems that meet the specific needs of their organization or customer.

AWS Infrastructure Layer - extra Resources

The following resources can be used to help more about Citrix virtualization on AWS requirements and leading practices:

• Communication Ports Used by Citrix Technologies: a good global reference for the communication ports used by different components of the Citrix virtualization stack.

Operations Layer Considerations

This section defines the operational activities that administrators perform on a periodic basis. Many of these are not specific to AWS, and are detailed in existing published documentation. In the following tables, we've summarized some of the more important or AWS specific tasks. Readers can refer to the Monitor topic in Citrix product documentation for more information.

On-Demand Tasks

The following table outlines the tasks that are expected to be performed on-demand based on application requirements and troubleshooting efforts.

Daily Periodic Tasks

The following table outlines the tasks that ought to be performed daily.

Weekly Periodic Tasks

The following table outlines the tasks that are to be performed on a weekly basis.

Monthly Periodic Tasks

The following table outlines the tasks that are to be performed on a monthly basis.

Yearly Periodic Tasks

The following table outlines the tasks that are to be performed on a yearly basis.

Sources

Goal of this reference architecture is to assist you with planning your own implementation. To make this job easier, we would like to provide you with source diagrams that you can adapt in your own detailed designs and implementation guides: source diagrams.