Citrix DaaS

Microsoft Azure Resource Manager cloud environments

Follow the guidance in this article when using the Microsoft Azure Resource Manager to provision virtual machines in your Citrix Virtual Apps or Citrix Virtual Desktops service deployment.

We assume you are familiar with the following:

Azure on-demand provisioning

With Azure on-demand provisioning, VMs are created only when Citrix Virtual Apps and Desktops initiates a power-on action, after the provisioning completes.

When you use MCS to create machine catalogs in the Azure Resource Manager, the Azure on-demand provisioning feature:

  • Reduces your storage costs
  • Provides faster catalog creation

When you create an MCS catalog, the Azure portal displays the network security group, network interfaces, base images, and identity disks in the resource groups.

The Azure portal does not show a VM until Citrix Virtual Apps and Desktops initiates a power-on action for it. Then, the VM’s status in the Full Configuration interface changes to On. There are two types of machines with the following differences:

  • For a pooled machine, the operating system disk and write-back cache exist only when the VM exists. When you shut down a pooled machine in the console, the VM is not visible in the Azure portal. There is a significant storage cost saving if you routinely shut down machines (for example, outside of working hours).
  • For a dedicated machine, the operating system disk is created the first time the VM is powered on. The VM in the Azure portal remains in storage until the machine identity is deleted. When you shut down a dedicated machine in the console, the VM is still visible in the Azure portal.

Connection to Azure Resource Manager

Create and manage connections describes the wizards that create a connection. The following information covers details specific to Azure Resource Manager connections.

Considerations:

  • Citrix recommends using Service Principal with contributor role. However, see Minimum permissions section to get the list of minimum permissions.
  • When creating the first connection, Azure prompts you to grant it the necessary permissions. For future connections you must still authenticate, but Azure remembers your previous consent and does not display the prompt again.
  • Accounts used for authentication must be a co-administrator of the subscription.
  • The account used for authentication must be a member of the subscription’s directory. There are two types of accounts to be aware of: ‘Work or School’ and ‘personal Microsoft account.’ See CTX219211 for details.
  • While you can use an existing Microsoft account by adding it as a member of the subscription’s directory, there can be complications if the user was previously granted guest access to one of the directory’s resources. In this case, they might have a placeholder entry in the directory that does not grant them the necessary permissions, and an error is returned.

    Rectify this by removing the resources from the directory and add them back explicitly. However, exercise this option carefully, because it has unintended effects for other resources that account can access.

  • There is a known issue where certain accounts are detected as directory guests when they are actually members. Configurations like this typically occurs with older established directory accounts. Workaround: add an account to the directory, which takes the proper membership value.
  • Resource groups are simply containers for resources, and they can contain resources from regions other than their own region. This can potentially be confusing if you expect resources displayed in a resource group’s region to be available.
  • Ensure that your network and subnet are large enough to host the number of machines you require. This requires some foresight, but Microsoft helps you specify the right values, with guidance about the address space capacity.

You can establish a host connection to Azure in two ways:

  • Authenticate to Azure to create a service principal.
  • Use the details from a previously created service principal to connect to Azure.

Create a service principal

Important:

This feature is not yet available for Azure China and Azure Germany subscriptions.

Before you start, authenticate to Azure. Ensure:

  • You have a user account in your subscription’s Azure Active Directory tenant.
  • The Azure AD user account is also a co-administrator for the Azure subscription that you want to use for provisioning resources.
  • Authentication requires global administrator permissions.

When you authenticate to Azure to create a service principal, an application is registered in Azure. A secret key (client secret) is created for the registered application. The registered application uses the client secret to authenticate to Azure AD. Be sure to change the client secret before it expires. You receive an alert on the console before the secret key expires. See Application secret and secret expiration date.

To authenticate to Azure to create a service principal, complete the following steps in the Add Connection and Resources wizard:

  1. On the Connection page, select Create a new connection, the Microsoft Azure connection type, and your Azure environment.

  2. Select which tools to use to create the virtual machines and then select Next.

  3. On the Connection Details page, enter your Azure subscription ID and a name for the connection. After you enter the subscription ID, the Create new button is enabled.

    Note:

    The connection name can contain 1–64 characters, and cannot contain only blank spaces nor the characters \/;:#.*?=<>|[]{}"'()'.

  4. Select Create new and then enter the Azure Active Directory account user name and password.

  5. Select Sign in.

  6. Select Accept to give Citrix Virtual Apps and Desktops the listed permissions. Citrix Virtual Apps and Desktops creates a service principal that allows it to manage Azure resources on behalf of the specified user.

  7. After you select Accept, you return to the Connection page in the wizard.

    Note:

    After you successfully authenticate to Azure, the Create new and Use existing buttons disappear. The Connection successful text appears, with a green check mark, indicating the successful connection to your Azure subscription.

  8. On the Connection Details page, select Next.

    Note:

    You cannot proceed to the next page until you successfully authenticate to Azure and consent to giving the required permissions.

  9. Configure resources for the connection. Resources comprise the region and the network.

    • On the Region page, select a region.
    • On the Network page, do the following:
      • Type a 1–64 character resource name to help identify the region and network combination. A resource name cannot contain only blank spaces nor the characters \/;:#.*?=<>|[]{}"'()'.
      • Select a virtual network/resource group pair. (If you have more than one virtual network with the same name, pairing the network name with the resource group provides unique combinations.) If the region you selected on the previous page does not have any virtual networks, return to that page and select a region that has virtual networks.
  10. On the Summary page, view a summary of settings and select Finish to complete your setup.

Use the details from a previously created service principal to connect to Azure

To create a service principal manually, connect to your Azure Resource Manager subscription and use the PowerShell cmdlets provided in the following sections.

Prerequisites:

  • SubscriptionId: Azure Resource Manager SubscriptionID for the subscription where you want to provision VDAs.
  • ActiveDirectoryID: Tenant ID of the application that you registered with Azure AD.
  • ApplicationName: Name for the application to be created in Azure AD.
  • ApplicationPassword: Password for the application. You use this password as the application secret when creating the host connection.

To create a service principal:

  1. Connect to your Azure Resource Manager subscription.

    Connect-AzAccount

  2. Select the Azure Resource Manager subscription where you want to create the service principal.

    Select-AzSubscription -SubscriptionID $SubscriptionId

  3. Create the application in your AD tenant.

    $AzureADApplication = New-AzADApplication -DisplayName $ApplicationName -HomePage "https://localhost/$ApplicationName" -IdentifierUris https://$ApplicationName -Password $ApplicationPassword

  4. Create a service principal.

    New-AzADServicePrincipal -ApplicationId $AzureADApplication.ApplicationId

  5. Assign a role to the service principal.

    New-AzRoleAssignment -RoleDefinitionName Contributor -ServicePrincipalName $AzureADApplication.ApplicationId –scope /subscriptions/$SubscriptionId

  6. From the output window of the PowerShell console, note the ApplicationId. You provide that ID when creating the host connection.

In the Add Connection and Resources wizard:

  1. On the Connection page, select Create a new connection, the Microsoft Azure connection type, and your Azure environment.

  2. Select which tools to use to create the virtual machines and then select Next.

  3. On the Connection Details page, enter your Azure subscription ID and a name for the connection.

    Note:

    The connection name can contain 1–64 characters, and cannot contain only blank spaces nor the characters \/;:#.*?=<>|[]{}"'()'.

  4. Select Use existing. In the Existing Service Principal Details window, enter the following settings for the existing service principal. After you enter the details, the Save button is enabled. Select Save. You cannot progress beyond this page until you provide valid details.

    • Subscription ID. Enter your Azure subscription ID. To obtain your subscription ID, sign in to the Azure portal and navigate to Subscriptions > Overview.
    • Active Directory ID (tenant ID). Enter the Directory (tenant) ID of the application that you registered with Azure AD.
    • Application ID. Enter the Application (client) ID of the application that you registered with Azure AD.
    • Application secret. Create a secret key (client secret). The registered application uses the key to authenticate to Azure AD. We recommend that you change keys regularly for security purposes. Be sure to save the key because you cannot retrieve the key later. See Application secret and secret expiration date.
    • Secret expiration date. Enter the date after which the application secret expires. You receive an alert on the console before the secret key expires. However, if the secret key expires, you receive errors.

      Note:

      For security purposes, the expiration period cannot be more than two years from now.

    • Authentication URL. This field is automatically populated and is not editable.
    • Management URL. This field is automatically populated and is not editable.
    • Storage suffix. This field is automatically populated and is not editable.
  5. After selecting Save, you return to the Connection Details page. Select Next to proceed to the next page.

  6. Configure resources for the connection. Resources comprise the region and the network.

    • On the Region page, select a region.
    • On the Network page, do the following:
      • Type a 1–64 character resource name to help identify the region and network combination. A resource name cannot contain only blank spaces nor the characters \/;:#.*?=<>|[]{}"'()'.
      • Select a virtual network/resource group pair. (If you have more than one virtual network with the same name, pairing the network name with the resource group provides unique combinations.) If the region you selected on the previous page does not have any virtual networks, return to that page and select a region that has virtual networks.
  7. On the Summary page, view a summary of settings and select Finish to complete your setup.

Application secret and secret expiration date

View the application ID

You can view the application ID in the Full Configuration interface.

In the Add Connection and Resources wizard, select the connection to view the details. The Details tab shows the Application ID.

Application ID on Add Connection and Resouces page

View the application secret

You can view the application secret in the Azure portal.

  1. Get the Application ID from the Full Configuration interface.
  2. Sign in to the Azure portal.
  3. In Azure, select Azure Active Directory.
  4. From App registrations in Azure AD, select your application.
  5. Go to Certificates & secrets.
  6. Click Client secrets.

Application secret in Azure portal

Change the secret expiration date

Use the Full Configuration interface to add or modify the expiration date for the application secret in use.

  1. In the Add Connection and Resources wizard, right-click a connection, and click Edit Connection.
  2. On the Connection Properties page, click Secret expiration date to add or modify the expiration date for the application secret in use.

Change secret expiration date

Create a new application secret

You can create a new application secret of a connection through the Azure portal.

  1. Select Azure Active Directory.
  2. From App registrations in Azure AD, select your application.
  3. Go to Certificates & secrets.
  4. Click Client secrets > New client secret.

    Create new application secret

  5. Provide a description of the secret and specify a duration. When you are done, select Add.

    Note:

    Be sure to save the client secret because you cannot retrieve it later.

  6. Copy the client secret value and the expiration date.
  7. In the Full Configuration interface, edit the corresponding connection and replace the content in the Application secret and Secret expiration date field with the values you copied.

Create a machine catalog using an Azure Resource Manager image

This information is a supplement to the guidance in Create machine catalogs.

An image is the template that is used to create the VMs in a machine catalog. Before creating the machine catalog, create an image in Azure Resource Manager. For general information about images, see Create machine catalogs.

Tip:

Use of unmanaged disk to provision VM is deprecated.

In the machine catalog creation wizard:

  • The Machine Type and Machine Management pages do not contain Azure-specific information. Follow the guidance in the Create machine catalogs article.

  • On the Master Image page, select a resource group and then navigate (drill down) through the containers to the Azure VHD or the Shared Image Gallery or the Azure ImageVersion you want to use as the image. The VHD or the ImageVersion must have a Citrix VDA installed on it. If the VHD is attached to a VM, stop the VM.

    Note:

    • Using a machine profile with trusted launch as Security Type is mandatory when you select an image or snapshot that has trusted launch enabled. You can then enable or disable SecureBoot and vTPM by specifying their values in the Machine Profile. Trusted Launch is not supported for Shared Image Gallery. For information about Azure trusted launch, see https://docs.microsoft.com/en-us/azure/virtual-machines/trusted-launch.
    • You can create a provisioning scheme using ephemeral OS disk on Windows with trusted launch. When you select an image with trusted launch, then you must select a machine profile with trusted launch that is enabled with vTPM. To create machine catalogs using ephemeral OS disk, see How to create machines using ephemeral OS disks.
    • When image replication is in progress, you can proceed and select the image as the master image and complete the setup. However, catalog creation might take longer to complete while the image is being replicated. MCS requires the replication to complete within an hour starting from catalog creation. If the replication times out, catalog creation fails. You can verify the replication status in Azure. Try again if the replication is still pending or after the replication completes.
    • When you select a master image for machine catalogs in Azure, MCS identifies the OS type based on the master image and machine profile you select. If MCS can’t identify it, select the OS type that matches the master image.

    To enable VMs in the catalog to inherit configurations from a machine profile, select the Use a machine profile check box. From the resource group list, locate a resource group as needed. Next, in the resource group, select a VM or ARM template spec to use as the machine profile.

    Validate the ARM template spec to make sure whether it can be used as a machine profile to create a machine catalog. There are two ways to validate the ARM template spec:

    • After you select the ARM template spec from the resource group list, click Next. Error messages appear if the ARM template spec has errors.
    • Run one of the following PowerShell commands:
      • Test-ProvInventoryItem -HostingUnitName <string> -InventoryPath <string>
      • Test-ProvInventoryItem -HostingUnitUid <Guid> -InventoryPath <string>

    Examples of configurations that VMs can inherit from a machine profile include:

    • Accelerated networking
    • Boot diagnostics
    • Host disk caching (relating to OS and MCSIO disks)
    • Machine size (unless otherwise specified)
    • Tags placed on the VM

    After you create the catalog, you can view the configurations that the image inherits from the machine profile. On the Machine Catalogs node, select the catalog to view its details in the lower pane. Then, click the Template Properties tab to view machine profile properties. The Tags section displays up to three tags. To view all tags placed on the VM, click View all.

    If you want MCS to provision VMs on an Azure dedicated host, enable the Use a host group check box and then select a host group from the list. A host group is a resource that represents a collection of dedicated hosts. A dedicated host is a service that provides physical servers that host one or more virtual machines. Your server is dedicated to your Azure subscription, not shared with other subscribers. When you use a dedicated host, Azure ensures that your VMs are the only machines running on that host. This feature is suitable for scenarios where you must meet regulatory or internal security requirements. To learn more about host groups and considerations for using them, see Azure dedicated hosts.

    Important:

    • Only host groups that have Azure auto-placement enabled are shown.
    • When using a machine profile, you cannot use a host group.
    • Using a host group changes the Virtual Machines page offered later in the wizard. Only machine sizes that the selected host group contains are shown on that page. Also, Availability Zones are selected automatically and not available for selection.
  • The Storage and License Types page appears only when you use an Azure Resource Manager image.

    Storage and License Types page

    You have the following storage types to use for the machine catalog:

    • Premium SSD. Offers a high-performance, low-latency disk storage option suitable for VMs with I/O-intensive workloads.
    • Standard SSD. Offers a cost-effective storage option that is suitable for workloads that require consistent performance at lower IOPS levels. An Azure identity disk is always created using Standard SSD.
    • Standard HDD. Offers a reliable, low-cost disk storage option suitable for VMs that run latency-insensitive workloads.
    • Azure ephemeral OS disk. Offers a cost-effective storage option that reuses the local disk of the VMs to host the operating system disk. Alternatively, you can use PowerShell to create machines that use ephemeral OS disks. For more information, see Azure ephemeral disks. Be aware of the following considerations when using an ephemeral OS disk:
      • Azure ephemeral OS disk and MCS I/O cannot be enabled at the same time.
      • To update machines that use ephemeral OS disks, you must select an image whose size does not exceed the size of the VM’s cache disk or temporary disk.
      • You cannot use the Retain system disk during power cycles option offered later in the wizard.

    The storage type determines which machine sizes are offered on the Virtual Machines page of the wizard. MCS configures premium and standard disks to use Locally Redundant Storage (LRS). LRS makes multiple synchronous copies of your disk data within a single data center. Azure ephemeral OS disks use the local disk of the VMs to store the operating system. For details about Azure storage types and storage replication, see the following:

    Select whether to use existing Windows licenses or Linux licenses.

    • Windows licenses: Using Windows licenses along with Windows images (Azure platform support images or custom images) lets you run Windows VMs in Azure at a reduced cost. There are two types of licenses:

      • Windows Server license. Lets you use your Windows Server or Azure Windows Server licenses, allowing you to use Azure Hybrid Benefits. For details, see https://azure.microsoft.com/en-us/pricing/hybrid-benefit/. Azure Hybrid Benefit reduces the cost of running VMs in Azure to the base compute rate, waiving the cost of extra Windows Server licenses from the Azure gallery.

      • Windows Client license. Lets you bring your Windows 10 licenses to Azure, allowing you to run Windows 10 VMs in Azure without the need for extra licenses. For details, see Client Access Licenses and Management Licenses.

    Note:

    The Windows Client license option varies depending on the operating system that you select during machine catalog setup. If you select Multi-session OS, the option appears as Use my Windows 10 licenses. If you select Single-session OS, the option appears as Use my Windows Client licenses.

    You can verify that the provisioned VM is using the licensing benefit by running the following PowerShell command: Get-AzVM -ResourceGroup MyResourceGroup -Name MyVM.

    Alternatively, you can use the Get-Provscheme PowerShell SDK to perform the verification. For example: Get-Provscheme -ProvisioningSchemeName "My Azure Catalog". For more information about this cmdlet, see https://developer-docs.citrix.com/projects/citrix-virtual-apps-desktops-sdk/en/latest/MachineCreation/Get-ProvScheme/.

    • Linux licenses: With bring-your-own-subscription (BYOS) Linux licenses, you do not have to pay for the software. The BYOS charge only includes the compute hardware fee. There are two types of licenses:

      • RHEL_BYOS: To use RHEL_BYOS type successfully, enable Red Hat Cloud Access on your Azure subscription.
      • SLES_BYOS: The BYOS versions of SLES include support from SUSE.

      You can set the LicenseType value to Linux options at New-ProvScheme and Set-ProvScheme.

      Example of setting LicenseType to RHEL_BYOS at New-ProvScheme:

      New-ProvScheme -CleanOnBoot -ProvisioningSchemeName "azureCatalog" -RunAsynchronously -Scope @() -SecurityGroup @() -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" /><Property xsi:type="StringProperty" Name="StorageAccountType" Value="StandardSSD_LRS" /><Property xsi:type="StringProperty" Name="ResourceGroups" Value="hu-dev-mcs" /><Property xsi:type="StringProperty" Name="OsType" Value="Linux" /><Property xsi:type="StringProperty" Name="LicenseType" Value="RHEL_BYOS" /></CustomProperties>'
      <!--NeedCopy-->
      

      Example of setting LicenseType to SLES_BYOS at Set-ProvScheme:

      Set-ProvScheme -ProvisioningSchemeName "azureCatalog" -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" /><Property xsi:type="StringProperty" Name="StorageAccountType" Value="StandardSSD_LRS" /><Property xsi:type="StringProperty" Name="ResourceGroups" Value="hu-dev-mcs" /><Property xsi:type="StringProperty" Name="OsType" Value="Linux" /><Property xsi:type="StringProperty" Name="LicenseType" Value="SLES_BYOS" /></CustomProperties>'
      <!--NeedCopy-->
      

      Note:

      If LicenseType value is empty, then the default values are Azure Windows Server License or Azure Linux License, depending on OsType value.

      Example of setting LicenseType to empty:

      Set-ProvScheme -ProvisioningSchemeName "azureCatalog" -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" /><Property xsi:type="StringProperty" Name="StorageAccountType" Value="StandardSSD_LRS" /><Property xsi:type="StringProperty" Name="ResourceGroups" Value="hu-dev-mcs" /><Property xsi:type="StringProperty" Name="OsType" Value="Linux" /></CustomProperties>'
      <!--NeedCopy-->
      

    See the following documents to understand License types and their benefits:

    Azure Shared Image Gallery (SIG) is a repository for managing and sharing images. It lets you make your images available throughout your organization. We recommend that you store an image in SIG when creating large non-persistent machine catalogs because doing that enables faster resets of VDA OS disks. After you select Place image in Azure Shared Image Gallery, the Azure Shared Image Gallery settings section appears, letting you specify more SIG settings:

    • Ratio of virtual machines to image replicas. Lets you specify the ratio of virtual machines to image replicas that you want Azure to keep. By default, Azure keeps a single image replica for every 40 non-persistent machines. For persistent machines, that number defaults to 1,000.

    • Maximum replica count. Lets you specify the maximum number of image replicas that you want Azure to keep. The default is 10.

  • On the Virtual Machines page, indicate how many VMs you want to create. You must specify at least one and select a machine size. After catalog creation, you can change the machine size by editing the catalog.

  • The NICs page does not contain Azure-specific information. Follow the guidance in the Create machine catalogs article.

  • On Disk Settings page, choose whether to enable write-back cache. After enabling write-back cache, you can do the following:

    • Configure the size of the disk and RAM used for caching temporary data. For more information, see Configure cache for temporary data.

    • Select the storage type for the write-back cache disk. The following storage options are available to use for the write-back cache disk:

      • Premium SSD
      • Standard SSD
      • Standard HDD
    • Select the type for the write-back cache disk.

      • Use non-persistent write-back cache disk. If selected, the write-back cache disk is deleted during power cycles. Any data redirected to it will be lost. If the VM’s temporary disk has sufficient space, it is used to host the write-back cache disk to reduce your costs. After catalog creation, you can check whether the provisioned machines use the temporary disk. To do so, click the catalog and verify the information on the Template Properties tab. If the temporary disk is used, you see Non-persistent Write-back Cache Disk and its value is Yes (using VM’s temporary disk). If not, you see Non-persistent Write-back Cache Disk and its value is Yes (not using VM’s temporary disk).
      • Use persistent write-back cache disk. If selected, the write-back cache disk persists for the provisioned VMs. Enabling the option increases your storage costs.

      For more information, see Create machine catalogs.

    • Choose whether to retain system disks for VDAs during power cycles.

      • Retain system disk during power cycles. By default, the system disk is deleted on shutdown and recreated on startup. This ensures that the disk is always in a clean state but results in longer VM restart times. If system writes are redirected to the RAM cache and overflow to the cache disk, the system disk remains unchanged. Enabling this option increases your storage costs but reduces VM restart times. Select Enable write-back cache to make this option available.
        • Retain VMs across power cycles. Select this option to retain your VM customization and to enable the VMs to be started through the Azure portal.
    • Choose whether to encrypt data on the machines provisioned in the catalog. Server-side encryption with a customer-managed encryption key lets you manage encryption at a managed disk level and protect data on the machines in the catalog. For more information, see Azure server side encryption.

  • On the Resource Group page, choose whether to create resource groups or use existing groups.

    • If you choose to create resource groups, select Next.
    • If you choose to use existing resource groups, select groups from the Available Provisioning Resource Groups list. Remember: Select enough groups to accommodate the machines you’re creating in the catalog. A message appears if you choose too few. You might want to select more than the minimum required if you plan to add more VMs to the catalog later. You can’t add more resource groups to a catalog after the catalog is created.

    For more information, see Azure resource groups.

  • The Machine Identities, Domain Credentials, and Summary pages do not contain Azure-specific information. Follow the guidance in the Create Machine Catalogs article.

Complete the wizard.

Use machine profile property values

The machine catalog uses the following properties that are defined in the custom properties:

  • Availability zone
  • Dedicated Host Group Id
  • Disk Encryption Set Id
  • OS type
  • License type
  • Storage type

If these custom properties are not defined explicitly, then the property values are set from the ARM template spec or VM, whichever is used as the machine profile. In addition, if ServiceOffering is not specified, then it will be set from the machine profile.

Note:

If some of the properties are missing from the machine profile and not defined in the custom properties, then the default values of the properties take place wherever applicable.

The following section describes some scenarios at New-ProvScheme and Set-ProvScheme when CustomProperties either have all the properties defined or values are derived from the MachineProfile.

  • New-ProvScheme Scenarios

    • MachineProfile has all the properties and CustomProperties are not defined. Example:

      New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpA.vm"

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="DiskEncryptionSetId" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="DedicatedHostGroupId" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="Zones" Value="<mpA-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • MachineProfile has some properties and CustomProperties are not defined. Example: MachineProfile only has LicenseType and OsType.

      New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpA.vm"

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="OSType" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<mpA-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • Both MachineProfile and CustomProperties define all properties. Example:

      New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpA.vm" -CustomProperties $CustomPropertiesA

      Custom properties take priority. The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="DiskEncryptionSetId" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="DedicatedHostGroupId" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="Zones" Value="<CustomPropertiesA-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • Some properties are defined in MachineProfile and some properties are defined in CustomProperties. Example:
      • CustomProperties define LicenseType and StorageAccountType
      • MachineProfile define LicenseType, OsType, and Zones

      New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpA.vm" -CustomProperties $CustomPropertiesA

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<mpA-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<CustomPropertiesA-value>"/>
       <Property xsi:type="StringProperty" Name="Zones" Value="<mpA-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • Some properties are defined in MachineProfile and some properties are defined in CustomProperties. In addition, ServiceOffering is not defined. Example:

      • CustomProperties define StorageType
      • MachineProfile define LicenseType
       New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mp.vm"
       -ServiceOffering "XDHyp:\HostingUnits\azureunit\serviceoffering.folder\<explicit-machine-size>.serviceoffering"
       <!--NeedCopy-->
      

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select ServiceOffering
       serviceoffering.folder\<explicit-machine-size>.serviceoffering
      
       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="explicit-storage-type"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="value-from-machineprofile"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • If the OsType is in neither in the CustomProperties nor in the MachineProfile, then:
      • The value is read from the master image.
      • If the master image is an unmanaged disk, the OsType is set to Windows. Example:

      New-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpA.vm" -MasterImageVM "XDHyp:\HostingUnits\azureunit\image.folder\linux-master-image.manageddisk"

      The value from the master image is written to the custom properties, in this case Linux.

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="OSType" Value="Linux"/>
       </CustomProperties>
       <!--NeedCopy-->
      
  • Set-ProvScheme Scenarios

    • An existing catalog with:

      • CustomProperties for StorageAccountType and OsType
      • MachineProfile mpA.vm that defines zones
    • Updates:

      • MachineProfile mpB.vm that defines StorageAccountType
      • A new set of custom properties $CustomPropertiesB that defines LicenseType and OsType

      Set-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpB.vm" -CustomProperties $CustomPropertiesB

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<mpB-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<CustomPropertiesB-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<CustomPropertiesB-value>"/>
           </CustomProperties>
       <!--NeedCopy-->
      
    • An existing catalog with:
      • CustomProperties for StorageAccountType and OsType
      • MachineProfile mpA.vm that defines StorageAccountType and LicenseType
    • Updates:
      • A new set of custom properties $CustomPropertiesB that defines StorageAccountType and OsType.

      Set-ProvScheme -CustomProperties $CustomPropertiesB

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<CustomPropertiesB-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<CustomPropertiesB-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<mp-A-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      
    • An existing catalog with:
      • CustomProperties for StorageAccountType and OsType
      • MachineProfile mpA.vm that defines Zones
    • Updates:
      • A MachineProfile mpB.vm that defines StorageAccountType and LicenseType
      • ServiceOffering is not specified

      Set-ProvScheme -MachineProfile "XDHyp:\HostingUnits\azureunit\machineprofile.folder\azure.resourcegroup\mpB.vm"

      The following values are set as custom properties for the catalog:

       Get-ProvScheme | select ServiceOffering
       serviceoffering.folder\<value-from-machineprofile>.serviceoffering
      
       Get-ProvScheme | select CustomProperties
       <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <Property xsi:type="StringProperty" Name="StorageAccountType" Value="<mpB-value>"/>
       <Property xsi:type="StringProperty" Name="OSType" Value="<prior-CustomProperties-value>"/>
       <Property xsi:type="StringProperty" Name="LicenseType" Value="<mpB-value>"/>
       </CustomProperties>
       <!--NeedCopy-->
      

Use PowerShell to enable Azure VM extensions

After you select the ARM template spec, run the following PowerShell commands to work with Azure VM extensions:

  • To view the list of supported Azure VM extensions: Get-ProvMetadataConfiguration
  • To add additional VM extensions: Add-ProvMetadataConfiguration. For example, Add-ProvMetadataConfiguration -PluginType "AzureRM" -ConfigurationName "Extension" -ConfigurationValue "CustomScriptExtension"

    If you try to add any of the following, the command fails with an error message:

    • Citrix defined extension.
    • Existing user defined extension.
    • Unsupported configuration keys. Currently, the supported configuration key is Extension.
  • To remove extensions from the list: Remove-ProvMetadataConfiguration. You can remove the extensions that you added.

Page File Location

In Azure environments, the page file location is set up when you first create a VM. The format of the page file setting is: page file location [min size] [max size] (the size in MB). For more information, see How to determine the appropriate page file size.

During image preparation, when you create the provisioning scheme, MCS determines the page file location based on certain rules. After you create the provisioning scheme, you cannot:

  • Change the VM size
  • Update the machine profile
  • Change the EOS and MCS I/O properties

Page file location determination

Features like EOS and MCS I/O have their own expected page file location and are exclusive to each other. The following table shows the expected page file location for each feature:

Feature Expected page file location
EOS OS disk
MCS I/O Azure temporary disk first, otherwise write-back cache disk

Note:

Even if you decouple image preparation from provisioning scheme creation, MCS correctly determines the page file location. The default page file location is on the OS disk.

Page file setting scenarios

The following table describes some of the possible scenarios of page file setting during image preparation and provisioning scheme update:

During Scenario Outcome
Image preparation You set the source image page file on the temporary disk, while the VM size that you specify in the provisioning scheme has no temporary disk The page file is placed on the OS
Image preparation You set the source image page file on the OS disk, while the VM size that you specify in the provisioning scheme has a temporary disk The page file is placed on the temporary disk
Image preparation You set the source image page file on the temporary disk and enable the ephemeral OS disk in the provisioning scheme The page file is placed on the OS disk
Provisioning scheme update You attempt to update the provisioning scheme. The original VM size has a temporary disk whereas the target VM has no temporary disk Rejects the change with an error message
Provisioning scheme update You attempt to update the provisioning scheme. The original VM size has no temporary disk whereas the target VM has a temporary disk Rejects the change with an error message

Update page file setting

Using PowerShell commands, you can specify page file settings, including the location and size. This overrides the page file settings determined by MCS. You can do this by running the following New-ProvScheme command during machine catalog creation:

New-ProvScheme -CleanOnBoot `
    -HostingUnitName "zijinnet" `
    -IdentityPoolName "PageFileSettingExample" `
    -ProvisioningSchemeName "PageFileSettingExample" `
    -InitialBatchSizeHint 1 `
    -MasterImageVM "XDHyp:\HostingUnits\zijinnet\image.folder\neal-zijincloud-resources.resourcegroup\CustomWin10VDA_OsDisk_1_9473d7c8a6174b2c8284c7d3efeea88f.manageddisk" `
    -NetworkMapping @{"0"="XDHyp:\\HostingUnits\\zijinnet\\virtualprivatecloud.folder\\East US.region\\virtualprivatecloud.folder\\neal-zijincloud-resources.resourcegroup\\neal-zijincloud-resources-vnet.virtualprivatecloud\\default.network"} `
    -ServiceOffering "XDHyp:\\HostingUnits\\zijinnet\\serviceoffering.folder\\Standard_B2ms.serviceoffering" `
    -CustomProperties '<CustomProperties xmlns=" http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"> `
    <Property xsi:type="StringProperty" Name="PersistOsDisk" Value="false"/> `
    <Property xsi:type="StringProperty" Name="PersistVm" Value="false"/> `
    <Property xsi:type="StringProperty" Name="PageFileDiskDriveLetterOverride" Value="d"/> `
    <Property xsi:type="StringProperty" Name="InitialPageFileSizeInMB" Value="2048"/> `
    <Property xsi:type="StringProperty" Name="MaxPageFileSizeInMB" Value="8196"/> `
    <Property xsi:type="StringProperty" Name="StorageAccountType" Value="Premium_LRS"/> `
    <Property xsi:type="StringProperty" Name="LicenseType" Value="Windows_Client"/> `
    </CustomProperties>'
<!--NeedCopy-->

For information on how to create a catalog using the Remote PowerShell SDK, see Creating a catalog using PowerShell.

Constraints:

  • You can update the page file setting only when you create the provisioning scheme by running the New-ProvScheme command. You cannot change the page file setting later.
  • You must provide all the custom properties (‘PageFileDiskDriveLetterOverride’, ‘InitialPageFileSizeInMB’, and ‘MaxPageFileSizeInMB’) in the New-ProvScheme command or none of them.
  • This feature is not supported through Citrix Studio.
  • The initial page file size must be between 16 MB and 16777216 MB.
  • The maximum page file size must be greater than or equal to the initial page file size and less than 16777216 MB.
  • You can set both the initial page file size and maximum page file size to zero at the same time.

Azure throttling

Azure Resource Manager throttles requests for subscriptions and tenants, routing traffic based on defined limits, tailored to the specific needs of the provider. See Throttling Resource Manager requests on the Microsoft site for more information. Limits exist for subscriptions and tenants, where managing many machines can become problematic. For example, a subscription containing many machines might experience performance problems related to power operations.

Tip:

For more information, see Improving Azure performance with Machine Creation Services.

To help mitigate these issues, Citrix DaaS allows you to remove MCS internal throttling to use more of the available request quota from Azure.

We recommend the following optimal settings when powering VMs on or off in large subscriptions, for example, those containing 1,000 VMs:

  • Absolute simultaneous operations: 500
  • Maximum new operations per minute: 2000
  • Max concurrency of operations: 500

Use the Full Configuration interface to configure Azure operations for a given host connection:

  1. From Manage > Full Configuration, select Hosting in the left pane.
  2. Select an Azure-related connection to edit it.
  3. In the Edit Connection wizard, select Advanced.
  4. On the Advanced page, use the configuration options to specify the number of simultaneous actions and maximum new actions per minute, and any additional connection options.

Azure throttling

MCS supports 500 maximum concurrent operations by default. Alternatively, you can use the Remote PowerShell SDK to set the maximum number of concurrent operations.

Use the PowerShell property, MaximumConcurrentProvisioningOperations, to specify the maximum number of concurrent Azure provisioning operations. When using this property, consider:

  • Default value of MaximumConcurrentProvisioningOperations is 500.
  • Configure the MaximumConcurrentProvisioningOperations parameter using the PowerShell command Set-item.

Azure resource groups

Azure provisioning resource groups provide a way to provision the VMs that provide applications and desktops to users. You can add existing empty Azure resource groups when you create an MCS machine catalog, or have new resource groups created for you. For information about Azure resource groups, see the Microsoft documentation.

Azure Resource Group Usage

There is no limit on the number of virtual machines, managed disks, snapshots, and images per Azure Resource Group. (The limit of 240 VMs per 800 managed disks per Azure Resource Group has been removed.)

  • When using a full scope service principal to create a machine catalog, MCS creates only one Azure Resource Group and uses that group for the catalog.
  • When using a narrow scope service principal to create a machine catalog, you must supply an empty, pre-created Azure Resource Group for the catalog.

Azure ephemeral disks

An Azure ephemeral disk allows you to repurpose the cache disk or temporary disk to store the OS disk for an Azure-enabled virtual machine. This functionality is useful for Azure environments that require a higher performant SSD disk over a standard HDD disk. To use ephemeral disks, you must set the custom property UseEphemeralOsDisk to true when running New-ProvScheme.

Note:

If the custom property UseEphemeralOsDisk is set to false or a value is not specified all provisioned VDAs continue to use a provisioned OS disk.

The following is an example set of custom properties to use in the provisioning scheme:

"CustomProperties": [
            {
                "Name": "UseManagedDisks",
                "Value": "true"
            },
            {
                "Name": "StorageType",
                "Value": "Standard_LRS"
            },
            {
                "Name": "UseSharedImageGallery",
                "Value": "true"
            },
            {
                "Name": "SharedImageGalleryReplicaRatio",
                "Value": "40"
            },
            {
                "Name": "SharedImageGalleryReplicaMaximum",
                "Value": "10"
            },
            {
                "Name": "LicenseType",
                "Value": "Windows_Server"
            },
            {
                "Name": "UseEphemeralOsDisk",
                "Value": "true"
            }
        ],
<!--NeedCopy-->

How to create machines using ephemeral OS disks

Ephemeral OS disks are controlled based on the UseEphemeralOsDisk property in the CustomProperties parameter.

Important considerations for ephemeral disks

To provision ephemeral OS disks using New-ProvScheme, consider the following constraints:

  • The VM size used for the catalog must support ephemeral OS disks.
  • The size of the cache or temporary disk associated with the VM size must be greater than or equal to the size of the OS disk.
  • The temporary disk size must be greater than the cache disk size.

Also consider these issues when:

  • Creating the provisioning scheme.
  • Modifying the provisioning scheme.
  • Updating the image.

Azure ephemeral disk and Machine Creation Services (MCS) storage optimization (MCS I/O)

Azure ephemeral OS disk and MCS I/O cannot be enabled at the same time.

The important considerations are as follows:

  • You cannot create a machine catalog with both ephemeral OS disk and MCS I/O enabled at the same time.
  • In the Machine Catalog Setup wizard, if you select Azure ephemeral OS disk on the Storage and License Types page, you do not get the option for write-back cache disk settings on the Disk Settings page.

    Azure ephemeral OS disk selected

    Write-back cache disk settings not available

  • The PowerShell parameters (UseWriteBackCache and UseEphemeralOsDisk) set to true in New-ProvScheme or Set-ProvScheme fails with proper error message.
  • For existing machine catalogs created with both features enabled, you can still:
    • update a machine catalog.
    • add or delete VMs.
    • delete a machine catalog.

Azure server side encryption

Citrix DaaS supports customer-managed encryption keys for Azure managed disks through Azure Key Vault. With this support you can manage your organizational and compliance requirements by encrypting the managed disks of your machine catalog using your own encryption key. For more information, see Server-side encryption of Azure Disk Storage.

When using this feature for managed disks:

  • To change the key that the disk is encrypted with, you change the current key in the DiskEncryptionSet. All resources associated with that DiskEncryptionSet change to be encrypted with the new key.

  • When you disable or delete your key, any VMs with disks using that key automatically shut down. After shutting down, the VMs are not usable unless the key is enabled again or you assign a new key. Any catalog using the key cannot be powered on, and you cannot add VMs to it.

Important considerations when using customer-managed encryption keys

Consider the following when using this feature:

  • All resources related to your customer-managed keys (Azure Key Vaults, disk encryption sets, VMs, disks, and snapshots) must reside in the same subscription and region.

  • Once you have enabled the customer-managed encryption key that you cannot disable it later. If you want to disable or remove the customer-managed encryption key, copy all the data to a different managed disk that is not using the customer-managed encryption key.

  • Disks created from encrypted custom images using server-side encryption and customer-managed keys must be encrypted using the same customer-managed keys. These disks must be in the same subscription.

  • Snapshots created from disks that are encrypted with server-side encryption and customer-managed keys must be encrypted with the same customer-managed keys.

  • Disks, snapshots, and images encrypted with customer-managed keys cannot move to another resource group and subscription.

  • Managed disks currently or previously encrypted using Azure Disk Encryption cannot be encrypted using customer-managed keys.

  • Refer to the Microsoft site for limitations on disk encryption sets per region.

Note:

See Quickstart: Create a Key Vault using the Azure portal for information on configuring Azure server side encryption.

Azure Customer-managed encryption key

When creating a machine catalog, you can choose whether to encrypt data on the machines provisioned in the catalog. Server-side encryption with a customer-managed encryption key lets you manage encryption at a managed disk level and protect data on the machines in the catalog. A Disk Encryption Set (DES) represents a customer-managed key. To use this feature, you must first create your DES in Azure. A DES is in the following format:

  • /subscriptions/12345678-1234-1234-1234-123456789012/resourceGroups/Sample-RG/providers/Microsoft.Compute/diskEncryptionSets/SampleEncryptionSet

Select a DES from the list. The DES you select must be in the same subscription and region as your resources. If your image is encrypted with a DES, use the same DES when creating the machine catalog. You cannot change the DES after you create the catalog.

If you create a catalog with an encryption key and later disable the corresponding DES in Azure, you can no longer power on the machines in the catalog or add machines to it.

Azure dedicated hosts

You can use MCS to provision VMs on Azure dedicated hosts. Before provisioning VMs on Azure dedicated hosts:

  • Create a host group.
  • Create hosts in that host group.
  • Ensure that there is sufficient host capacity reserved for creating catalogs and virtual machines.

You can create a catalog of machines with host tenancy defined through the following PowerShell script:

New-ProvScheme <otherParameters> -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <Property xsi:type="StringProperty" Name="HostGroupId" Value="myResourceGroup/myHostGroup" />
 ...other Custom Properties...
 </CustomProperties>
<!--NeedCopy-->

When using MCS to provision virtual machines on Azure dedicated hosts, consider:

  • A Dedicated host is a catalog property and cannot be changed once the catalog is created. Dedicated tenancy is currently not supported on Azure.
  • A pre-configured Azure host group, in the region of the hosting unit, is required when using the HostGroupId parameter.
  • Azure auto-placement is required. This functionality makes a request to onboard the subscription associated with the host group. For more information, see VM Scale Set on Azure Dedicated Hosts - Public Preview. If auto-placement is not enabled, MCS throws an error during catalog creation.

Use Azure Shared Image Gallery as a published image repository for MCS provisioned machines in Azure. You can store a published image in the gallery to accelerate the creation and hydration of OS disks, improving start and application launch times for non-persistent VMs. Shared image gallery contains the following three elements:

  • Gallery. Images are stored here. MCS creates one gallery for each machine catalog.
  • Gallery Image Definition. This definition includes information (operating system type and state, Azure region) about the published image. MCS creates one image definition for each image created for the catalog.
  • Gallery Image Version. Each image in a Shared Image Gallery can have multiple versions, and each version can have multiple replicas in different regions. Each replica is a full copy of the published image. Citrix DaaS creates one Standard_LRS image version (version 1.0.0) for each image with the appropriate number of replicas in the catalog’s region, based on the number of machines in the catalog, the configured replica ratio, and the configured replica maximum.

Note:

Shared Image Gallery functionality only works with managed disks. It is not available for legacy machine catalogs.

For more information, see Azure shared image gallery overview.

Use the New-ProvScheme command to create a provisioning scheme with Shared Image Gallery support. Use the Set-ProvScheme command to enable or disable this feature for a provisioning scheme and to change the replica ratio and replica maximum values.

Three custom properties were added to provisioning schemes to support the Shared Image Gallery feature:

UseSharedImageGallery

  • Defines whether to use the Shared Image Gallery to store the published images. If set to True, the image is stored as a Shared Image Gallery image, otherwise the image is stored as a snapshot.
  • Valid values are True and False.
  • If the property is not defined, the default value is False.

SharedImageGalleryReplicaRatio

  • Defines the ratio of machines to gallery image version replicas.
  • Valid values are integer numbers greater than 0.
  • If the property is not defined, default values are used. The default value for persistent OS disks is 1000 and the default value for non-persistent OS disks is 40.

SharedImageGalleryReplicaMaximum

  • Defines the maximum number of replicas for each gallery image version.
  • Valid values are integer numbers greater than 0.
  • If the property is not defined, the default value is 10.
  • Azure currently supports up to 10 replicas for a gallery image single version. If the property is set to a value greater than that supported by Azure, MCS attempts to use the specified value. Azure generates an error, which MCS logs then leaves the current replica count unchanged.

Tip:

When using Shared Image Gallery to store a published image for MCS provisioned catalogs, MCS sets the gallery image version replica count based on the number of machines in the catalog, the replica ratio, and the replica maximum. The replica count is calculated by dividing the number of machines in the catalog by the replica ratio (rounding up to the nearest integer value) and then capping the value at the maximum replica count. For example, with a replica ratio of 20 and a maximum of 5, 0–20 machines have one replica created, 21–40 have 2 replicas, 41–60 have 3 replicas, 61–80 have 4 replicas, 81+ have 5 replicas.

The existing machine catalog uses Shared Image Gallery. Use the Set-ProvScheme command to update the custom properties for all existing machines in the catalog and any future machines:

Set-ProvScheme -ProvisioningSchemeName catalog-name -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Property xsi:type="StringProperty" Name="StorageType" Value="Standard_LRS"/> <Property xsi:type="StringProperty" Name="UseManagedDisks" Value="True"/> <Property xsi:type="StringProperty" Name="UseSharedImageGallery" Value="True"/> <Property xsi:type="IntProperty" Name="SharedImageGalleryReplicaRatio" Value="30"/> <Property xsi:type="IntProperty" Name="SharedImageGalleryReplicaMaximum" Value="20"/></CustomProperties>'
<!--NeedCopy-->

For this use case:

  1. Run Set-ProvScheme with the UseSharedImageGallery flag set to True. Optionally include the SharedImageGalleryReplicaRatio and SharedImageGalleryReplicaMaximum properties.
  2. Update the catalog.
  3. Power cycle the machines to force an update.

For example:

Set-ProvScheme -ProvisioningSchemeName catalog-name -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Property xsi:type="StringProperty" Name="StorageType" Value="Standard_LRS"/> <Property xsi:type="StringProperty" Name="UseManagedDisks" Value="True"/> <Property xsi:type="StringProperty" Name="UseSharedImageGallery" Value="True"/> <Property xsi:type="IntProperty" Name="SharedImageGalleryReplicaRatio" Value="30"/> <Property xsi:type="IntProperty" Name="SharedImageGalleryReplicaMaximum" Value="20"/></CustomProperties>'
<!--NeedCopy-->

Tip:

The parameters SharedImageGalleryReplicaRatio and SharedImageGalleryReplicaMaximum are not required. After the Set-ProvScheme command completes the Shared Image Gallery image has not yet been created. Once the catalog is configured to use the gallery, the next catalog update operation stores the published image in the gallery. The catalog update command creates the gallery, the gallery image, and the image version. Power cycling the machines updates them, at which point the replica count is updated, if appropriate. From that time, all existing non-persistent machines are reset using the Shared Image Gallery image and all newly provisioned machines are created using the image. The old snapshot is cleaned up automatically within a few hours.

For this use case:

  1. Run Set-ProvScheme with the UseSharedImageGallery flag set to False or not defined.
  2. Update the catalog.
  3. Power cycle the machines to force an update.

For example:

Set-ProvScheme -ProvisioningSchemeName catalog-name -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Property xsi:type="StringProperty" Name="StorageType" Value="Standard_LRS"/> <Property xsi:type="StringProperty" Name="UseManagedDisks" Value="True"/> <Property xsi:type="StringProperty" Name="UseSharedImageGallery" Value="False"/></CustomProperties>'
<!--NeedCopy-->

Tip:

Unlike updating from a snapshot to a Shared Image Gallery catalog, the custom data for each machine is not yet updated to reflect the new custom properties. Run the following command to see the original Shared Image Gallery custom properties: Get-ProvVm -ProvisioningSchemeName catalog-name. After the Set-ProvScheme command completes the image snapshot has not yet been created. Once the catalog is configured to not use the gallery, the next catalog update operation stores the published image as a snapshot. From that time, all existing non-persistent machines are reset using the snapshot and all newly provisioned machines are created from the snapshot. Power cycling the machines updates them, at which point the custom machine data is updated to reflect that UseSharedImageGallery is set to False. The old Shared Image Gallery assets (gallery, image, and version) are automatically cleaned up within a few hours.

Provision machines into specified Availability Zones

You can provision machines into specific Availability Zones in Azure environments. You can achieve that using the Full Configuration interface or PowerShell.

Note:

If no zones are specified, MCS lets Azure place the machines within the region. If more than one zone is specified, MCS randomly distributes the machines across them.

Configuring Availability Zones in the Full Configuration interface

When creating a machine catalog, you can specify Availability Zones into which you want to provision machines. On the Virtual Machines page, select one or more Availability Zones where you want to create machines.

There are two reasons that no Availability Zones are available: The region has no Availability Zones or the selected machine size is unavailable.

Configuring Availability Zones through PowerShell

Using PowerShell, you can view the Citrix DaaS offering inventory items by using Get-Item. For example, to view the Eastern US region Standard_B1ls service offering:

$serviceOffering = Get-Item -path "XDHyp:\Connections\my-connection-name\East US.region\serviceoffering.folder\Standard_B1ls.serviceoffering"
<!--NeedCopy-->

To view the zones, use the AdditionalData parameter for the item:

$serviceOffering.AdditionalData

If Availability Zones are not specified, there is no change in how machines are provisioned.

To configure Availability Zones through PowerShell, use the Zones custom property available with the New-ProvScheme operation. The Zones property defines a list of Availability Zones to provision machines into. Those zones can include one or more Availability Zones. For example, <Property xsi:type="StringProperty" Name="Zones" Value="1, 3"/> for Zones 1 and 3.

Use the Set-ProvScheme command to update the zones for a provisioning scheme.

If an invalid zone is provided, the provisioning scheme is not updated, and an error message appears providing instructions on how to fix the invalid command.

Tip:

If you specify an invalid custom property, the provisioning scheme is not updated and a relevant error message appears.

Azure ephemeral disk

Azure ephemeral disks allow you to repurpose the cache or temporary disk to store the OS disk for an Azure-enabled virtual machine. This functionality is useful for Azure environments that require a higher performant SSD disk over a standard HDD disk.

Note:

Persistent catalogs do not support ephemeral OS disks.

Ephemeral OS disks require that your provisioning scheme use managed disks and a Shared Image Gallery. For more information, see Azure shared image gallery.

Using PowerShell to configure an ephemeral disk

To configure an Azure ephemeral OS disk for a catalog, use the UseEphemeralOsDisk parameter in Set-ProvScheme. Set the value of the UseEphemeralOsDisk parameter to true.

Note:

To use this feature, you must also enable the parameters UseManagedDisks and UseSharedImageGallery.

For example:

Set-ProvScheme -ProvisioningSchemeName catalog-name -CustomProperties <CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" />
<Property xsi:type="StringProperty" Name="UseSharedImageGallery" Value="true" />
<Property xsi:type="StringProperty" Name="UseEphemeralOsDisk" Value="true" />
</CustomProperties>'
<!--NeedCopy-->

Storing an ephemeral OS temporary disk

You have the option of storing an ephemeral OS disk on the VM temp disk or a resource disk. This functionality enables you to use an ephemeral OS disk with a VM that either doesn’t have a cache, or has insufficient cache. Such VMs have a temp or resource disk to store an ephemeral OS disk, such as Ddv4.

Consider the following:

  • An ephemeral disk is stored either in the VM cache disk, or the VMs temporary (resource) disk. The cache disk is preferred over the temporary disk, unless the cache disk is not large enough to hold the contents of the OS disk.
  • For updates, a new image that is larger than the cache disk but smaller than the temp disk results in replacing the ephemeral OS disk with the VM’s temp disk.

Preserving a provisioned virtual machine when power cycling

Choose whether to preserve a provisioned virtual machine when power cycling. Use the PowerShell parameter New-ProvScheme CustomProperties. This parameter supports an extra property, PersistVm, used to determine if a provisioned virtual machine persists when power cycled. Set the PersistVm property to true to persist a virtual machine when powered off, or set the property to false to ensure that the virtual machine is not preserved when powered off.

Note:

The PersistVm property only applies to a provisioning scheme with the properties CleanOnBoot and UseWriteBackCache enabled. If the PersistVm property is not specified for non-persistent virtual machines, they are deleted from the Azure environment when powered off.

In the following example, the New-ProvScheme CustomProperties parameter sets the PersistVmproperty to true:

<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" />
<Property xsi:type="StringProperty" Name="StorageType" Value="Standard_LRS" />
<Property xsi:type="StringProperty" Name="PersistWBC" Value="false" />
<Property xsi:type="StringProperty" Name="PersistOsDisk" Value="true" />
<Property xsi:type="StringProperty" Name="PersistVm" Value="true" />
<Property xsi:type="StringProperty" Name="ResourceGroups" Value="demo-resourcegroup" />
<Property xsi:type="StringProperty" Name="LicenseType" Value="Windows_Client" />
</CustomProperties>
<!--NeedCopy-->

In the following example, the New-ProvScheme CustomProperties parameter preserves write back cache by setting PersistVM to true:

 New-ProvScheme
 -AzureAdJoinType "None"
 -CleanOnBoot
 -CustomProperties "<CustomProperties xmlns=`"http://schemas.citrix.com/2014/xd/machinecreation`" xmlns:xsi=`"http://www.w3.org/2001/XMLSchema-instance`"><Property xsi:type=`"StringProperty`" Name=`"UseManagedDisks`" Value=`"true`" /><Property xsi:type=`"StringProperty`" Name=`"StorageType`" Value=`"Standard_LRS`" /><Property xsi:type=`"StringProperty`" Name=`"PersistWBC`" Value=`"false`" /><Property xsi:type=`"StringProperty`" Name=`"PersistOsDisk`" Value=`"true`" /><Property xsi:type=`"StringProperty`" Name=`"PersistVm`" Value=`"true`" /><Property xsi:type=`"StringProperty`" Name=`"ResourceGroups`" Value=`"demo-resourcegroup`" /><Property xsi:type=`"StringProperty`" Name=`"LicenseType`" Value=`"Windows_Client`" /></CustomProperties>"
 -HostingUnitName "demo"
 -IdentityPoolName "NonPersistent-MCSIO-PersistVM"
 -MasterImageVM "XDHyp:\HostingUnits\demo\image.folder\scale-test.resourcegroup\demo-snapshot.snapshot"
 -NetworkMapping @ {"0"="XDHyp:\HostingUnits\demo\\virtualprivatecloud.folder\East US.region\virtualprivatecloud.folder\ji-test.resourcegroup\jitest-vnet.virtualprivatecloud\default.network"}
-ProvisioningSchemeName "NonPersistent-MCSIO-PersistVM"
 -ServiceOffering "XDHyp:\HostingUnits\demo\serviceoffering.folder\Standard_B2ms.serviceoffering" -UseWriteBackCache
 -WriteBackCacheDiskSize 127
 -WriteBackCacheMemorySize 256
 <!--NeedCopy-->

Tip:

The PersistVm property determines whether to preserve a provisioned virtual machine. The PersistOsdisk property determines whether to persist the OS disk. To preserve a provisioned virtual machine, preserve the OS disk first. You cannot delete the OS disk without first deleting the virtual machine. You can use the PersistOsdisk property without using specifying the PersistVm parameter.

Storage types

Select different storage types for virtual machines in Azure environments that use MCS. For target VMs, MCS supports:

  • OS disk: premium SSD, SSD, or HDD
  • Write back cache disk: premium SSD, SSD, or HDD

When using these storage types, consider the following:

  • Ensure that your VM supports the selected storage type.
  • If your configuration uses an Azure ephemeral disk, you do not get the option for write-back cache disk setting.

Tip:

StorageType is configured for an OS type and storage account. WBCDiskStorageType is configured for write back cache storage type. For a normal catalog, StorageType is required. If WBCDiskStorageType is not configured, the StorageType is used as the default for WBCDiskStorageType.

If WBCDiskStorageType is not configured, then StorageType is used as the default for WBCDiskStorageType

Configuring storage types

To configure storage types for VM, use the StorageType parameter in New-ProvScheme. Set the value of the StorageType parameter to one of the supported storage types.

The following is an example set of the CustomProperties parameter in a provisioning scheme:

Set-ProvScheme -ProvisioningSchemeName catalog-name -CustomProperties '<CustomProperties xmlns="http://schemas.citrix.com/2014/xd/machinecreation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Property xsi:type="StringProperty" Name="UseManagedDisks" Value="true" />
<Property xsi:type="StringProperty" Name="StorageType" Value="Premium_LRS" />
<Property xsi:type="StringProperty" Name="LicenseType" Value="Windows_Client" />
</CustomProperties>'
<!--NeedCopy-->

You can display information for an Azure VM, including OS disk and type, snapshot and gallery image definition. This information is displayed for resources on the master image when a machine catalog is assigned. Use this functionality to view and select either a Linux or Windows image. A PowerShell property, TemplateIsWindowsTemplate, was added to the AdditionDatafield parameter. This field contains Azure-specific information: VM type, OS disk, gallery image information, and OS type information. Setting TemplateIsWindowsTemplate to True indicates that the OS type is Windows; setting TemplateIsWindowsTemplate to False indicates that the OS type is Linux.

Tip:

Information displayed by the TemplateIsWindowsTemplate PowerShell property is derived from the Azure API. Sometimes, this field might be empty. For example, a snapshot from a data disk does not contain the TemplateIsWindowsTemplate field because the OS type cannot be retrieved from a snapshot.

For example, set the Azure VM AdditionData parameter to True for Windows OS type using PowerShell:

PS C:\Users\username> (get-item XDHyp:\HostingUnits\mynetwork\image.folder\username-dev-testing-rg.resourcegroup\username-dev-tsvda.vm).AdditionalData
Key Value
ServiceOfferingDescription Standard_B2ms
HardDiskSizeGB 127
ResourceGroupName FENGHUAJ-DEV-TESTING-RG
ServiceOfferingMemory 8192
ServiceOfferingCores 2
TemplateIsWindowsTemplate True
ServiceOfferingWithTemporaryDiskSizeInMb 16384
SupportedMachineGenerations Gen1,Gen2
<!--NeedCopy-->

Retrieve region name information for Azure VMs, managed disks, snapshots, Azure VHD, and ARM templates

You can display region name information for an Azure VM, managed disks, snapshots, Azure VHD, and ARM templates. This information is displayed for the resources on the master image when a machine catalog is assigned. A PowerShell property called RegionName displays the region name information when you run the PowerShell command with the AdditionalData parameter.

For example, use the following PowerShell command to get a VM information in Azure.

PS C:\Windows\system32> (get-item XDHyp:\HostingUnits\myAzureNetwork\image.folder\hu-dev-testing-rg.resourcegroup\hu-dev-tsvda.vm).AdditionalData
Key Value
HardDiskSizeGB 127
ResourceGroupName HU-DEV-TESTING-RG
RegionName East US
TemplateIsWindowsTemplate True
LicenseType
ServiceOfferingDescription Standard_B2ms
ServiceOfferingMemory 8192
ServiceOfferingCores 2
SupportedMachineGenerations Gen1,Gen2
ServiceOfferingWithTemporaryDiskSizeInMb 16384
SecurityType
SecureBootEnabled
VTpmEnabled
<!--NeedCopy-->

Azure Marketplace

Citrix DaaS supports using a master image on Azure that contains plan information to create a machine catalog. For more information, see Microsoft Azure Marketplace.

Tip:

Some images found on the Azure Marketplace, like the standard Windows Server image, do not append plan information. Citrix DaaS feature is for paid images.

Use the procedure in this section to view Shared Image Gallery images in Citrix Studio. These images can optionally be used for a master image. To put the image into a Shared Image Gallery, create an image definition in a gallery.

Azure Marketplace Shared Image Gallery

In the Publishing options page, verify the purchase plan information.

The purchase plan information fields are initially empty. Populate those fields with the purchase plan information used for the image. Failure to populate purchase plan information can cause the machine catalog process to fail.

Azure Marketplace verifies VDA publishing options

After verifying the purchase plan information, create an image version within the definition. This is used as the master image. Click Add version:

Azure Marketplace add VDA version

In the Version details section, select the image snapshot or managed disk as the source:

Azure Marketplace select VDA options

About Azure permissions

This section contains the minimum and general permissions required for Azure.

Minimum permissions

Minimum permissions give better security control. However, new features that require additional permissions will fail because of using only minimum permissions.

Creating a host connection

Add a new host connection using the information obtained from Azure.

"Microsoft.Network/virtualNetworks/read",
"Microsoft.Compute/virtualMachines/read",
"Microsoft.Compute/disks/read",
<!--NeedCopy-->

Power management of VMs

Power on or off the machine instances.

"Microsoft.Compute/virtualMachines/read",
"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Compute/virtualMachines/deallocate/action",
"Microsoft.Compute/virtualMachines/start/action",
"Microsoft.Compute/virtualMachines/restart/action",
<!--NeedCopy-->

Creating, updating, or deleting VMs

Create a machine catalog, then add, delete, update machines, and delete the machine catalog.

Following is the list of minimum permissions required when the master image is managed disk or snapshots are located in the same region as the hosting connection.

"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Resources/deployments/validate/action",
"Microsoft.Compute/virtualMachines/read",
"Microsoft.Compute/virtualMachines/write",
"Microsoft.Compute/virtualMachines/delete",
"Microsoft.Compute/virtualMachines/deallocate/action",
"Microsoft.Compute/snapshots/read",
"Microsoft.Compute/snapshots/write",
"Microsoft.Compute/snapshots/delete",
"Microsoft.Compute/snapshots/beginGetAccess/action",
"Microsoft.Compute/snapshots/endGetAccess/action",
"Microsoft.Compute/disks/read",
"Microsoft.Compute/disks/write",
"Microsoft.Compute/disks/delete",
"Microsoft.Compute/disks/beginGetAccess/action",
"Microsoft.Compute/disks/endGetAccess/action",
"Microsoft.Network/virtualNetworks/read",
"Microsoft.Network/virtualNetworks/subnets/join/action",
"Microsoft.Network/virtualNetworks/subnets/read",
"Microsoft.Network/networkSecurityGroups/read",
"Microsoft.Network/networkSecurityGroups/write",
"Microsoft.Network/networkSecurityGroups/delete",
"Microsoft.Network/networkSecurityGroups/join/action",
"Microsoft.Network/networkInterfaces/read",
"Microsoft.Network/networkInterfaces/write",
"Microsoft.Network/networkInterfaces/delete",
"Microsoft.Network/networkInterfaces/join/action",
<!--NeedCopy-->

You need the following extra permissions based on minimal permissions for the following features:

  • If the master image is a VHD in a storage account located in the same region as the hosting connection:

     "Microsoft.Storage/storageAccounts/read",
     "Microsoft.Storage/storageAccounts/listKeys/action",
     <!--NeedCopy-->
    
  • If the master image is an ImageVersion from the Shared Image Gallery:

     "Microsoft.Compute/galleries/read",
     "Microsoft.Compute/galleries/images/read",
     "Microsoft.Compute/galleries/images/versions/read",
     <!--NeedCopy-->
    
  • If the master image is a managed disk. Snapshots, or VHD is in a region different from the region of hosting connection:

     "Microsoft.Storage/storageAccounts/read",
     "Microsoft.Storage/storageAccounts/listKeys/action",
     "Microsoft.Storage/storageAccounts/write",
     "Microsoft.Storage/storageAccounts/delete",
     <!--NeedCopy-->
    
  • If you use Citrix-managed resource group:

     "Microsoft.Resources/subscriptions/resourceGroups/write",
     "Microsoft.Resources/subscriptions/resourceGroups/delete",
     <!--NeedCopy-->
    
  • If you put the master image in Shared Image Gallery:

     "Microsoft.Compute/galleries/write",
     "Microsoft.Compute/galleries/images/write",
     "Microsoft.Compute/galleries/images/versions/write",
     "Microsoft.Compute/galleries/read",
     "Microsoft.Compute/galleries/images/read",
     "Microsoft.Compute/galleries/images/versions/read",
     "Microsoft.Compute/galleries/delete",
     "Microsoft.Compute/galleries/images/delete",
     "Microsoft.Compute/galleries/images/versions/delete",
     <!--NeedCopy-->
    
  • If you use Azure dedicated host support:

     "Microsoft.Compute/hostGroups/read",
     "Microsoft.Compute/hostGroups/write",
     "Microsoft.Compute/hostGroups/hosts/read",
     <!--NeedCopy-->
    
  • If you use Server Side Encryption (SSE) with Customer Managed Keys (CMK):

     "Microsoft.Compute/diskEncryptionSets/read",
     <!--NeedCopy-->
    
  • If you deploy VMs using ARM templates (machine profile):

     "Microsoft.Resources/deployments/write",
     "Microsoft.Resources/deployments/operationstatuses/read",
     "Microsoft.Resources/deployments/read",
     "Microsoft.Resources/deployments/delete",
     <!--NeedCopy-->
    

Creating, updating, and deleting machines with unmanaged disk

Following is the list of minimum permissions required when the master image is VHD and use resource group as provided by admin:

"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Storage/storageAccounts/delete",
"Microsoft.Storage/storageAccounts/listKeys/action",
"Microsoft.Storage/storageAccounts/read",
"Microsoft.Storage/storageAccounts/write",
"Microsoft.Compute/virtualMachines/deallocate/action",
"Microsoft.Compute/virtualMachines/delete",
"Microsoft.Compute/virtualMachines/read",
"Microsoft.Compute/virtualMachines/write",
"Microsoft.Resources/deployments/validate/action",
"Microsoft.Network/networkInterfaces/delete",
"Microsoft.Network/networkInterfaces/join/action",
"Microsoft.Network/networkInterfaces/read",
"Microsoft.Network/networkInterfaces/write",
"Microsoft.Network/networkSecurityGroups/delete",
"Microsoft.Network/networkSecurityGroups/join/action",
"Microsoft.Network/networkSecurityGroups/read",
"Microsoft.Network/networkSecurityGroups/write",
"Microsoft.Network/virtualNetworks/subnets/read",
"Microsoft.Network/virtualNetworks/read",
"Microsoft.Network/virtualNetworks/subnets/join/action"
<!--NeedCopy-->

General permission

Contributor role has full access to manage all resources. This set of permissions does not block you from getting new features.

The following set of permissions provides the best compatibility going forward although it does include more permissions than needed with the current feature set:

"Microsoft.Compute/diskEncryptionSets/read",
"Microsoft.Compute/disks/beginGetAccess/action",
"Microsoft.Compute/disks/delete",
"Microsoft.Compute/disks/endGetAccess/action",
"Microsoft.Compute/disks/read",
"Microsoft.Compute/disks/write",
"Microsoft.Compute/galleries/delete",
"Microsoft.Compute/galleries/images/delete",
"Microsoft.Compute/galleries/images/read",
"Microsoft.Compute/galleries/images/versions/delete",
"Microsoft.Compute/galleries/images/versions/read",
"Microsoft.Compute/galleries/images/versions/write",
"Microsoft.Compute/galleries/images/write",
"Microsoft.Compute/galleries/read",
"Microsoft.Compute/galleries/write",
"Microsoft.Compute/hostGroups/hosts/read",
"Microsoft.Compute/hostGroups/read",
"Microsoft.Compute/hostGroups/write",
"Microsoft.Compute/snapshots/beginGetAccess/action",
"Microsoft.Compute/snapshots/delete",
"Microsoft.Compute/snapshots/endGetAccess/action",
"Microsoft.Compute/snapshots/read",
"Microsoft.Compute/snapshots/write",
"Microsoft.Compute/virtualMachines/deallocate/action",
"Microsoft.Compute/virtualMachines/delete",
"Microsoft.Compute/virtualMachines/read",
"Microsoft.Compute/virtualMachines/restart/action",
"Microsoft.Compute/virtualMachines/start/action",
"Microsoft.Compute/virtualMachines/write",
"Microsoft.Network/networkInterfaces/delete",
"Microsoft.Network/networkInterfaces/join/action",
"Microsoft.Network/networkInterfaces/read",
"Microsoft.Network/networkInterfaces/write",
"Microsoft.Network/networkSecurityGroups/delete",
"Microsoft.Network/networkSecurityGroups/join/action",
"Microsoft.Network/networkSecurityGroups/read",
"Microsoft.Network/networkSecurityGroups/write",
"Microsoft.Network/virtualNetworks/subnets/read",
"Microsoft.Network/virtualNetworks/read",
"Microsoft.Network/virtualNetworks/subnets/join/action",
"Microsoft.Resources/deployments/operationstatuses/read",
"Microsoft.Resources/deployments/read",
"Microsoft.Resources/deployments/validate/action",
"Microsoft.Resources/deployments/write",
"Microsoft.Resources/deployments/delete",
"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Resources/subscriptions/resourceGroups/write",
"Microsoft.Resources/subscriptions/resourceGroups/delete",
"Microsoft.Storage/storageAccounts/delete",
"Microsoft.Storage/storageAccounts/listKeys/action",
"Microsoft.Storage/storageAccounts/read",
"Microsoft.Storage/storageAccounts/write",
<!--NeedCopy-->

More information