Citrix DaaS Power Management of Azure Virtual Machines with REST API

Introduction

With Citrix DaaS, you can power manage Machine Creation Services (MCS) provisioned virtual machines across various supported hypervisors and cloud services. The power management operation provides you:

• Optimal user experience
• Cost management and power savings

In Azure environments, you can create an MCS machine catalog that supports hibernation. Using this feature, you can suspend a virtual machine when the session is idle for some time and then reconnect to the previous state of the virtual machine when a user signs in again.

Quote

Note:

The hibernation capability applies to only Single-session OS (persistent and non-persistent) machine catalogs. When hibernating a virtual machine, Azure signals the guest operating system to perform suspend-to-disk. During hibernation, Azure persists the memory (RAM) contents of the virtual machine in the OS disk and deallocates the virtual machine. Later, when starting the virtual machine, the RAM contents are restored back from the OS disk, and applications and processes that were previously running in the virtual machine resume from their last state.

This guide only covers the Microsoft Azure Hibernation functionality using REST API.

Once a virtual machine is in a hibernated/deallocated state, you are not billed for the virtual machine usage. You only pay for the storage (OS disks, data disks) and networking resources (IPs, and so forth) attached to the virtual machine. As a result, hibernating a virtual machine can be useful in scenarios such as:

1. Virtual desktops and development and test scenarios where the virtual machines can be hibernated/deallocated during non-business hours.

2. Applications that require considerable time to load due to memory components. These applications can be initialized on virtual machines and hibernated/deallocated. These pre-warmed virtual machines can then be quickly started when needed, with the applications already up and running in the desired state.

    

   

    

What happens when hibernating a virtual machine? When hibernation is triggered on a virtual machine using the Azure Portal, CLI, PS, SDKs, or APIs, Azure signals the guest operating system to perform suspend-to-disk (S4).

During hibernation, the contents of the RAM are saved on the OS disk, and the virtual machine is deallocated (that is, the virtual machine releases the lease on the underlying hardware and is powered off). Virtual machine states and billing have more details on the virtual machine deallocated state.

During hibernation, data in the temporary disk does not persist.

Once a virtual machine is hibernated/deallocated, the OS disk, data disks, and NICs remain attached to your virtual machine. Static IP addresses do not change when the virtual machine is suspended.

Supported virtual machine sizes

Virtual machine sizes with up to 32 GB RAM from the following virtual machine series support hibernation.

• Dasv5-series
• Dadsv5-series
• Dsv5-series
• Ddsv5-series

Supported Operating Systems

• Linux
• Ubuntu Server 22.04 LTS
• Ubuntu Server 20.04 LTS
• Ubuntu Server 18.04 LTS
• Debian 10 with backport kernel
• Windows
• Windows Server 2022
• Windows Server 2019
• Windows 11 Pro
• Windows 11 Enterprise
• Windows 11 Enterprise multi-session
• Windows 10 Pro
• Windows 10 Enterprise
• Windows 10 Enterprise multi-session

Known issues

• Debian 11 cannot be tested due to Azure issues.

Azure-related prerequisites to use hibernation

You must enable the feature for your subscription before you can use hibernation. See [Prerequisites to use hibernation](### Prerequisites to use hibernation). Hibernation can be enabled on the virtual machine at the time of creating the virtual machine.

• Hibernation must be supported by the virtual machine size.
• Hibernation must be supported by the OS.
• If a virtual machine is being created from an OS disk or a Compute Gallery image, then the OS disk or Gallery Image definition must support hibernation.
• You can use a persistent OS disk which is large enough to store the contents of the RAM, OS, and other applications running on the virtual machine.
• The virtual machine must have the Azure virtual machine Agent installed if you are using the Windows or Linux Hibernate Extensions.

Quote

Note:

MCS automatically sets the hibernation property for the generated resources. You do not need to configure the properties of the master resources to support hibernation.

Limitations

• You cannot enable hibernation on existing virtual machines.
• You cannot resize a virtual machine if it has hibernation enabled.
• When a virtual machine is hibernated/deallocated, you cannot attach/detach any disks or NICs to the virtual machine. To do so, you must move the virtual machine to a stopped/deallocated state by stopping the virtual machine and then attach/detach disks and NICs.
• When a virtual machine is hibernated/deallocated, you cannot modify the disks and NICs associated with the virtual machine. To do so, you must move the virtual machine to a stopped/deallocated state by stopping the virtual machine and then modify the disks and NICs.
• When a virtual machine is hibernated/deallocated, there is no capacity guarantee to ensure that there will be sufficient capacity to start the virtual machine later. In rare cases, if you encounter capacity issues, you can try starting the virtual machine later.
• If a virtual machine has a Capacity Reservation associated with it and is hibernated/deallocated, the Capacity Reservation does not ensure that the virtual machine will have the capacity to resume.
• You can only hibernate/deallocate a virtual machine using the Azure Portal, CLI, PowerShell, SDKs, and API. Hibernating the virtual machine using guest OS operations does not result in the virtual machine moving to a hibernated/deallocated state and the virtual machine remains billed.
• You cannot disable hibernation on a virtual machine once it's enabled.

The following are not supported with hibernation:

• Ephemeral OS disk
• Shared disk
• Availability Sets
• Virtual machines Uniform
• Spot virtual machines
• Managed images
• Azure Backup
• Capacity reservations

Windows Limitations:

• Applications such as Device Guard and Credential Guard that require virtualization-based security (VBS) are only supported with hibernation when Trusted Launch is enabled on the virtual machine and Nested Virtualization is enabled in the guest OS. Hibernation is supported with Nested Virtualization only when Trusted Launch is enabled on the virtual machine.
• The page file cannot be on the temp disk for a hibernation-capable VM

Linux Limitations:

• Hibernation is not supported with Trusted Launch for Linux virtual machines.

IMPORTANT:
The mentioned limitation of Windows "The page file cannot be on the temp disk for a hibernation-capable VM" is not valid for MCS-based deployments:
The page file of the Windows image can be on the temporary disk, MCS will move it to the C drive during image preparation!

Required permissions for Power Management

Minimum permissions give better security control. However, new features that require more permissions fail if only minimum permissions are given.

    "Microsoft.Compute/virtualMachines/read", "Microsoft.Resources/subscriptions/resourceGroups/read", "Microsoft.Compute/virtualMachines/deallocate/action", "Microsoft.Compute/virtualMachines/start/action", "Microsoft.Compute/virtualMachines/restart/action",

Citrix Cloud-related prerequisites to use hibernation

For Citrix DaaS to support Azure Hibernation, you need to enable two toggles in DaaS > Home > Preview features.

1. Ability to provision hibernation-capable virtual machines

2. Autoscale support for hibernation

    

   

    

Create hibernation-capable virtual machines and Machine Catalogs

In Azure environments, you can create an MCS machine catalog that supports hibernation. Using this feature, you can suspend a virtual machine when the session is idle for some time and then reconnect to the previous state of the virtual machine when a user signs in again.

Quote

Note:

The hibernation capability applies to only Single-session OS (persistent and non-persistent) machine catalogs.

To use the hibernation capability, you can do the following. However, before proceeding, see the prerequisites (link to be added) and limitations (link to be added).

• Create and manage a hibernation-capable machine catalog using the Full Configuration interface or PowerShell commands. You must select a machine profile that supports hibernation. (link to the topics to be added).
• Create a machine catalog for existing hibernation-capable virtual machines using the Full configuration interface. You can create a machine catalog containing both hibernation-capable and incapable virtual machines. However, if you want hibernation-related functionality, create the machine catalog with only hibernation-enabled virtual machines. (link to the topic to be added)
• Enable hibernation on existing MCS-provisioned virtual machines using PowerShell commands. (link to the topic to be added)
• Check the hibernation property of a machine catalog, virtual machine, and broker machine using PowerShell commands. (link to the topic to be added).

Prerequisites to use hibernation

To use hibernation, make sure to complete the following tasks on your Azure subscription:

1. Enable the feature for your subscription using the Azure Portal.

    

   

    

2. Check using REST-API-Call.

    

   

    

3. REST-API-Call to determine if Hibernation-Preview is enabled.

    GET https://management.azure.com/subscriptions/XXXXXXX-XXXX-XXXX-XXXXXXXXXX/providers/Microsoft.Features/providers/Microsoft.Compute/features/virtual machineHibernationPreview?api-version=2021-07-01 Authorization: Bearer  {{Bearer-Token-Value}}

1. REST-API-Call - Return if everything is correct.

    

   

    

2. If Hibernation-Preview is not enabled, register for the feature using a REST-API call.

    POST https://management.azure.com/subscriptions/XXXXXXX-XXXX-XXXX-XXXXXXXXXX/providers/Microsoft.Features/providers/Microsoft.Compute/features/virtual machineHibernationPreview/register?api-version=2021-07-01 Authorization: Bearer  {{Bearer-Token-Value}}

1. REST-API-Call - Return if everything is correct.

    

   

    

2. Install the Azure Virtual Machine Agent on the master image for both Windows and Linux. The page file of the Windows image can be on the temporary disk. MCS sets the page file location to the 😄 drive in the base disk when hibernation is enabled on the machine catalog.

3. On Windows-based virtual machines, the Agent is normally installed automatically.

4. To check if the Agent is installed, run the following PowerShell script.

   Get-Azvirtual machine  -Name "CTX-HibTest-M" -ResourceGroupName "HibTest" -status
   

    

   

    

    

   

    

5. Use a virtual machine size in your subscription that supports hibernation as detailed in Supported virtual machine sizes.

6. Create a hibernation-capable machine profile (virtual machine or template spec) so that virtual machines inherit the hibernation-capability. To create the virtual machine, see the Microsoft user guide on hibernation. To create the template spec, open the Azure Portal. Choose a virtual machine whose configuration you want to use in the template. Select Export template in the left pane.

    

   

    

7. Clear the Include parameters checkbox. Copy the context and save it as a JSON file, for example VMExportTemplate.json.

    

   

    

8. Modify the local template file VMExportTemplate.json. The parameter hibernationEnabled must be true. You can specify a supported virtual machine size. However, you can also specify the machine size while creating the catalog.

9. Add the template for the network interface resource to the JSON file VMExportTemplate.json. As a result, you have an ARM template file containing two resources.

    

   

    

10. Select Azure Portal > Template specs > Import template > Choose local template file to import this template file as an ARM template spec.

     

    

     

11. After the ARM Template specification is created, you can use it as a machine profile.

Quote

Note:

It might take a few minutes to sync to Citrix Studio.

Creating a Hibernation-enabled virtual machine using Azure Portal

Assuming all prerequisites are met, you can create a Hibernation-enabled Azure virtual machine using Azure Portal.

If you do not see the “Enable Hibernation” checkbox, the feature activation is not completed or failed. Keep in mind to choose the correct machine size otherwise an error occurs.

Creating a Hibernation-enabled virtual machine using REST-API

Assuming all prerequisites are met, the following REST-API calls create a Hibernation-enabled virtual machine - in this example Windows 11 Enterprise in the newest available version.

To access Microsoft Azure using REST-API calls, Postman is used as an API client.

1. The first step is creating a Bearer Token for Authentication.

    

   

    

2. Create a dedicated Resource Group (Recommended).

    

   

    

3. REST-API-Call to create a Resource Group.

       PUT https://management.azure.com/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/resourcegroups/HibTest-REST?api-version=2021-04-01 
   
       Authorization: Bearer  {{Bearer-Token-Value}}
   
       Body:
       {
       "location": "eastus"
       }
   

4. REST-API-Call - Return if everything is correct.

       {
           "id": "/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/resourceGroups/HibTest-REST",
           "name": "HibTest-REST",
           "type": "Microsoft.Resources/resourceGroups",
           "location": "eastus",
           "properties": {
               "provisioningState": "Succeeded"
           }
       }
   

5. Check which virtual machines are available for creation.

    

   

    

       GET
       https://management.azure.com/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/providers/Microsoft.Compute/locations/eastus/publishers/MicrosoftWindowsDesktop/artifacttypes/vmimage/offers?api-version=2023-07-01  
   
       Authorization: Bearer  {{Bearer-Token-Value}}
   

6. REST-API-Call - Return if everything is correct.

       [
           {
               "location": "eastus",
               "name": "office-365",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/office-365"
           },
           {
               "location": "eastus",
               "name": "test_sj_win_client",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/test_sj_win_client"
           },
           {
               "location": "eastus",
               "name": "Windows-10",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/Windows-10"
           },
           {
               "location": "eastus",
               "name": "windows-10-1607-vhd-client-prod-stage",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-10-1607-vhd-client-prod-stage"
           },
           {
               "location": "eastus",
               "name": "windows-10-20h2-vhd-client-prod-stage",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-10-20h2-vhd-client-prod-stage"
           },
           {
               "location": "eastus",
               "name": "windows-10-ppe",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-10-ppe"
           },
           {
               "location": "eastus",
               "name": "windows-11",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11"
           },
           {
               "location": "eastus",
               "name": "windows-7",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-7"
           },
           {
               "location": "eastus",
               "name": "windows-ent-cpc",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-ent-cpc"
           },
           {
               "location": "eastus",
               "name": "windows-evd",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-evd"
           },
           {
               "location": "eastus",
               "name": "windows10preview",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows10preview"
           },
           {
               "location": "eastus",
               "name": "windows11preview",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows11preview"
           },
           {
               "location": "eastus",
               "name": "windows11preview-arm64",
               "id": "/Subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows11preview-arm64"
           }
       ]
   

7. Check which SKUs are available for creation.

    

   

    

       GET
       https://management.azure.com/subscriptions/XXXXXXXXX/providers/Microsoft.Compute/locations/eastus/publishers/MicrosoftWindowsDesktop/artifacttypes/vmimage/offers/windows-11/skus?api-version=2023-07-01 
   
       Authorization: Bearer  {{Bearer-Token-Value}}
   

8. REST-API-Call - Return if everything is correct.

       [
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-avd",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-avd"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-ent",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-ent"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-entn",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-entn"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-pro",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-pro"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-pro-zh-cn",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-pro-zh-cn"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-21h2-pron",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-21h2-pron"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-avd",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-avd"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-ent",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-ent"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-entn",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-entn"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-pro",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-pro-zh-cn",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro-zh-cn"
           },
           {
               "properties": {
                   "automaticOSUpgradeProperties": {
                       "automaticOSUpgradeSupported": false
                   }
               },
               "location": "eastus",
               "name": "win11-22h2-pron",
               "id": "/Subscriptions/XXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pron"
           }
       ]
   

9. Check the correct Image-Name for creation.

    

   

    

   Quote

   Note

   Windows 11 ENT is multi-session capable and is not supported, and Windows 11 PRO and ENT are usable.

       GET
       https://management.azure.com/subscriptions/XXXXXXXXXXX/providers/Microsoft.Compute/locations/eastus/publishers/MicrosoftWindowsDesktop/artifacttypes/vmimage/offers/windows-11/skus/win11-22h2-pro/versions?$top=5&api-version=2023-07-01 
   
       Authorization: Bearer  {{Bearer-Token-Value}}
   

10. REST-API-Call - Return if everything is correct:

        [
            {
                "location": "eastus",
                "name": "22621.1848.230621",
                "id": "/Subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro/Versions/22621.1848.230621"
            },
            {
                "location": "eastus",
                "name": "22621.1992.230708",
                "id": "/Subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro/Versions/22621.1992.230708"
            },
            {
                "location": "eastus",
                "name": "22621.2134.230801",
                "id": "/Subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro/Versions/22621.2134.230801"
            },
            {
                "location": "eastus",
                "name": "22621.2283.230901",
                "id": "/Subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro/Versions/22621.2283.230901"
            },
            {
                "location": "eastus",
                "name": "22621.2428.231001",
                "id": "/Subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/Providers/Microsoft.Compute/Locations/eastus/Publishers/MicrosoftWindowsDesktop/ArtifactTypes/virtual machineImage/Offers/windows-11/Skus/win11-22h2-pro/Versions/22621.2428.231001"
            }
        ]
    

11. Now we have all the parameters to create the virtual machine.

     

    

     

        PUT
        https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXX/resourceGroups/HibTest-REST/providers/Microsoft.Compute/virtualMachines/HibTest-REST?api-version=2023-07-01 
    
        Authorization: Bearer  {{Bearer-Token-Value}}
    
        Body:
    
        {
        "location": "eastus",
        "properties": {
            "hardwareProfile": {
            "vmSize": "Standard_D2s_v5"
            },
            "additionalCapabilities": {
            "hibernationEnabled": true
            },
            "storageProfile": {
            "imageReference": {
                "publisher": "MicrosoftWindowsDesktop",
                "offer": "windows-11",
                "sku": "win11-22h2-pro",
                "version": "latest"
            },
            "osDisk": {
                "caching": "ReadWrite",
                "managedDisk": {
                "storageAccountType": "Standard_LRS"
                },
                "name": "vmOSdisk",
                "createOption": "FromImage"
            }
            },
            "networkProfile": {
            "networkApiVersion": "2020-11-01",
            "networkInterfaceConfigurations": [
                {
                "name": "NW-HibTest-REST",
                "properties": {
                    "primary": true,
                    "deleteOption": "Delete",
                    "ipConfigurations": [
                    {
                        "name": "IP-HibTest-REST",
                        "properties": {
                            "subnet": {
                        "id": "/subscriptions/XXXXXXXXXXXXX/resourceGroups/Hibtest-REST/providers/Microsoft.Network/virtualNetworks/TACG-HibTest-DCCC-vnet/subnets/default"
                            },
                        "primary": true,
                        "publicIPAddressConfiguration": {
                            "name": "PubIP-HibTest-REST",
                            "sku": {
                            "name": "Standard",
                            "tier": "Global"
                            },
                            "properties": {
                            "deleteOption": "Detach",
                            "publicIPAllocationMethod": "Static"
                            }
                        }
                        }
                    }
                    ]
                }
                }
            ]
            },
            "osProfile": {
            "adminUsername": "XXXXXXXXX",
            "computerName": "HibTest-REST-M",
            "adminPassword": "XXXXXXXXXX"
            }
    
        }
        }
    

12. REST-API-Call - Return if everything is correct.

        {
            "name": "HibTest-REST",
            "id": "/subscriptions/XXXXXXXXXXXXXXX/resourceGroups/HibTest-REST/providers/Microsoft.Compute/virtualMachines/HibTest-REST",
            "type": "Microsoft.Compute/virtualMachines",
            "location": "eastus",
            "properties": {
                "hardwareProfile": {
                    "vmSize": "Standard_D2s_v5"
                },
                "provisioningState": "Creating",
                "vmId": "7f7e3238-XXXXXXXXXXXXXXXXXXXXXXXXXXX",
                "additionalCapabilities": {
                    "hibernationEnabled": true
                },
                "storageProfile": {
                    "imageReference": {
                        "publisher": "MicrosoftWindowsDesktop",
                        "offer": "windows-11",
                        "sku": "win11-22h2-pro",
                        "version": "latest",
                        "exactVersion": "22621.2428.231001"
                    },
                    "osDisk": {
                        "osType": "Windows",
                        "name": "vmOSdisk",
                        "createOption": "FromImage",
                        "caching": "ReadWrite",
                        "managedDisk": {
                            "storageAccountType": "Standard_LRS"
                        },
                        "deleteOption": "Detach",
                        "diskSizeGB": 127
                    },
                    "dataDisks": [],
                    "diskControllerType": "SCSI"
                },
                "osProfile": {
                    "computerName": "HibTest-REST",
                    "adminUsername": "azadmin",
                    "windowsConfiguration": {
                        "provisionvirtual machineAgent": true,
                        "enableAutomaticUpdates": true,
                        "patchSettings": {
                            "patchMode": "AutomaticByOS",
                            "assessmentMode": "ImageDefault"
                        },
                        "enablevirtual machineAgentPlatformUpdates": false
                    },
                    "secrets": [],
                    "allowExtensionOperations": true,
                    "requireGuestProvisionSignal": true
                },
                "networkProfile": {
                    "networkInterfaces": [
                        {
                            "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/HibTest-REST/providers/Microsoft.Network/networkInterfaces/NW-HibTest-REST-8f52559f",
                            "properties": {
                                "primary": true,
                                "deleteOption": "Delete"
                            }
                        }
                    ]
                },
                "timeCreated": "2023-10-24T07:09:07.8074523+00:00"
            },
            "resources": [
                {
                    "name": "AzureHibernateExtension",
                    "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/HibTest-REST/providers/Microsoft.Compute/virtualMachines/HibTest-REST/extensions/AzureHibernateExtension",
                    "type": "Microsoft.Compute/virtualMachines/extensions",
                    "location": "eastus",
                    "properties": {
                        "autoUpgradeMinorVersion": true,
                        "forceUpdateTag": "41295d67-42f2-413d-99b8-770ba6247922",
                        "provisioningState": "Created",
                        "enableAutomaticUpgrade": false,
                        "publisher": "Microsoft.CPlat.Core",
                        "type": "WindowsHibernateExtension",
                        "typeHandlerVersion": "1.0",
                        "settings": {
                            "allowReboot": true
                        }
                    }
                }
            ]
        }
    

13. Now create the Machine Catalog.

Creating a Hibernation-capable Machine Catalog using the Citrix Cloud GUI

The following screenshots show the steps for creating a Hibernation-capable Machine Catalog.

1.  

   

    

2.  

   

    

3.  

   

    

4.  

   

    

5.  

   

    

6.  

   

    

7.  

   

    

8.  

   

    

9. Hovering over the virtual machine-based Machine Profile reveals that Hibernation is supported on this profile.

10.  

    

     

11.  

    

     

12.  

    

     

13.  

    

     

14.  

    

     

15.  

    

     

16.  

    

     

17.  

    

     

    Creation of the Machine Catalog is complete.

18. To determine the Hibernation capabilities of the Machine Catalog and the virtual machines in the MC we run some checks using PowerShell.

        //Check Hibernation capability of Provisioning scheme just created
    
        (Get-ProvScheme -ProvisioningSchemeName MC-AZ-HibTest).virtual machineMetadata -join "" | ConvertFrom-Json | Select HibernationEnabled
    
        //Check Hibernation capability of virtual machine(s) in MC
    
        (Get-Provvirtual machine -virtual machineName "HibTest-1").Customvirtual machinedata | ConvertFrom-Json | Select SupportsHibernation
    

     

    

     

Creating a Hibernation-capable Machine Catalog using REST-API

After you meet all the requirements to use hibernation, you can create a hibernation-capable machine catalog using Rest-APIs. The following prerequisites must be met:

• Ensure that you have a valid bearer token.
• Ensure that you have the siteid How to get the site id.
• Ensure you have created a connection and resource pool in hosting.
• Ensure you have a domain administrator account encoded in UTF-8 and encoded as Base64 string. You must set the X-AdminCredential using the encoded credentials.

REST-API-Call to create an Azure-based Machine Catalog

1. As this call is done asynchronously we only get a “202 Accepted” response. For determining the progress or if an error occurred we need to periodically call the Job progress.

    

   

    

2. REST-API-Call to create an Azure-based Machine Catalog.

       POST
       https://api-eu.cloud.com/cvad/manage/MachineCatalogs?async=true 
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
       X-AdminCredential: Basic MjFceDIx…
   
       Body:
       {
           "Name": "MC-AZ-HIB-NAME",
           "AllocationType":"Static",
           "MinimumFunctionalLevel":"LMAX",
           "PersistUserChanges":"OnLocal",
           "ProvisioningType":"MCS",
           "IsPowerManaged": true,
           "SessionSupport":"SingleSession",
           "Scopes":["00000000-0000-0000-0000-000000000000"],
           "Tenants":[],
           "Zone":"The Austrian Citrix Guy - AzHib",
           "VdaUpgradeType":"NotSet",
           "ProvisioningScheme":{
               "MasterImagePath":"XDHyp:\\HostingUnits\\NW-TACG-XXXXX\\image.folder\\CTX-Hibtest2.resourcegroup\\CTX-HibTest-M_OsDisk.manageddisk",
               "CpuCount":null,
               "MemoryMB":null,
               "UseWriteBackCache":false,
               "NumTotalMachines":1,
               "NetworkMapping":[
                   {"DeviceNameOrId":"default","NetworkDeviceNameOrId":"0",
                   "NetworkPath":"XDHyp:\\HostingUnits\\NW-TACG-XXXXX\\virtualprivatecloud.folder\\East US.region\\virtualprivatecloud.folder\\CTX-Hibtest2.resourcegroup\\TACG-HibTest-DCCC-vnet.virtualprivatecloud\\default.network"
                   }],
                   "Metadata": [
                   {
                       "Name": "SupportsHibernation",
                       "Value": "True"
                   }
               ],
               "IdentityType":"ActiveDirectory",
               "MachineAccountCreationRules":{
                   "NamingScheme":"MCS-REST-W11-#","NamingSchemeType":"Numeric","Domain":"hib.the-austrian-citrix-guy.at","OU":"CN=Computers,DC=hib,DC=the-austrian-citrix-guy,DC=at"},
               "MachineProfilePath":"XDHyp:\\HostingUnits\\NW-TACG-XXXXX\\machineprofile.folder\\CTX-Hibtest2.resourcegroup\\CTX-HibTest-M.vm",
               "PrepareImage":true,
               "DedicatedTenancy":false,
               "SecurityGroups":null,
               "UseFullDiskCloneProvisioning":false,
               "CustomProperties":[{"Name":"UseManagedDisks","Value":"true"},{"Name":"OsType","Value":"Windows"},{"Name":"StorageType","Value":"Premium_LRS"},{"Name":"LicenseType","Value":"Windows_Server"},{"Name":"Zones","Value":"1"}],
               "ServiceOfferingPath":"XDHyp:\\HostingUnits\\NW-TACG-XXXXX\\serviceoffering.folder\\Standard_D2s_v5.serviceoffering"},
               "AdminFolder":"0"
       }
   

3. REST-API-Call - Return if everything is correct.

    

   

    

4. Get the Job ID to being able to monitor the progress. Therefore we have to search all jobs to find the current job ID. Due to the number of jobs returned we can search the response for Type “CreateMachineCatalog” and the date and time.

    

   

    

5. REST-API-Call to get all jobs to determine current job ID.

   GET
   https://api-eu.cloud.com/cvad/manage/Jobs  
   
   Authorization:      Bearer  {{Bearer-Token-Value}}
   Citrix-CustomerId: {{Citrix-CustomerID}}
   Citrix-InstanceID: {{Citrix-SiteID}}
   

6. REST-API-Call - Return if everything is correct - look for “CreateMachineCatalog” and current date/time.

   },
       {
       "Id": "44696345-7762-46bb-9d17-a9e7e149e876",
       "Type": "CreateMachineCatalog",
       "OverallProgressPercent": 100,
       "IsCancellable": false,
       "Parameters": [
            {
               "Name": "name",
               "Value": "MC-EX-AZ-TZ"
           },
           {
               "Name": "request",
                   Value": ""
           },
           {
               "Name": "ProgressDescriptionContents",
               "Value": "Created 1 of 1 virtual machines."
           }
           ],
           "SubJobs": [
           {
               "Id": "00000000-0000-0000-0000-000000000000",
               "Type": "Unknown",
               "OverallProgressPercent": 100,
               "IsCancellable": false,
               "Parameters": [
                   {
                       "Name": "ProgressDescriptionContents",
                       "Value": "Gathering required information"
                   }
               ],
               "SubJobs": null,
               "Status": "Complete",
               "ResultLocation": null,
               "ErrorString": null,
               "ErrorCode": "Unknown",
               "ErrorParameters": null,
               "CreationTime": "2023-11-08T13:22:36.504+00:00",
               "FormattedCreationTime": "2023-11-08T13:22:36Z",
               "StartTime": "2023-11-08T13:22:36.614+00:00",
               "FormattedStartTime": "2023-11-08T13:22:36Z",
               "EndTime": "2023-11-08T13:22:36.848+00:00",
               "FormattedEndTime": "2023-11-08T13:22:36Z"
               },
   

7. The ID parameter contains the current job ID we can further use.

8. Call the job id to see the current status.

    

   

    

9. REST-API-Call to get the status of the current job ID.

   GET
   https://api-eu.cloud.com/cvad/manage/Jobs/44696345-xxxxxx 
   
   Authorization:      Bearer  {{Bearer-Token-Value}}
   Citrix-CustomerId: {{Citrix-CustomerID}}
   Citrix-InstanceID: {{Citrix-SiteID}}
   

10. REST-API-Call - Return - the value of “Status” shows if the job completed successfully or not. If “Status” is “Complete” then the creation of the Machine Catalog is completed. If it is not, scroll through the response to determine where an error and which error occurred.

        {
        "Id": "44696345-7762-46bb-9d17-a9e7e149e876",
        "Type": "CreateMachineCatalog",
        "OverallProgressPercent": 100,
        "IsCancellable": false,
        "Parameters": [
            {
                "Name": "name",
                "Value": "MC-EX-AZ-TZ"
            },
            {
                "Name": "request",
                "Value": XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            },
            {
                "Name": "ProgressDescriptionContents",
                "Value": "Created 1 of 1 virtual machines."
            }
        ],
        "SubJobs": [
            {
                "Id": "00000000-0000-0000-0000-000000000000",
                "Type": "Unknown",
                "OverallProgressPercent": 100,
                "IsCancellable": false,
                "Parameters": [
                    {
                        "Name": "ProgressDescriptionContents",
                        "Value": "Gathering required information"
                    }
                ],
                "SubJobs": null,
                "Status": "Complete",
                "ResultLocation": null,
                "ErrorString": null,
                "ErrorCode": "Unknown",
                "ErrorParameters": null,
                "CreationTime": "2023-11-08T13:22:36.504+00:00",
                "FormattedCreationTime": "2023-11-08T13:22:36Z",
                "StartTime": "2023-11-08T13:22:36.614+00:00",
                "FormattedStartTime": "2023-11-08T13:22:36Z",
                "EndTime": "2023-11-08T13:22:36.848+00:00",
                "FormattedEndTime": "2023-11-08T13:22:36Z"
            },
            {
                "Id": "00000000-0000-0000-0000-000000000000",
                "Type": "Unknown",
                "OverallProgressPercent": 100,
                "IsCancellable": false,
                "Parameters": [
                    {
                        "Name": "ProgressDescriptionContents",
                        "Value": "Creating Machine Catalog"
                    }
                ],
                "SubJobs": null,
                "Status": "Complete",
                "ResultLocation": null,
                "ErrorString": null,
                "ErrorCode": "Unknown",
                "ErrorParameters": null,
                "CreationTime": "2023-11-08T13:22:36.536+00:00",
                "FormattedCreationTime": "2023-11-08T13:22:36Z",
                "StartTime": "2023-11-08T13:22:36.848+00:00",
                "FormattedStartTime": "2023-11-08T13:22:36Z",
                "EndTime": "2023-11-08T13:22:37.651+00:00",
                "FormattedEndTime": "2023-11-08T13:22:37Z"
            },
            {
                "Id": "00000000-0000-0000-0000-000000000000",
                "Type": "Unknown",
                "OverallProgressPercent": 100,
                "IsCancellable": false,
                "Parameters": [
                    {
                        "Name": "ProgressDescriptionContents",
                        "Value": "Creating Identity Pool"
                    }
                ],
                "SubJobs": null,
                "Status": "Complete",
                "ResultLocation": null,
                "ErrorString": null,
                "ErrorCode": "Unknown",
                "ErrorParameters": null,
                "CreationTime": "2023-11-08T13:22:36.551+00:00",
                "FormattedCreationTime": "2023-11-08T13:22:36Z",
                "StartTime": "2023-11-08T13:22:37.866+00:00",
                "FormattedStartTime": "2023-11-08T13:22:37Z",
                "EndTime": "2023-11-08T13:22:42.857+00:00",
                "FormattedEndTime": "2023-11-08T13:22:42Z"
            },
            {
                "Id": "00000000-0000-0000-0000-000000000000",
                "Type": "Unknown",
                "OverallProgressPercent": 100,
                "IsCancellable": false,
                "Parameters": [
                    {
                        "Name": "ProgressDescriptionContents",
                        "Value": "Copying the master image"
                    }
                ],
                "SubJobs": null,
                "Status": "Complete",
                "ResultLocation": null,
                "ErrorString": null,
                "ErrorCode": "Unknown",
                "ErrorParameters": null,
                "CreationTime": "2023-11-08T13:22:36.599+00:00",
                "FormattedCreationTime": "2023-11-08T13:22:36Z",
                "StartTime": "2023-11-08T13:22:43.056+00:00",
                "FormattedStartTime": "2023-11-08T13:22:43Z",
                "EndTime": "2023-11-08T13:30:31.09+00:00",
                "FormattedEndTime": "2023-11-08T13:30:31Z"
            },
            {
                "Id": "00000000-0000-0000-0000-000000000000",
                "Type": "AddMachineCatalogMachine",
                "OverallProgressPercent": 100,
                "IsCancellable": false,
                "Parameters": [
                    {
                        "Name": "ProgressDescriptionContents",
                        "Value": "Created 1 of 1 virtual machines."
                    }
                ],
                "SubJobs": null,
                "Status": "Complete",
                "ResultLocation": null,
                "ErrorString": null,
                "ErrorCode": "Unknown",
                "ErrorParameters": null,
                "CreationTime": "2023-11-08T13:22:36.614+00:00",
                "FormattedCreationTime": "2023-11-08T13:22:36Z",
                "StartTime": "2023-11-08T13:30:31.293+00:00",
                "FormattedStartTime": "2023-11-08T13:30:31Z",
                "EndTime": "2023-11-08T13:31:07.65+00:00",
                "FormattedEndTime": "2023-11-08T13:31:07Z"
            }
        ],
        "Status": "Complete",
        "ResultLocation": "/Jobs/44696345-7762-46bb-9d17-a9e7e149e876/Results",
        "ErrorString": null,
        "ErrorCode": "Unknown",
        "ErrorParameters": [],
        "CreationTime": "2023-11-08T13:22:25.086+00:00",
        "FormattedCreationTime": "2023-11-08T13:22:25Z",
        "StartTime": "2023-11-08T13:22:25.102+00:00",
        "FormattedStartTime": "2023-11-08T13:22:25Z",
        "EndTime": "2023-11-08T13:31:08.265+00:00",
        "FormattedEndTime": "2023-11-08T13:31:08Z"
    }
    

11. This completes the creatoin of an Azure-based Machine Catalog using REST-API calls.

Editing/Updating a Machine Catalog

If you already have hibernation-capable virtual machines and want to use DaaS to suspend and resume them, create machine catalogs to import those virtual machines to DaaS for power management.

Quote

Note:

You can create a machine catalog containing both hibernation-capable and -incapable virtual machines. However, if you want hibernation-related functionality, you must create the machine catalog with only hibernation-capable virtual machines.

If the current Machine Catalog supports Hibernation, you cannot:

• Change the virtual machine size Service Offering to a Hibernation-incapable size
• Change the Machine Profile to a Hibernation-incapable profile

If the current Machine Catalog does not support Hibernation, you can:

• Change the machine profile to a Hibernate-enabled profile using the Full Configuration interface or use PowerShell commands.

You can enable Azure Hibernation on the following existing Machine Catalogs:

• Windows MCS-provisioned virtual machines of a machine catalog created without a temporary disk.
• Linux MCS-provisioned virtual machines of a machine catalog created with and without a temporary disk.

Quote

Note:

The existing MCS-provisioned virtual machines must have an Azure virtual machine Agent installed (see above).

Check hibernation property

You can check the hibernation property of a machine catalog, virtual machine, and a broker machine using the Full Configuration interface:

Virtual machines deployed in a Machine Catalog created without a Machine Profile results in Hibernation being unsupported.

1. We can change the Provisioning Scheme of the Machine Catalog to use a Machine Profile.

       Set-ProvScheme -provisioningSchemeName 'MC-AZ-HibTest-NoHib' -machineProfile 'XDHyp:\HostingUnits\NW-TACG-HibTestNN\machinepro
       file.folder\CTX-Hibtest2.resourcegroup\CTX-HibTest-M.vm' -ServiceOffering 'XDHyp:\HostingUnits\NW-TACG-HibTestNN\serviceoffering.folder\Standard_D2
       s_v5.serviceoffering'
   

2. Request update on existing virtual machines in a machine catalog.

       Set-Provvirtual machineUpdateTimeWindow -ProvisioningSchemeName` 'MC-AZ-HibTest-NoHib' -virtual machineName 'HibTest-NoHib-1`
   

3. Restart the virtual machine.

       New-BrokerHostingPowerAction -MachineName 'HibTest-NoHib-1' -Action Restart
   

    

   

    

4. Check if MC is now enabled to support Hibernation.

       (Get-ProvScheme -provisioningSchemeName 'MC-AZ-HibTest-NoHib').virtual machineMetadata -join "" |  ConvertFrom-Json | Select HibernationEna
       bled
   

    

   

    

    

   

    

5. Check if the virtual machine is now enabled to support Hibernation.

       (Get-Provvirtual machine -virtual machineName 'HibTest-NoHib-1').CustomVmData | ConvertFrom-Json | Select SupportsHibernation
   

    

   

    

    

   

    

    

   

    

We updated an existing Machine Catalog which initially had no support for Azure Hibernation using PowerShell to finally support Azure Hibernation.

How to use Hibernation feature

Hibernation-capable virtual machines can be hibernated/deallocated in two different ways:

1. Initiate Hibernation of the virtual machine via the Studio action menu

   • Full Configuration interface:
     • Select a virtual machine in the hibernation-capable machine catalog and select Start. -After the Power State is On, right-click the virtual machine, and select Suspend. Click Yes to confirm the action. -The Power State changes from Suspending to Suspended. -You can check the status of the virtual machine in the Azure portal.

2. To resume and reconnect back to a hibernated/deallocated virtual machine, use one of the following:

   • Administrators can resume the virtual machine using the Full Configuration interface.
   • End users can start the virtual machine using the Citrix Workspace menu.

Quote

Note:

You can do the following power management operations on the hibernated/deallocated virtual machines: (Suspend virtual machine from the running state, Resume virtual machine from the suspended state, Force shut down virtual machine from a suspended state, and Force restart virtual machine from the suspended state)

Enabling and Configuring Autoscale using the Full Configuration Interface

Citrix Autoscale provides a consistent, high-performance solution to proactively power manages your machines. It aims to balance costs and user experience. Autoscale incorporates the deprecated Smart Scale technology into the Manage console’s power management solution.

Autoscale enables proactive power management of all registered single-session and multi-session OS machines in a delivery group. It supports all platforms that Citrix DaaS supports - Citrix Hypervisor, AWS, Google Cloud Platform, Microsoft Azure Resource Manager, virtual machineware vSphere, and many more.

Autoscale powers on and off machines in a delivery group based on a schedule that you set. A schedule includes the number of active machines for each time slot, with peak and off-peak times defined.

Autoscale supports both multi-session OS and single-session OS delivery groups. There are three user interfaces to be aware of:

• Autoscale user interface for multi-session OS delivery groups (formerly RDS delivery groups)
• Autoscale user interface for single-session OS random (pooled) delivery groups (formerly pooled VDI delivery groups)
• Autoscale user interface for single-session OS static delivery groups (formerly static VDI delivery groups)

For managing virtual machines with Azure Hibernation enabled keep in mind that currently only single-session OS delivery groups are supported.

Enabling Autoscale

1. You can enable Autoscale by choosing the corresponding Delivery group and click Manage Autoscale.

    

   

    

2. In the next window click Enable autoscale.

    

   

    

Basic concepts of Citrix Autoscale for Azure Hibernation-enabled virtual machines.

Schedules:

• Autoscale powers machines on and off based on the selected schedule. Autoscale lets you set multiple schedules that include specific days of the week and adjust the number of machines available during those times. If you expect a set of users to consume the machine resources at a specific time on specific days, Autoscale helps provide an optimized experience.
• The schedule is based on the time zone of the delivery group.
• Autoscale treats only those machines that are registered with the site as part of the available capacity in the calculations it makes. “Registered” means that the machine is available for use or already in use. Doing so ensures that only machines that can accept user sessions are included in the capacity for the delivery group.

Quote

Note:

Those machines are powered on during the schedule, whether there are sessions running on them.

Capacity buffer:

• A Capacity buffer is used to add spare capacity to the current demand to account for dynamic load increases.

Autoscale lets you set the capacity buffer separately for peak and off-peak times. A lesser value in the capacity buffer field decreases the cost because Autoscale powers on less spare capacity. A greater value ensures an optimized user experience so that users do not have to wait for more machines to power on when launching sessions. By default, the capacity buffer is 10%. For single-session OS delivery groups, the capacity buffer is defined as a percentage of the total number of machines in the delivery group.

Quote

Note:

The capacity buffer results in machines being powered on when the total spare capacity drops to a level below “X” percent of the total capacity of the delivery group. Doing so reserves the required percentage of spare capacity.

Power policies:

• The Power policies control the behavior of the Autoscale depending on Session status:
  • The session on the virtual machine is disconnected. The virtual machine is put into hibernation or shuts down when the specified disconnection time elapses, depending on the action you configured (the action to choose to put the machine into Hibernation is “Suspend”). By default, no action is assigned to disconnected machines. You can define actions separately for peak and off-peak times.

The session is logged off:

• The virtual machine is suspended or shut down when the specified logoff time elapses, depending on the actions you configured (the action to choose to put the machine into Hibernation is “Suspend”). By default, no action is assigned to logged-off machines. You can define actions separately for peak and off-peak times.

Setting the Peak Times on the schedule:

• You can set the Peak Times in 30min-intervals by clicking in the time-schedule. Each blue frame represents a timeslot marked as Peak Time. The Peak Times need not to be consecutive, non-consecutive timeslots are possible.

• Non-consecutive timeslots are possible.

Enabling and Configuring Autoscale using Rest API

In the following snippets we assume that the Bearer Token for Authentication of each REST Call is valid.

1. First we check whether Autoscale is already activated in the respective desktop group.

    

   

    

       GET
       https://api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
   
       Response shortened:
       {
           "AppAccessPolicy": null,
           "AppProtectionKeyLoggingRequired": false,
           "AppProtectionScreenCaptureRequired": false,
           "AutomaticPowerOnForAssigned": true,
           "AutomaticPowerOnForAssignedDuringPeak": false,
           "AutoScaleEnabled": true,
           "RestrictAutoscaleTag": null,
           "ColorDepth": "TwentyFourBit",
           "DefaultDesktopIconId": "1",
           "DefaultDesktopPublishedName": "BG-AZ-HibTest",
           "DesktopsInUse": 0,
           "DesktopsNeverRegistered": 0,
           "DesktopsPreparing": 0,
           "HdxSslEnabled": null,
           "IsPowerManaged": true,
           "LingerSettings": }
   

2. If Autoscale is not enabled, you can enable it using a PATCH call.

    

   

    

       PATCH
       https://api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
       Raw Body: 
       {
       "AutoScaleEnabled": "true"
       }
   
       Response:
       Response code 204 No Content
   

3. Check if a schedule is already configured.

    

   

    

       GET
       api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest/PowerTimeSchemes
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
   
       Response:
       {
           "Items": [
               {
                   "DaysOfWeek": [
                       "Saturday",
                       "Sunday"
                   ],
                   "Name": "BG-AZ-HibTest_Weekend",
                   "DisplayName": "Weekend",
                   "PeakHours": null,
                   "PeakTimeRanges": [],
                   "PoolSize": null,
                   "PoolSizeSchedule": [],
                   "PoolUsingPercentage": false,
                   "Id": "28"
               },
               {
                   "DaysOfWeek": [
                       "Monday",
                       "Tuesday",
                       "Wednesday",
                       "Thursday",
                       "Friday"
                   ],
                   "Name": "PSWeekdays",
                   "DisplayName": "PSWeekdays",
                   "PeakHours": null,
                   "PeakTimeRanges": [
                       "08:00-19:00"
                   ],
                   "PoolSize": null,
                   "PoolSizeSchedule": null,
                   "PoolUsingPercentage": false,
                   "Id": "33"
               }
           ]
       }
   

   A new schedule can only be configured if there are no coincidences regarding the Peak Time settings otherwise an error occurs!

4. Assuming the Peak Times are not set, a schedule can be created.

    

   

    

       POST
       api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest/PowerTimeSchemes
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
       Raw Body: 
       {
       "DaysOfWeek": [
           "Monday",
           "Tuesday",
           "Wednesday",
           "Thursday",
           "Friday"
       ],
       "Name": "APIWeekdays",
       "DisplayName": "APIWeekdays",
       "PeakTimeRanges": [
           "09:00-17:00"
       ],
       "PoolSizeSchedule": [
           {
           "TimeRange": "00:00-08:00",
           "PoolSize": 0
           }
       ],
       "PoolUsingPercentage": false
       }
   
       Response:
       {
           "DaysOfWeek": [
               "Monday",
               "Tuesday",
               "Wednesday",
               "Thursday",
               "Friday"
           ],
           "Name": "APIWeekdays",
           "DisplayName": "APIWeekdays",
           "PeakHours": null,
           "PeakTimeRanges": [
               "09:00-17:00"
           ],
           "PoolSize": null,
           "PoolSizeSchedule": [],
           "PoolUsingPercentage": false,
           "Id": "39"
       }
   

5. Check if Power Policies are set.

    

   

    

       GET
       api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest/
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
       Raw Body: 
       {
       }
   
       Response (shortened):
       "MachineOperatingSystems": [
               {
                   "Name": "Windows 11",
                   "Value": 1
               }
           ],
           "MachineType": "Unknown",
           "OffMachines": 0,
           "OffPeakBufferSizePercent": 0,
           "OffPeakDisconnectAction": "Suspend",
           "OffPeakDisconnectTimeoutMinutes": 1,
           "OffPeakExtendedDisconnectAction": "Nothing",
           "OffPeakExtendedDisconnectTimeoutMinutes": 0,
           "OffPeakLogOffAction": "Suspend",
           "OffPeakLogOffTimeoutMinutes": 1,
           "PeakAutoscaleAssignedPowerOnIdleTimeoutMinutes": 0,
           "PeakAutoscaleAssignedPowerOnIdleAction": "Nothing",
           "PeakBufferSizePercent": 0,
           "PeakDisconnectAction": "Suspend",
           "PeakDisconnectTimeoutMinutes": 1,
           "PeakExtendedDisconnectAction": "Nothing",
           "PeakExtendedDisconnectTimeoutMinutes": 0,
           "PeakLogOffAction": "Suspend",
           "PeakLogOffTimeoutMinutes": 1,
   

6. If Power Policies must be set.

    

   

    

7. A response code of “204 No Content” means successful completion.

       PATCH
       https://api-eu.cloud.com/cvad/manage/Deliverygroups/BG-AZ-HibTest
   
       Authorization:      Bearer  {{Bearer-Token-Value}}
       Citrix-CustomerId: {{Citrix-CustomerID}}
       Citrix-InstanceID: {{Citrix-SiteID}}
       Raw Body: 
       {
       "OffPeakDisconnectAction": "Suspend",
       "OffPeakDisconnectTimeoutMinutes": 2,
       "OffPeakLogOffAction": "Suspend",
       "OffPeakLogOffTimeoutMinutes": 2,
       "PeakDisconnectAction": "Suspend",
       "PeakDisconnectTimeoutMinutes": 2,
       "PeakLogOffAction": "Suspend",
       "PeakLogOffTimeoutMinutes": 2
       }
   
       Response:
       Response code 204 No Content 
   

   Now all relevant Autoscale settings for hibernating Azure-based virtual machines are set and active.

Troubleshooting guide

Unable to create a virtual machine with hibernation enabled

If you are unable to create a virtual machine with hibernation enabled, ensure that you are using a virtual machine size, OS version that supports Hibernation. Refer to the supported virtual machine sizes, OS versions section in the user guide in addition to the limitations section for more details. Some common error codes that you might observe include:

Unable to hibernate a virtual machine

If you are unable to hibernate/deallocate a virtual machine, first check whether hibernation is enabled on the virtual machine. For example, using the GET virtual machine API, you can check if hibernation is enabled on the virtual machine.

    "properties": {
            "vmId": "XXX",
            "hardwareProfile": {
                "vmSize": "Standard_D4s_v5"
            },
            "additionalCapabilities": {
                "hibernationEnabled": true
            },

If hibernation is enabled on the virtual machine, check if hibernation is successfully enabled in the guest OS.

For Windows, you can check the status of the Hibernation extension to see if the extension was able to successfully configure the guest OS for hibernation.

The virtual machine instance view would have the final output of the extension :

Also, confirm that hibernation is enabled as a sleep state inside the guest. The expected output for the guest will look like this.

If hibernate is not listed as a supported sleep state, there is a reason associated with it which will help determine why hibernate is not supported. For example, the following would happen if guest hibernation has not been configured for the virtual machine.

If either the extension, or the guest sleep state reports an error, you need to update the guest configurations as per the error descriptions to resolve the issue. After fixing all the issues, you can validate that hibernation has been enabled successfully inside the guest by running the powercfg /a command - which returns Hibernate as one of the sleep states. Also validate that the AzureHibernateExtension returns to a Succeeded state. If the extension is still in a failed state, then you need to update the extension state by triggering the Reapply virtual machine API.

Quote

Note:

If the extension remains in a failed state, you will not can hibernate the virtual machine.

Commonly seen issues where the extension fails

Windows Guest Unable to Hibernate

If a hibernation operation succeeds, the following events are seen in the guest:

If the guest fails to hibernate/deallocate, then all or some of these events are missing.

Commonly Seen Issues

Unable to Resume a Virtual Machine

Starting a hibernated/deallocated virtual machine is similar to starting a stopped/deallocated virtual machine. For errors and troubleshooting steps related to starting a virtual machine, refer to this guide.

In addition to commonly seen issues while starting virtual machines, certain issues are specific to starting a hibernated/deallocated virtual machine.

Windows guest resume status through virtual machine instance view

For Windows virtual machines, when you start a virtual machine from a hibernated/deallocated state, you can use the virtual machine instance view to get more details on whether the guest successfully resumed from its previous hibernated/deallocated state or if it failed to resume and instead did a cold boot.

1. When the guest successfully resumes, the virtual machine instance view output is:

    

   

   

    

2. If the Windows guest fails to resume from its previous state and instead cold boots, then the virtual machine instance view response is:

    

   

    

Windows Guest events while resuming

If a guest successfully resumes, the following guest events are available:

If the guest fails to resume, all or some of these events are missing.

To troubleshoot why the guest failed to resume, the following logs are needed:

1. Event logs on the guest: Microsoft-Windows-Kernel-Power, Microsoft-Windows-Kernel-General, Microsoft-Windows-Kernel-Boot.
2. On bug check, a guest crash dump is needed.

FAQs

1. What are the charges for using this feature?

   • Once a virtual machine is placed in a hibernated/deallocated state, you are not charged for the virtual machine, just like how you are not charged for virtual machines in a stopped/deallocated state. You are only charged for the OS disk, data disks and any static IPs associated with the virtual machine.

2. Can I enable hibernation on existing virtual machines?

   • No, you cannot enable hibernation on existing virtual machines. You can only enable hibernation at the time of creating a virtual machine.

3. Can I resize a virtual machine with hibernation enable?

   • No. Once you enable hibernation on a virtual machine, you cannot resize the virtual machine.

4. Can I modify a virtual machine once it is in a hibernated/deallocated state?

   • No, once a virtual machine is in a hibernated/deallocated state, you cannot perform actions like resizing the virtual machine and modifying the disks. Also, you cannot detach any disks or networking resources that are currently attached to the virtual machine or attach new resources to the virtual machine. You can however stop/deallocate or delete the virtual machine if you want to detach these resources.

5. What is the difference between stopping/deallocating and hibernating/deallocating a virtual machine?

   • When you stop/deallocate a virtual machine, the virtual machine shuts down without persisting the memory contents. You can resize a stopped/deallocated virtual machine and detach/attach disks to the virtual machine. When you hibernate/deallocate a virtual machine, the memory contents are first persisted in the OS disk and then the virtual machine hibernates. You cannot resize virtual machines in a hibernate/deallocated state, nor detach/attach disks and networking resources to the virtual machine.

6. Can you disable hibernation?

   • No, you cannot disable hibernation on a virtual machine.

7. Can I initiate hibernation from within the virtual machine?

   • To hibernate/deallocate a virtual machine you can use the Azure Portal, CLI, PowerShell commands, SDKs, APIs, and Citrix Autoscale. Triggering hibernation from inside the virtual machine will still result in your virtual machine being billed for the compute resources.

8. When a virtual machine is hibernated, is there a capacity assurance at the time of starting the virtual machine?

   • No, there is no capacity assurance for starting hibernated/deallocated virtual machines. In rare scenarios if you encounter a capacity issue, then you can try starting the virtual machine later.