Citrix Hypervisor 8.2

Thin provisioned shared GFS2 block storage

June 13, 2022

Contributed by:

Thin provisioning better utilizes the available storage by allocating disk storage space to VDIs as data is written to the virtual disk, rather than allocating the full virtual size of the VDI in advance. Thin provisioning enables you to significantly reduce the amount of space required on a shared storage array, and with that your Total Cost of Ownership (TCO).

Thin provisioning for shared block storage is of particular interest in the following cases:

You want increased space efficiency. Images are sparsely and not thickly allocated.
You want to reduce the number of I/O operations per second on your storage array. The GFS2 SR is the first SR type to support storage read caching on shared block storage.
You use a common base image for multiple virtual machines. The images of individual VMs will then typically utilize even less space.
You use snapshots. Each snapshot is an image and each image is now sparse.
Your storage does not support NFS and only supports block storage. If your storage supports NFS, we recommend you use NFS instead of GFS2.
You want to create VDIs that are greater than 2 TiB in size. The GFS2 SR supports VDIs up to 16 TiB in size.

The shared GFS2 type represents disks as a filesystem created on an iSCSI or HBA LUN. VDIs stored on a GFS2 SR are stored in the QCOW2 image format.

Prerequisites

Before you begin, ensure the following prerequisites are met:

All Citrix Hypervisor servers in the clustered pool must have at least 2 GiB of control domain memory.
All hosts in the cluster must use static IP addresses for the cluster network.
We recommend that you use clustering only in pools containing at least three hosts, as pools of two hosts are sensitive to self-fencing the entire pool.
If you have a firewall between the hosts in your pool, ensure that hosts can communicate on the cluster network using the following ports:
- TCP: 8892, 21064
- UDP: 5404, 5405
For more information, see Communication ports used by Citrix technologies.
If you are clustering an existing pool, ensure that high availability is disabled. You can enable high availability again after clustering is enabled.
We strongly recommend that you use a bonded network for your clustered pool that is not used for any other traffic.
You have a block-based storage device that is visible to all Citrix Hypervisor servers in the resource pool.

Set up a clustered pool to use a shared GFS2 SR

To use shared GFS2 storage, the Citrix Hypervisor resource pool must be a clustered pool. Enable clustering on your pool before creating a GFS2 SR.

Note:

Clustered pools behave differently to non-clustered pools. For more information about cluster behavior, see Clustered pools.

If you prefer, you can set up clustering on your pool by using XenCenter. For more information, see the XenCenter product documentation.

To use the xe CLI to create a clustered pool:

Create a bonded network to use as the clustering network.

Note:

We strongly recommend that you use a dedicated bonded network for your clustered pool. Do not use this network for any other traffic.

On the Citrix Hypervisor server that you want to be the pool master, complete the following steps:
1. Open a console on the Citrix Hypervisor server.
2. Name your resource pool by using the following command:
```
xe pool-param-set name-label="New Pool" uuid=<pool_uuid>
```
3. Create a network for use with the bonded NIC by using the following command:
```
xe network-create name-label=bond0
```
  The UUID of the new network is returned.
4. Find the UUIDs of the PIFs to use in the bond by using the following command:
```
xe pif-list
```
5. Create your bonded network in either active-active mode, active-passive mode, or LACP bond mode. Depending on the bond mode you want to use, complete one of the following actions:
  - To configure the bond in active-active mode (default), use the bond-create command to create the bond. Using commas to separate the parameters, specify the newly created network UUID and the UUIDs of the PIFs to be bonded:
    xe bond-create network-uuid=<network_uuid> / pif-uuids=<pif_uuid_1>,<pif_uuid_2>,<pif_uuid_3>,<pif_uuid_4>
    Type two UUIDs when you are bonding two NICs and four UUIDs when you are bonding four NICs. The UUID for the bond is returned after running the command.
  - To configure the bond in active-passive or LACP bond mode, use the same syntax, add the optional mode parameter, and specify lacp or active-backup:
    xe bond-create network-uuid=<network_uuid> pif-uuids=<pif_uuid_1>, / <pif_uuid_2>,<pif_uuid_3>,<pif_uuid_4> / mode=balance-slb | active-backup | lacp
After you have created your bonded network on the pool master, when you join other Citrix Hypervisor servers to the pool, the network and bond information is automatically replicated to the joining server.

For more information, see Networking.
Create a resource pool of at least three Citrix Hypervisor servers.

Repeat the following steps on each Citrix Hypervisor server that is a (non-master) pool member:
1. Open a console on the Citrix Hypervisor server.
2. Join the Citrix Hypervisor server to the pool on the pool master by using the following command:
```
xe pool-join master-address=master_address master-username=administrators_username master-password=password
```
  The value of the master-address parameter must be set to the fully qualified domain name of the Citrix Hypervisor server that is the pool master. The password must be the administrator password set when the pool master was installed.
For more information, see Hosts and resource pools.
For every PIF that belongs to this network, set disallow-unplug=true.
1. Find the UUIDs of the PIFs that belong to the network by using the following command:
```
xe pif-list
```
2. Run the following command on a Citrix Hypervisor server in your resource pool:
```
xe pif-param-set disallow-unplug=true uuid=<pif_uuid>
```
Enable clustering on your pool. Run the following command on a Citrix Hypervisor server in your resource pool:
```
xe cluster-pool-create network-uuid=<network_uuid>
```
Provide the UUID of the bonded network that you created in an earlier step.

Ensure that storage multipathing is set up between your clustered pool and your GFS2 SR. For more information, see Storage multipathing.

Create a shared GFS2 SR

You can create your shared GFS2 SR on an iSCSI or an HBA LUN.

Create a shared GFS2 over iSCSI SR

You can create GFS2 over iSCSI SRs by using XenCenter. For more information, see Software iSCSI storage in the XenCenter product documentation.

Alternatively, you can use the xe CLI to create a GFS2 over iSCSI SR.

Device-config parameters for GFS2 SRs:

Parameter Name	Description	Required?
`provider`	The block provider implementation. In this case, `iscsi`.	Yes
`target`	The IP address or hostname of the iSCSI filer that hosts	Yes
`targetIQN`	The IQN target of iSCSI filer that hosts the SR	Yes
`SCSIid`	Device SCSI ID	Yes

You can find the values to use for these parameters by using the xe sr-probe-ext command.

xe sr-probe-ext type=<type> host-uuid=<host_uuid> device-config:=<config> sm-config:=<sm_config>

Start by running the following command:
```
xe sr-probe-ext type=gfs2 device-config:provider=iscsi
```
The output from the command prompts you to supply additional parameters and gives a list of possible values at each step.
Repeat the command, adding new parameters each time.
When the command output starts with The following SRs were found:, you can use the device-config parameters that you specified to locate the SR when running the xe sr-create command.

To create a shared GFS2 SR on a specific LUN of an iSCSI target, run the following command on a server in your clustered pool:

xe sr-create type=gfs2 name-label="Example GFS2 SR" --shared \
   device-config:provider=iscsi device-config:targetIQN=target_iqns \
   device-config:target=portal_address device-config:SCSIid=scsci_id

If the iSCSI target is not reachable while GFS2 filesystems are mounted, some hosts in the clustered pool might fence.

For more information about working with iSCSI SRs, see Software iSCSI support.

Create a shared GFS2 over HBA SR

You can create GFS2 over HBA SRs by using XenCenter. For more information, see Hardware HBA storage in the XenCenter product documentation.

Alternatively, you can use the xe CLI to create a GFS2 over HBA SR.

Device-config parameters for GFS2 SRs:

Parameter name	Description	Required?
`provider`	The block provider implementation. In this case, `hba`.	Yes
`SCSIid`	Device SCSI ID	Yes

You can find the values to use for the SCSIid parameter by using the xe sr-probe-ext command.

xe sr-probe-ext type=<type> host-uuid=<host_uuid> device-config:=<config> sm-config:=<sm_config>

Start by running the following command:
```
xe sr-probe-ext type=gfs2 device-config:provider=hba
```
The output from the command prompts you to supply additional parameters and gives a list of possible values at each step.
Repeat the command, adding new parameters each time.
When the command output starts with The following SRs were found:, you can use the device-config parameters that you specified to locate the SR when running the xe sr-create command.

To create a shared GFS2 SR on a specific LUN of an HBA target, run the following command on a server in your clustered pool:

xe sr-create type=gfs2 name-label="Example GFS2 SR" --shared \
  device-config:provider=hba device-config:SCSIid=device_scsi_id

For more information about working with HBA SRs, see Hardware host bus adapters.

Constraints

Shared GFS2 storage currently has the following constraints:

As with any thin-provisioned SR, if the GFS2 SR usage grows to 100%, further writes from VMs fail. These failed writes can then lead to failures within the VM or possible data corruption or both.
XenCenter shows an alert when your SR usage grows to 80%. Ensure that you monitor your GFS2 SR for this alert and take the appropriate action if seen. On a GFS2 SR, high usage causes a performance degradation. We recommend that you keep your SR usage below 80%.
VM migration with storage migration (live or offline) is not supported for VMs whose VDIs are on a GFS2 SR. You also cannot migrate VDIs from another type of SR to a GFS2 SR.
The FCoE transport is not supported with GFS2 SRs.
Trim/unmap is not supported on GFS2 SRs.
CHAP is not supported on GFS2 SRs.
MCS full clone VMs are not supported with GFS2 SRs.
Using multiple GFS2 SRs in the same MCS catalog is not supported.
Performance metrics are not available for GFS2 SRs and disks on these SRs.
Changed block tracking is not supported for VDIs stored on GFS2 SRs.
You cannot export VDIs that are greater than 2 TiB as VHD or OVA/OVF. However, you can export VMs with VDIs larger than 2 TiB in XVA format.
We do not recommend using a thin provisioned LUN with GFS2. However, if you do choose this configuration, you must ensure that the LUN always has enough space to allow Citrix Hypervisor to write to it.
Clustered pools only support up to 16 hosts per pool.
For cluster traffic, you must use a bonded network that uses at least two different network switches. Do not use this network for any other purposes.
Changing the IP address of the cluster network by using XenCenter requires clustering and GFS2 to be temporarily disabled.
Do not change the bonding of your clustering network while the cluster is live and has running VMs. This action can cause the cluster to fence.
If you have an IP address conflict (multiple hosts having the same IP address) on your clustering network involving at least one host with clustering enabled, the hosts do not fence. To fix this issue, resolve the IP address conflict.

The official version of this content is in English. Some of the Cloud Software Group documentation content is machine translated for your convenience only. Cloud Software Group has no control over machine-translated content, which may contain errors, inaccuracies or unsuitable language. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, suitability, or correctness of any translations made from the English original into any other language, or that your Cloud Software Group product or service conforms to any machine translated content, and any warranty provided under the applicable end user license agreement or terms of service, or any other agreement with Cloud Software Group, that the product or service conforms with any documentation shall not apply to the extent that such documentation has been machine translated. Cloud Software Group will not be held responsible for any damage or issues that may arise from using machine-translated content.

Was this helpful

Thin provisioned shared GFS2 block storage

June 13, 2022

Contributed by:

June 13, 2022

Contributed by:

Thin provisioned shared GFS2 block storage

Prerequisites

Set up a clustered pool to use a shared GFS2 SR

Create a shared GFS2 SR

Create a shared GFS2 over iSCSI SR

Create a shared GFS2 over HBA SR

Constraints

In this article