Container management

XenServer includes two new features to enhance deployments of Docker Containers on XenServer

  • Support for CoreOS Linux VMs and configuring Cloud Config Drives

  • Container Management for CoreOS, Debian 8, Ubuntu 14.04, and RHEL/CentOS/OEL 7

  • Preview of Container Management for Windows Server Containers on Windows Server 2016 Technology Preview

CoreOS is a minimalist Linux distribution which has become popular for hosting Docker applications. The CoreOS Cloud Config Drive allows the customization of various operating system configuration options. When Container Management is enabled on a VM, XenServer becomes aware of any Docker containers running in the VM.

Note:

For information on how to install CoreOS guests, configure Cloud-Config parameters, and manage Docker containers, see the XenCenter Help. Press F1 or click Help.

The Container Management Supplemental Park enables XenServer to take the following actions:

  • Query the VMs
  • Interact with Cloud Config Drives
  • Discover application containers
  • Display application containers within XenCenter’s Infrastructure view.

XenCenter also enables interaction with the containers to allow for start, stop and pause operations, and other monitoring capabilities. For more information, see Container Management Supplemental Pack.

What is Docker

Docker is an open platform for developers and system administrators to build, ship, and run distributed applications. A Docker container comprises just the application and its dependencies. It runs as an isolated process in user space on the host operating system, sharing the kernel and base filesystem with other containers. For more information, see https://www.docker.com/whatisdocker.

Note:

The XenServer Container Management feature complements, but not does replace the Docker environment. You can use one of the many Docker management tools available to manage individual Docker Engine instances in the VMs.

Container Management Supplemental Pack

The Container Management Supplemental Pack provides:

Monitoring and Visibility: allows you to see which VMs are in use for Docker hosting, and which containers on the VM are running.

Diagnostics: access is provided to basic container information such as forwarded network ports, and originating Docker image name. This feature can help accelerate investigations into problems where either the infrastructure and applications layers maybe impacted.

Performance: gives insight into which containers are running on that VM. Depending on the information provided by the operating system, it provides information on the processes and applications running on the container, and the CPU resource consumed.

Control Applications: enables you to use XenCenter to start, stop, and pause (if supported by the operating system) application containers enabling rapid termination of problematic applications.

Note:

XenServer supports installing Supplemental Packs using XenCenter. For information on how to install a supplemental pack using XenCenter, see the XenCenter Help. If you would prefer to install using the xe CLI, see the XenServer Supplemental Packs and the DDK guide.

Manage Docker containers by using XenCenter

This section contains information on managing your CoreOS VMs using XenCenter. To manage CoreOS VMs, complete the following steps:

  1. Install or upgrade your host to XenServer 7.6.

  2. Install the XenCenter shipped with XenServer 7.6.

  3. Install the Container Management Supplemental pack available from the Citrix website.

  4. Create a CoreOS VM and include a config drive for the VM.

    When you create a CoreOS VM in XenCenter, the New VM wizard prompts you to specify cloud-config parameters for your VM. The config drive provides user data for the VM instance. If you are planning to use XenServer to manage containers running inside the VM, create a config drive.

    By default, XenCenter includes a predefined set of parameters on the Cloud-Config Parameters page. You can modify these parameters based on your requirements. For detailed information about supported configuration parameters, see the CoreOS documentation.

    Warning:

    Container Management may not work if you do not create a config drive for the VM.

  5. Enable container management for the VM. You can update this setting on the VM’s Properties tab in XenCenter.

Note:

If you migrate a Container Managed VM between pools, Container Management stops working for the VM. This behavior is because Container Management is implemented using a pool-specific key. To enable Container Management functionality again for the VM, update the Cloud Config Drive configuration in the VM preferences.

Manage containers on other Linux guests

CoreOS VMs that are created with the default Cloud Config Drive configuration are automatically prepared for Container Management. Just only need to enable the feature. Other Linux guests can be prepared manually. This feature is supported for Debian 8, Ubuntu 14.04, and RHEL/CentOS/OEL 7.x VMs only.

To prepare a Linux guest manually:

  1. Ensure that the VM has XenServer Tools installed, and that the VM network is configured as described in Network Requirements and Security.

  2. Install Docker, Ncat and SSHD inside the VM.

    For Ubuntu 14.04: apt-get install docker.io nmap openssh-server

    For RHEL/CentOS/OEL 7.x: yum install docker nmap openssh-server

  3. Enable autostart for docker.service:

    systemctl enable docker.service

  4. Start docker.service

    systemctl start docker.service

    Use a non-root user for container management. Add the user to the ‘docker’ group to provide access to Docker.

  5. Prepare the VM for container management; run the following command on the control domain (dom0) on one of the hosts in the pool:

    xscontainer-prepare-vm -v vm_uuid -u username

    Where vm_uuid is the VM to be prepared, and username is the user name on the VM that the Container Management uses for management access.

The preparation script guides you through the process and automatically enables container management for this VM.

Note:

If you migrate a Container Managed VM between pools, Container Management stops working for the VM. This behavior is because Container Management is implemented using a pool-specific key. To enable Container Management functionality again for the VM, run the xscontainer-prepare-vm command again on the VM. Even after running this command, the original XenServer pool might keep access to the VM.

Access the Docker Container console and logs

For Linux VMs, XenCenter enables customers to access the container console and view logs to manage and monitor applications running on Docker containers. To access the container console and logs using XenCenter:

  1. Select the container in the Resources pane.

  2. On the Container General Properties section, click View Console to view the container console. To see the console logs, click View Log. This action opens an SSH client on the machine running XenCenter.

  3. When prompted, log into the SSH client using the VM user name and password.

    Note:

    Customers can automate the authentication process by configuring their public/private SSH keys. See the following section for details.

Automate the authentication process (optional)

When accessing the container console and logs, customers are required to enter the login credentials of the VM to authenticate SSH connections. However, customers can automate the authentication process to avoid entering the credentials manually. Follow the instructions below to configure the automatic authentication process:

  1. Generate a public/private key pair.

  2. Add the public SSH key to the user directory on the VM running the container.

    • For containers running on a CoreOS VM, add the public key to the Cloud-Config Parameters section on the VM’s General tab in XenCenter.
    • For containers running on Ubuntu 14.04, RHEL/CentOS/Oracle Linux 7, and Debian 8, manually add the public key to ~/.ssh/authorized_keys.
  3. Add the private SSH key to the %userprofile% directory on the machine running XenCenter and rename the key as ContainerManagement.ppk.

Manage Windows Server Containers

Windows Server Containers are part of the Windows Server 2016 guest operating system. They allow the encapsulation of Windows applications by isolating processes into their own namespace. XenServer Container Management supports monitoring and managing Windows Server Containers on Windows Server 2016 guest operating systems.

Note:

Windows Server 2016 VMs must be configured with one or more static IP addresses for TLS communication, as TLS server certificates are bound to certain IP addresses.

To prepare Windows Server Containers for Container Management:

  1. Ensure that the VM has XenServer Tools installed, and that the VM network is configured as described in Network Requirements and Security.

  2. Install Windows Server Container support inside the VM as described in Microsoft Documentation. Windows Server Containers are not Hyper-V Containers.

  3. Create a file called daemon.json in the folder C:\ProgramData\docker\config with the contents:

          {
            "hosts": ["tcp://0.0.0.0:2376", "npipe://"],
            "tlsverify": true,
            "tlscacert": "C:\ProgramData\docker\certs.d\ca.pem",
            "tlscert": "C:\ProgramData\docker\certs.d\server-cert.pem",
            "tlskey": "C:\ProgramData\docker\certs.d\server-key.pem"
          }
    
  4. Prepare the VM for container management; run one of the following commands on the control domain (dom0) on one of the hosts in the pool:

    Option 1 (for single-user VMs): Have XenServer generate TLS certificates for this VM.

    Important:

    This option is only safe where only a single user has access to the VM. The TLS server and client keys are injected into the VM using a virtual CD. This information can be copied by malicious users during the preparation.

    xscontainer-prepare-vm -v vm_uuid -u root --mode tls --generate-certs
    

    Where vm_uuid is the VM to be prepared. Follow the on-screen instructions to complete the process of preparing Windows Server Containers. It involves interacting with dom0 and the VM.

    Option 2: To configure XenServer with externally generated TLS certificates

    xscontainer-prepare-vm -v vm_uuid -u root --mode tls \
    --client-cert client_cert --client-key client_key --ca-cert ca_cert
    

    Where vm_uuid is the VM to be prepared, client_cert is the TLS client certificate, client_key is the TLS client key, and ca_cert is the CA certificate. This option assumes that Docker is already configured for TLS inside the VM.

Network requirements and security

Important:

For container management to work, it may be necessary to relax security requirements regarding network isolation.

For maximum security of virtualization environments, Citrix recommends that administrators partition the network by isolating XenServer’s management network (with XenServer Control Domain) from the VMs.

Enabling container management requires a route between these two networks, which increases the risk of malicious VMs attacking the management network (that is, dom0). To mitigate the risk of allowing traffic between VM and the management network, we advise the configuration of firewall rules to allow only trusted sources to initiate a connection between the two networks.

Do not use this feature in production in the following cases:

  • If this recommended network configuration doesn’t match your risk profile
  • If you lack the necessary network or firewall expertise to secure this route sufficiently for your specific use-case

Network partitioning and firewalls

As with other VMs, do not connect container managed VMs directly to XenServer’s management network to provide necessary isolation.

For Container Management to work, managed VMs have to be reachable from the XenServer’ s Control Domain (dom0). To monitor containers on Linux-based operating systems, the networking topology and firewalls must allow outbound SSH connections from dom0 to Container Managed VMs. To monitor Windows Server Containers, the networking topology and firewalls must allow outbound Docker TLS (destination TCP port 2376) connections from dom0 to Container Managed VMs.

To mitigate the risk of allowing traffic between VM and the management network, pass all traffic through an external stateful firewall. This firewall must be manually set up and configured by an expert according to your specific business and security requirement.

The following section contains an example configuration:

To secure connections between the networks:

  • Prevent all connections between the XenServer management network (that is including dom0) and the VM network (that is including container managed VMs) either way.

Add exceptions for enabling Container Management:

  • To monitor Linux-based operating system, allow dom0 to have outbound SSH (TCP port 22) connections (both NEW and ESTABLISHED) to Container Managed VMs.

  • To monitor Windows Server containers, allow dom0 to have outbound Docker TLS (TCP port 2376) connections (both NEW and ESTABLISHED) to Container Managed VMs.

  • Allow Container Managed VMs to reply to (ESTABLISHED) SSH and Docker TLS connections initiated by dom0.

Authentication on Linux-based operating systems

XenServer’s Container Management uses a pool-specific 4096-bit private/public RSA-key-pair to authenticate on Container Managed VMs. The private key is stored in the XenServer Control Domain (dom0). The respective public-key is registered in Container Managed VMs during the preparation, either using the Cloud Config Drive or ~user/.ssh/authorized_keys file. As usual with all private/public key-pairs, the private key must be kept securely, as it allows for password-less access to all Container Managed VMs. This access includes both currently managed VMs and VMs managed in the past.

XenServer’s Container Management attempts to reach Container Managed VMs through any of the IP addresses advertised by the XenServer Tools running inside the VM. After an initial connection, XenServer stores the public key of container managed VMs and validates that the key matches on any subsequent connection. Ensure that only the Container Managed VM can be contacted through its advertised IP (using IP Source Guard or similar means). If the network topology cannot ensure this behavior, Citrix recommends that administrators confirm the SSH hostkey that the Container Management obtained when making the first connection to the VM.

The key can be accessed by using the following command:

xe vm-parm-get-uuid=vm_uuid param-name=other-config  /
  param-key=xscontainer-sshhostkey

vm_uuid is the UUID of the VM

Authentication for Windows Server Containers

XenServer uses SSL or TLS to monitor and control Windows Server Containers. In this instance XenServer acts as the SSL/TLS client, and Windows Server VMs act as the SSL/TLS server. Keys are stored in both Dom0 and the VM.

Important:

  • The client key must be kept securely, as it allows for password-less access to Docker on the VM
  • The server key must be kept securely, as it serves to authenticate the monitoring connection to the VM

When XenServer Container Management generates TLS certificates and keys by using the –generate-certs option, temporary CA, server, and client certificates are generated for a specific pool and VM. Certificates use sha256 hash and are valid for up to 2*365 days. After this time, repeat the preparation. The TLS connection is always established using a AES128-SHA cipher.

Notes

When using XenServer Container Management and Docker, be aware of the following behaviors:

  • Renaming a container does not trigger the Container Management view to update. Additionally on Ubuntu 14.04 the pause or unpause of a container from outside XenCenter doesn’t trigger the view to update. This behavior can mean that XenServer might not show the current (renamed/paused/unpaused) container-status. The underlying cause is that the view only gets refreshed following Docker event notifications. As a workaround, the refresh can be triggered by performing an action (that is, start or stop) on an unrelated container on the same VM.