Enabling VNC for Linux VMs

VMs might not be set up to support Virtual Network Computing (VNC), which XenServer uses to control VMs remotely, by default. Before you can connect with the XenCenter graphical console, you need to ensure that the VNC server and an X display manager are installed on the VM and properly configured. This section describes the procedures for configuring VNC on each of the supported Linux operating system distributions to allow proper interactions with the XenCenter graphical console.

CentOS-based VMs should use the instructions for the Red Hat-based VMs below, as they use the same base code to provide graphical VNC access. CentOS 4 is based on Red Hat Enterprise Linux 4, and CentOS 5 is based on Red Hat Enterprise Linux 5.

Enabling a Graphical Console on Debian Squeeze VMs

Note

Before enabling a graphical console on your Debian Squeeze VM, ensure that you have installed the Linux guest agent. See Install the Linux Guest Agent for details.

The graphical console for Debian Squeeze virtual machines is provided by a VNC server running inside the VM. In the recommended configuration, this is controlled by a standard display manager so that a login dialog is provided.

  1. Install your Squeeze guest with the desktop system packages, or install GDM (the display manager) using apt (following standard procedures).

  2. Install the Xvnc server using apt-get (or similar):

    apt-get install vnc4server
    

    Note

    Significant CPU time can be taken by the Debian Squeeze Graphical Desktop Environment, which uses the Gnome Display Manager version 3 daemon. Citrix strongly advises that customers uninstall the Gnome Display Manager gdm3 package and install the gdm package as follows:

    apt-get install gdm
    apt-get purge gdm3
    
  3. Set up a VNC password (not having one is a serious security risk) using the vncpasswd command, passing in a file name to write the password information to. For example:

    vncpasswd /etc/vncpass
    
  4. Modify your gdm.conf file (/etc/gdm/gdm.conf) to configure a VNC server to manage display 0 by extending the [servers] and [daemon] sections as follows:

    [servers]
    0=VNC
    [daemon]
    VTAllocation=false
    [server-VNC]
    name=VNC
    command=/usr/bin/Xvnc -geometry 800x600 -PasswordFile /etc/vncpass BlacklistTimeout=0
    flexible=true
    
  5. Restart GDM, and then wait for the graphical console to be detected by XenCenter:

    /etc/init.d/gdm restart
    

    Note

    You can check that the VNC server is running using a command like ps ax | grep vnc.

Enabling a Graphical Console on Red Hat, CentOS, or Oracle Linux VMs

Note

Before setting up your Red Hat VMs for VNC, be sure that you have installed the Linux guest agent. See Install the Linux Guest Agent for details.

To configure VNC on Red Hat VMs, you need to modify the GDM configuration. The GDM configuration is held in a file whose location varies depending on the version of Red Hat Linux you are using. Before modifying it, first determine the location of this configuration file. This file will then be modified in several subsequent procedures in this section.

Determining the Location of your VNC Configuration File

If you are using Red Hat Linux version 4 the GDM configuration file is /etc/X11/gdm/gdm.conf. This is a unified configuration file that contains default values as specified by the provider of your version of GDM in addition to your own customized configuration. This type of file is used by default in older versions of GDM, as included in these versions of Red Hat Linux.

If you are using Red Hat Linux version 5.x the GDM configuration file is /etc/gdm/custom.conf. This is a split configuration file that contains only user-specified values that override the default configuration. This type of file is used by default in newer versions of GDM, as included in these versions of Red Hat Linux.

Configuring GDM to use VNC

  1. As root on the text CLI in the VM, run the command rpm -q vnc-server gdm. The package names vnc-server and gdm should appear, with their version numbers specified.

    If these package names are displayed, the appropriate packages are already installed. If you see a message saying that one of the packages is not installed, then you may not have selected the graphical desktop options during installation. You will need to install these packages before you can continue. See the appropriate Red Hat Linux x86 Installation Guide for details regarding installing additional software on your VM.

  2. Open the GDM configuration file with your preferred text editor and add the following lines to the file:

    [server-VNC]
    name=VNC Server
    command=/usr/bin/Xvnc -SecurityTypes None -geometry 1024x768 -depth 16 \
    -BlacklistTimeout 0
    flexible=true
    
    • With configuration files on 4.x, this should be added above the [server-Standard] section.

    • With configuration files on Red Hat Linux 5.x, this should be added into the empty [servers] section.

  3. Modify the configuration so that the Xvnc server is used instead of the standard X server:

    • If you are using Red Hat Linux 3 or 4, there will be a line just above that reads:

       0=Standard
      

      Modify it to read:

       0=VNC
      
    • If you are using Red Hat Linux 5.x or greater, add the above line just below the [servers] section and before the [server-VNC] section.

  4. Save and close the file.

  5. Restart GDM for your change in configuration to take effect, by running the command /usr/sbin/gdm-restart.

Note

Red Hat Linux uses runlevel 5 for graphical startup. If your installation is configured to start up in runlevel 3, change this for the display manager to be started (and therefore to get access to a graphical console).

Firewall Settings

Configuring VNC firewall settings, RHELThe firewall configuration by default does not allow VNC traffic to go through. If you have a firewall between the VM and XenCenter, you need to allow traffic over the port that the VNC connection uses. By default, a VNC server listens for connections from a VNC viewer on TCP port 5900 + n, where n is the display number (usually just zero). So a VNC server setup for Display-0 will listen on TCP port 5900, Display-1 is TCP-5901, and so on. Consult your firewall documentation to make sure these ports are open.

You might want to further customize your firewall configuration if you want to use IP connection tracking or limit the initiation of connections to be from one side only.

To customize Red Hat-based VMs firewall to open the VNC port

  1. For Red Hat Linux 4.x and 5.x, use system-config-securitylevel-tui.

  2. Select “Customize” and add 5900 to the other ports list.

Alternatively, you can disable the firewall until the next reboot by running the command service iptables stop, or permanently by running chkconfig iptables off. This can of course expose extra services to the outside world and reduce the overall security of your VM.

VNC Screen Resolution

If after connecting to a VM with the graphical console the screen resolution is mismatched (for example, the VM display is too large to comfortably fit in the Graphical Console pane), you can control it by setting the VNC server geometry parameter as follows:

  1. Open the GDM configuration file with your preferred text editor.

  2. Find the [server-VNC] section you added above.

  3. Edit the command line to read, for example:

    command=/usr/bin/Xvnc -SecurityTypes None -geometry 800x600
    

    where the value of the geometry parameter can be any valid screen width and height.

  4. Save and close the file.

Enabling VNC for RHEL, CentOS, or OEL 6.x VMs

If you are using Red Hat Linux version 6.x, the GDM configuration file is /etc/gdm/custom.conf. This is a split configuration file that contains only user-specified values that override the default configuration. This type of file is used by default in newer versions of GDM, as included in these versions of Red Hat Linux.

During the operating system installation, select Desktop mode.

  1. On the RHEL installation screen, select Desktop, Customize now, and then click Next.

    This displays the Base System screen, ensure that Legacy UNIX compatibility is selected.

  2. Select Desktop, Optional packages, then click Next.

    This displays the Packages in Desktop window, select tigervnc-server-<version_number> and then click Next

Work through the following steps to continue the setup of your RHEL 6.x VMs:

  1. Open the GDM configuration file with your preferred text editor and add the following lines to the appropriate sections:

    [security]
    DisallowTCP=false
    
    [xdmcp]
    Enable=true
    
  2. Create the file, /etc/xinetd.d/vnc-server-stream:

    service vnc-server
    {
                  id = vnc-server
             disable = no
                type = UNLISTED
                port = 5900
         socket_type = stream
                wait = no
                user = nobody
               group = tty
              server = /usr/bin/Xvnc
         server_args = -inetd -once -query localhost -SecurityTypes None -geometry 800x600 -depth 16
    }
    
  3. Enter the following command to start the xinetd service:

    # service xinetd start
    
  4. Open the file /etc/sysconfig/iptables and add the following line. The line should be added above the line reading, -A INPUT -j REJECT --reject-with icmp-host-prohibited:

    -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900 -j ACCEPT
    
  5. Enter the following command to restart iptables:

    # service iptables restart
    
  6. Enter the following command to restart gdm:

    # telinit 3
    # telinit 5
    

Note

Red Hat Linux uses runlevel 5 for graphical startup. If your installation is configured to start up in runlevel 3, change this for the display manager to be started (and therefore to get access to a graphical console).

Setting up SLES-based VMs for VNC

Note

Before setting up your SUSE Linux Enterprise Server VMs for VNC, be sure that you have installed the Linux guest agent. See Install the Linux Guest Agent for details.

SLES has support for enabling “Remote Administration” as a configuration option in YaST. You can select to enable Remote Administration at install time, available on the Network Services screen of the SLES installer. This allows you to connect an external VNC viewer to your guest to allow you to view the graphical console; the methodology for using the SLES remote administration feature is slightly different than that provided by XenCenter, but it is possible to modify the configuration files in your SUSE Linux VM such that it is integrated with the graphical console feature.

Checking for a VNC Server

Before making configuration changes, verify that you have a VNC server installed. SUSE ships the tightvnc server by default; this is a suitable VNC server, but you can also use the standard RealVNC distribution if you prefer.

You can check that you have the tightvnc software installed by running the command:

rpm -q tightvnc

Enabling Remote Administration

If Remote Administration was not enabled during installation of the SLES software, you can enable it as follows:

  1. Open a text console on the VM and run the YaST utility:

    yast
    
  2. Use the arrow keys to select Network Services in the left menu, then Tab to the right menu and use the arrow keys to select Remote Administration. Press Enter.

  3. In the Remote Administration screen, Tab to the Remote Administration Settings section. Use the arrow keys to select Allow Remote Administration and press Enter to place an X in the check box.

  4. Tab to the Firewall Settings section. Use the arrow keys to select Open Port in Firewall and press Enter to place an X in the check box.

  5. Tab to the Finish button and press Enter.

  6. A message box is displayed, telling you that you will need to restart the display manager for your settings to take effect. Press Enter to acknowledge the message.

  7. The original top-level menu of YaST appears. Tab to the Quit button and press Enter.

Modifying the xinetd Configuration

After enabling Remote Administration, you need to modify a configuration file if you want to allow XenCenter to connect, or else use a third party VNC client.

  1. Open the file /etc/xinetd.d/vnc in your preferred text editor.

    The file contains sections like the following:

    service vnc1
    {
    socket_type = stream
    protocol    = tcp
    wait        = no
    user        = nobody
    server      = /usr/X11R6/bin/Xvnc
    server_args = :42 -inetd -once -query localhost -geometry 1024x768 -depth 16
    type        = UNLISTED
    port        = 5901
    }
    
  2. Edit the port line to read

    port = 5900
    
  3. Save and close the file.

  4. Restart the display manager and xinetd service with the following commands:

    /etc/init.d/xinetd restart
    rcxdm restart
    

SUSE Linux uses runlevel 5 for graphical startup. If your remote desktop does not appear, verify that your VM is configured to start up in runlevel 5.

Firewall Settings

Configuring VNC firewall settings, SLESBy default the firewall configuration does not allow VNC traffic to go through. If you have a firewall between the VM and XenCenter, you need to allow traffic over the port that the VNC connection uses. By default, a VNC server listens for connections from a VNC viewer on TCP port 5900 + n, where n is the display number (usually just zero). So a VNC server setup for Display-0 will listen on TCP port 5900, Display-1 is TCP-5901, and so on Consult your firewall documentation to make sure these ports are open.

You might want to further customize your firewall configuration if you want to use IP connection tracking or limit the initiation of connections to be from one side only.

To Open the VNC Port on SLES 10.x VMs’ Firewall

  1. Open a text console on the VM and run the YaST utility:

    yast
    
  2. Use the arrow keys to select Security and Users in the left menu, then Tab to the right menu and use the arrow keys to select Firewall. Press Enter.

  3. In the Firewall screen, use the arrow keys to select the Allowed Services in the left menu.

  4. Tab to the Firewall Configuration: Allowed Services fields on the right. Use the arrow keys to select the Advanced button (near the bottom right, just above the Next button) and press Enter.

  5. In the Additional Allowed Ports screen, enter 5900 in the TCP Ports field. Tab to the OK button and press Enter.

  6. Tab to the Next button and press Enter, then in the Summary screen Tab to the Accept button and press Enter, and finally on the top-level YaST screen Tab to the Quit button and press Enter.

  7. Restart the display manager and xinetd service with the following commands:

    /etc/init.d/xinetd restart
    rcxdm restart
    

Alternatively, you can disable the firewall until the next reboot by running the rcSuSEfirewall2 stop command, or permanently by using YaST. This can of course expose additional services to the outside world and reduce the overall security of your VM.

To Open the VNC Port on SLES 11.x VMs’ Firewall

  1. Open a text console on the VM and run the YaST utility:

    yast
    
  2. Use the arrow keys to select Security and Users in the left menu, then Tab to the right menu and use the arrow keys to select Firewall. Press Enter.

  3. In the Firewall screen, use the arrow keys to select Custom Rules in the left menu and then press Enter.

  4. Tab to the Add button in the Custom Allowed Rules section and then press Enter.

  5. In the Source Network field, enter 0/0. Tab to the Destination Port field and enter 5900.

  6. Tab to the Add button and then press Enter.

  7. Tab to the Next button and press Enter, then in the Summary screen Tab to the Finish button and press Enter, and finally on the top-level YaST screen Tab to the Quit button and press Enter.

  8. Restart the display manager and xinetd service with the following commands:

    /etc/init.d/xinetd restart
    rcxdm restart
    

Alternatively, you can disable the firewall until the next reboot by running the rcSuSEfirewall2 stop command, or permanently by using YaST. This can of course expose additional services to the outside world and reduce the overall security of your VM.

VNC Screen Resolution

If after connecting to a Virtual Machine with the Graphical Console the screen resolution is mismatched (for example, the VM display is too big to comfortably fit in the Graphical Console pane), you can control it by setting the VNC server geometry parameter as follows:

  1. Open the /etc/xinetd.d/vnc file with your preferred text editor and find the service_vnc1 section (corresponding to displayID 1).

  2. Edit the geometry argument in the server-args line to the desired display resolution. For example,

    server_args  = :42 -inetd -once -query localhost -geometry 800x600 -depth 16
    

    where the value of the geometry parameter can be any valid screen width and height.

  3. Save and close the file.

  4. Restart the VNC server:

    /etc/init.d/xinetd restart
    rcxdm restart
    

Checking Runlevels

Red Hat and SUSE Linux VMs use runlevel 5 for graphical startup. This section describes how to verify that your VM is configured to start up in runlevel 5 and how to change it if it is not.Linux runlevels

  1. Check /etc/inittab to see what the default runlevel is set to. Look for the line that reads:

    id:n:initdefault:
    

    If n is not 5, edit the file to make it so.

  2. You can run the command telinit q ; telinit 5 after this change to avoid having to actually reboot to switch runlevels.