Product Documentation

Alerts and notifications

Jun 04, 2018

Monitor alerts

Alerts are displayed in Director on the dashboard and other high level views with warning and critical alert symbols. Alerts are available for Platinum licensed Sites. Alerts update automatically every minute; you can also update alerts on demand. 

localized image

A warning alert (amber triangle) indicates that the warning threshold of a condition has been reached or exceeded.

A critical alert (red circle) shows that the critical threshold of a condition has been reached or exceeded.

You can view more detailed information on alerts by selecting an alert from the sidebar, clicking the Go to Alerts link at the bottom of the sidebar or by selecting Alerts from the top of the Director page.

In the Alerts view, you can filter and export alerts. For example, Failed Server OS machines for a specific Delivery Group over the last month, or all alerts for a specific user. For more information, see Export reports

localized image

Citrix alerts. Citrix alerts are alerts monitored in Director that originate from Citrix components. You can configure Citrix alerts within Director in Alerts > Citrix Alerts Policy. As part of the configuration, you can set notifications to be sent by email to individuals and groups when alerts exceed the thresholds you have set up. You can configure the notification as Octoblu webhooks, or SNMP traps also. For more information on setting up Citrix Alerts, see Create alerts policies.

Built-in alert policies. A set of built-in alert policies with predefined threshold values are available for Delivery Groups and Server OS VDAs scope. This feature requires Delivery Controller(s) version 7.18 or later. You can modify the threshold parameters of the built-in alert policies in Alerts Citrix Alerts Policy. These policies are created when there is at least one alert target -a Delivery Group or a Server OS VDA defined in your Site.

In case you upgrade Director and your Site, the alert policies from your previous Director instance are carried over. Built-in alert policies are created only if no corresponding alert rules exist in the Monitor database. 

For the threshold values of the built-in alert policies, see the Alerts policies conditions section.

localized image

SCOM alerts. SCOM alerts display alert information from Microsoft System Center 2012 Operations Manager (SCOM) to provide a more comprehensive indication of data center health and performance within Director. For more information, see SCOM alerts.

The number of alerts displayed next to the alerts icons before you expand the sidebar are the combined sum of Citrix and SCOM alerts.

Create alerts policies

localized image

To create a new alerts policy, for example, to generate an alert when a specific set of session count criteria are met:

  1. Go to Alerts > Citrix Alerts Policy and select, for example, Server OS Policy.
  2. Click Create.
  3. Name and describe the policy, then set the conditions that have to be met for the alert to be triggered. For example, specify Warning and Critical counts for Peak Connected Sessions, Peak Disconnected Sessions, and Peak Concurrent Total Sessions. Warning values must not be greater than Critical values. For more information, see Alerts policies conditions.
  4. Set the Re-alert interval. If the conditions for the alert are still met, the alert is triggered again at this time interval and, if set up in the alert policy, an email notification is generated. A dismissed alert does not generate an email notification at the re-alert interval.
  5. Set the Scope. For example, set for a specific Delivery Group.
  6. In Notification preferences, specify who should be notified by email when the alert is triggered. You have to specify an email server on the Email Server Configuration tab in order to set email Notification preferences in Alerts Policies.
  7. Click Save.

For information about Octoblu webhook configuration, see Configure alerts policies with Octoblu webhooks.

For information about SNMP trap configuration, see Configure alerts policies with SNMP traps.

Creating a policy with 20 or more Delivery Groups defined in the Scope might take approximately 30 seconds to complete the configuration. A spinner is displayed during this time.

Creating more than 50 policies for up to 20 unique Delivery Groups (1000 Delivery Group targets in total) might result in an increase in response time (over 5 seconds).

Moving a machine containing active sessions from one Delivery Group to another might trigger erroneous Delivery Group alerts that are defined using machine parameters.

Alerts policies conditions

All built-in alert policies are defined for alert and re-alert intervals of 60 minutes.

Alert policy condition

Description and recommended actions

Built-in alert scope and threshold

Peak Connected Sessions

Number of peak connected sessions.

  • Check Director Session Trends view for peak connected sessions.
  • Check to ensure that there is enough capacity to accommodate the session load.
  • Add new machines if needed.

Not defined

Peak Disconnected Sessions

Number of peak disconnected sessions.

  • Check Director Session Trends view for peak disconnected sessions.
  • Check to ensure that there is enough capacity to accommodate session load.
  • Add new machines if needed.
  • Log off disconnected sessions if needed.

Not defined

Peak Concurrent Total Sessions

Number of peak concurrent sessions.

  • Check Director Session Trends view in Director for peak concurrent sessions.
  • Check to ensure that there is enough capacity to accommodate session load.
  • Add new machines if needed.
  • Log off disconnected sessions if needed.

Not defined

CPU

Percentage of CPU usage.

  • Identify the processes or resources consuming CPU.
  • End the process if necessary. Ending the process causes unsaved data to be lost.
  • If all is working as expected, add additional CPU resources in the future.

Note: The policy setting, Enable resource monitoring, is allowed by default for the monitoring of CPU and memory performance counters on machines with VDAs. If this policy setting is disabled, alerts with CPU and memory conditions are not triggered. For more information, see Monitoring policy settings.

Defined for

Scope:
Delivery Group
Server OS 

Threshold values:
Warning - 80%
Critical - 90%

 

Memory

Percentage of Memory usage.

  • Identify the processes or resources consuming memory.
  • End the process if necessary. Ending the process causes unsaved data to be lost.
  • If all is working as expected, add additional memory in the future.

Note: The policy setting, Enable resource monitoring, is allowed by default for the monitoring of CPU and memory performance counters on machines with VDAs. If this policy setting is disabled, alerts with CPU and memory conditions are not triggered. For more information, see Monitoring policy settings.

Defined for

Scope:
Delivery Group
Server OS 

Threshold values:

Warning - 80%
Critical - 90%

Connection Failure Rate

Percentage of connection failures over the last hour. Calculated based on the total failures to total connections attempted.

  • Check Director Connection Failures Trends view for events logged from the Configuration log.
  • Determine if applications or desktops are reachable.

Not defined

Connection Failure Count

Number of connection failures over the last hour.

  • Check Director Connection Failures Trends view for events logged from the Configuration log.
  • Determine if applications or desktops are reachable.

Not defined

ICA RTT (Average)

Average ICA round-trip time.

  • Check NetScaler HDX Insight or MAS for a breakdown of the ICA RTT to determine the root cause. For more information, see the NetScaler Insight Center - HDX Insight or NetScaler MAS - Analytics: HDX Insight documentation.
  • If NetScaler is not available, check the Director User Details view for the ICA RTT and Latency, and determine if it is a network problem or XenApp and XenDesktop issue. 

Not defined

ICA RTT (No. of Sessions)

Number of sessions that exceed the threshold ICA round-trip time.

Defined for

Scope:
Delivery Group
Server OS

Threshold values:
Warning - 300ms for 5 or more sessions
Critical - 400ms for 10 or more sessions

ICA RTT (% of Sessions)

Percentage of sessions that exceed the average ICA round-trip time.

Not defined

ICA RTT (User)

ICA round-trip time that is applied to sessions launched by the specified user. The alert is triggered if ICA RTT is greater than the threshold in at least one session.

Not defined

Failed Machines (Desktop OS)

Number of failed Desktop OS machines.

Failures can occur for various reasons as shown in the Director Dashboard and Filters views. Run Citrix Scout diagnostics to determine the root cause. For more information, see Troubleshoot user issues.

Defined for

Scope: Delivery Group

Threshold values:
Warning - 1
Critical - 2

Failed Machines (Server OS)

Number of failed Server OS machines.

Failures can occur for various reasons as shown in the Director Dashboard and Filters views. Run Citrix Scout diagnostics to determine the root cause.

Defined for

Scope: Delivery Group

Threshold values:
Warning - 1
Critical - 2

Average Logon Duration

Average logon duration for logons that occurred over the last hour.

  • Check the Director Dashboard to get up-to-date metrics regarding the logon duration. A large number of users logging in during a short timeframe can increase the logon duration.
  • Check the baseline and break down of the logons to narrow down the cause.
    For more information, see Diagnose user logon issues.

Defined for

Scope:
Delivery Group
Server OS

Threshold values:
Warning - 45 seconds Critical - 60 seconds

Logon Duration (User)

Logon duration for logons for the specified user that occurred over the last hour.

Not defined

Load Evaluator Index

Value of the Load Evaluator Index over the last 5 minutes.

  • Check Director for Server OS Machines that might have a peak load (Max load).
  • View both Dashboard (failures) and Trends Load Evaluator Index report.

Defined for

Scope:
Delivery Group
Server OS

Threshold values:
Warning - 80%
Critical - 90% 

 

Configure alerts policies with Octoblu webhooks

Note: On Nov 29th, 2017, Citrix shut down its free Octoblu.com Cloud Service. As a result, we recommend that you discontinue integrating Octoblu with Director. For more information on Citrix’s announcement to shut down Octoblu.com, see the blog, The Future of Octoblu and Citrix Workspace IoT.

Configure alerts policies with Octoblu webhooks to initiate IoT services. This feature requires Delivery Controller(s) version 7.11 or later.

Examples of IoT services that can utilize alerts include sending SMS notifications to support staff or integrating with custom incident resolution platforms to help in tracking notifications.

You can configure an alert policy with an HTTP callback or an HTTP POST using PowerShell cmdlets. They are extended to support webhooks.

For information on the creation of a new Octoblu workflow and obtaining the corresponding webhook URL, see the Octoblu Developer Hub.

To configure an Octoblu webhook URL for a new alert policy or an existing policy, use the following PowerShell cmdlets.

Create a new alerts policy with a webhook URL:

command Copy

$policy = New-MonitorNotificationPolicy -Name <Policy name> -Description <Policy description> -Enabled $true -Webhook <Webhook URL>

Add a webhook URL to an existing alerts policy:

command Copy

Set-MonitorNotificationPolicy - Uid <Policy id> -Webhook <Webhook URL>

For help on the PowerShell commands, use the PowerShell help, for example:

command Copy

Get-Help  <Set-MonitorNotificationPolicy>

For more information on configuring alert policies with PowerShell, see Director 7.7: Managing and Configuring Alerts and Notifications Using Powershell in Advanced Concepts.

Notifications generated from the alert policy trigger the webhook with a POST call to the webhook URL. The POST message contains the notification information in JSON format:

{"NotificationId" : <Notification Id>,

"Target" : <Notification Target Id>,

"Condition" : <Condition that was violated>,

"Value" : <Threshold value for the Condition>,

"Timestamp": <Time in UTC when notification was generated>,

"PolicyName": <Name of the Alert policy>,

"Description": <Description of the Alert policy>,

"Scope" : <Scope of the Alert policy>,

"NotificationState": <Notification state critical, warning, healthy or dismissed>,

"Site" : <Site name>}

Configure alerts policies with SNMP traps

When an alert configured with an SNMP trap triggers, the corresponding SNMP trap message is forwarded to the configured network listener for further processing. Citrix alerts support traps of SNMP version 2 and later. Currently, the trap message can be forwarded to one listener.

Note: This feature requires Delivery Controller(s) version 7.12 or later.

To configure SNMP traps, use the following PowerShell cmdlets:

  • Get the current SNMP server configuration:
command Copy

Get-MonitorNotificationSnmpServerConfiguration

  • Set server configuration for SNMP version 2:
command Copy

Set-MonitorNotificationSnmpServerConfiguration -ServerName <Server IP> -PortNumber <Port ID> -SnmpSender <Sender name> -CommunityString public -Protocol V2

  • Set server configuration for SNMP version 3:
command Copy

$authpass = "<authentication password>" | ConvertTo-SecureString -AsPlainText -Force

$privpass = "<Privacy password>" | ConvertTo-SecureString -AsPlainText -Force

Set-MonitorNotificationSnmpServerConfiguration -ServerName <Server IP> -PortNumber <Port ID> -SnmpSender <Sender name> -EngineId <Engine Id> -AuthPassword $authpass -PrivPassword $privpass -PrivPasswordProtocol <Privacy password protocol> -AuthPasswordProtocol <Authentication password protocol> -Protocol V3

  • Enable SNMP trap for an existing alert policy:
command Copy

Set-MonitorNotificationPolicy -IsSnmpEnabled $true -Uid <Policy ID>

  • Create a new alert policy with SNMP trap configuration:
command Copy

$policy = New-MonitorNotificationPolicy -Name <Policy name> -IsSnmpEnabled $true -Description <Policy description> -Enabled $true

The structure of the OIDs in the SNMP trap messages from Director is as follows:
1.3.6.1.4.1.3845.100.1.<UID>
Here, <UID> is generated serially for every alert policy defined in Director. The OIDs are hence unique to each user environment.

  • Use 1.3.6.1.4.1.3845.100.1 to filter all trap messages from Director.
  • Use 1.3.6.1.4.1.3845.100.1.<UID> to filter and handle traps messages for specific alerts.

Use the following cmdlet to get the UIDs for the alert policies defined in your environment:

command Copy

Get-MonitorNotificationPolicy

You can forward the SNMP traps to SCOM. To do this, configure SCOM with the Delivery Controller to listen to the trap messages.

SCOM alerts

SCOM integration with Director lets you view alert information from SCOM on the Dashboard and in other high-level views in Director.

SCOM alerts are displayed on-screen alongside Citrix alerts. You can access and drill down into SCOM alerts from SCOM tab in the side bar.

You can view historical alerts up to one month old, sort, filter, and export the filtered information to CSV, Excel, and PDF report formats. For more information, see Export reports.

Configure SCOM integration

SCOM integration uses remote PowerShell 3.0 or later to query data from the SCOM Management Server and it maintains a persistent runspace connection in the user's Director session. Director and SCOM server must have the same PowerShell version.

localized image

The requirements for SCOM integration are:

  • Windows Server 2012 R2
  • System Center 2012 R2 Operations Manager
  • PowerShell 3.0 or later (PowerShell version on Director and the SCOM server must match)
  • Quad Core CPU with 16 GB RAM (recommended)
  • A primary Management Server for SCOM must be configured in the Director web.config file. You can do this using the DirectorConfig tool.
Note:
  • Citrix recommends that the Director administrator account is configured as a SCOM Operator role so that can full alert information can be retrieved in Director. If this is not possible, a SCOM administrator account can be configured in the web.config file using the DirectorConfig tool.
  • Citrix recommends that you do not configure more than 10 Director administrators per SCOM Management Server to ensure optimal performance.

On the Director server:

1. Type Enable-PSRemoting to enable PowerShell remoting.

2. Add the SCOM Management Server to the TrustedHosts list. Open a PowerShell prompt  and execute the following command(s):

a. Get the current list of TrustedHosts

command Copy

Get-Item WSMAN:\localhost\Client\TrustedHosts

b. Add the FQDN of the SCOM Management Server to the list of TrustedHosts. <Old Values> represents the existing set of entries returned from Get-Item cmdlet.

command Copy

Set-Item WSMAN:\localhost\Client\TrustedHosts -Value "<FQDN SCOM Management Server>,<Old Values>"

3. Configure SCOM using the DirectorConfig tool.

command Copy

C:\inetpub\wwwroot\Director\tools\DirectorConfig.exe /configscom

On the SCOM Management server:

1. Assign Director administrators to a SCOM administrator role.

a. Open the SCOM Management console and go to Administration > Security > User Roles.

b. In User Roles, you can create a new User Role or modify an existing one. There are four categories of SCOM operator roles that define the nature of access to SCOM data. For example, a Read-Only role does not see the Administration pane and cannot discover or manage rules, machines or accounts. An Operator role is a full administrator role.

Note: The following operations are not available if the Director administrator is assigned to a non-operator role: 

i. If there are multiple management servers configured and the primary management server is not available, the Director administrator cannot connect to the secondary management server. The primary management server is the server configured in the Director web.config file, that is the same server as the one specified with the DirectorConfig tool in step 3 above. The secondary management servers are peer management servers of the primary server.

ii. While filtering alerts, the Director administrator cannot search for the alert source. This requires an operator level permission.

c. To modify any User Role, right-click on the role, then click Properties.

d. In the User Role Properties dialog, you can add or remove Director administrators from the specified user role.

2. Add Director administrators to the Remote Management Users group on the SCOM Management server. This allows the Director administrators to establish a remote PowerShell connection.

3. Type Enable-PSRemoting to enable PowerShell remoting.

4. Set the WS-Management properties limits:

a. Modify MaxConcurrentUsers:

In CLI:

command Copy

winrm set winrm/config/winrs @{MaxConcurrentUsers = "20"}

In PS:

command Copy

Set­-Item WSMan:\localhost\Shell\MaxConcurrentUsers 20

b. Modify MaxShellsPerUser:

In CLI:

command Copy

winrm set winrm/config/winrs @{MaxShellsPerUser="20"}

In PS:

command Copy

Set-Item WSMan:\localhost\Shell\MaxShellsPerUser 20

c. Modify MaxMemoryPerShellMB:

In CLI:

command Copy

winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"}

In PS:

command Copy

Set­-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 1024

5. To ensure that SCOM integration works in mixed domain environments, set the following registry entry.

Path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

Key: LocalAccountTokenFilterPolicy

Type: DWord

Value: 1

Caution: Editing the registry incorrectly can cause serious problems that might require you to reinstall your operating system. Citrix cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk. Be sure to back up the registry before you edit it.

Once SCOM integration is set up you might see the message "Cannot get the latest SCOM alerts. View the Director server event logs for more information". The server event logs help identify and correct the problem. Causes can include:

  • Loss of network connectivity at the Director or SCOM machine.
  • The SCOM service is not available or too busy to respond.
  • Failed authorization due to a change in permissions for the configured user.
  • An error in Director while processing the SCOM data.
  • PowerShell version mismatch between Director and SCOM server.