uberAgent

Citrix ADC Monitoring & Alerting

uberAgent 5.2 introduced Citrix ADC monitoring (formerly NetScaler ADC). This functionality lets you check the appliance health, gateway data volume, and service availability of your ADCs. While it is great to have all this information in dashboards, it sometimes makes sense to get notified when things are not working the way they should. This article demonstrates how to use Splunk’s alerting capabilities to configure just that.

Splunk Alerting

If you are not familiar with Splunk alerting here is the explanation in a nutshell: you create a search; every time it produces a result you will get notified. Several notification methods are available (email, webhooks and so on). More on this topic is available in Splunk’s documentation.

Example Searches for Citrix ADC Alerts

We created some example searches showing what is possible with uberAgent’s data and Splunk’s alerting capabilities. Feel free to customize them to your needs or use them as a starting point for your own creations.

Max Bandwidth per Appliance

The purchased license determines the maximum amount of data the ADC will process - it is a software limitation. Hence it is important to keep an eye on the bandwidth usage to react in a timely manner and buy a bigger license to not slow down the users. Here is an example where you will be notified when the bandwidth usage exceeded 900 megabytes.

| pivot `uA_DM_CitrixADC_AppliancePerformance` CitrixADC_AppliancePerformance
   max(ApplianceMBReceived) as "max(ApplianceMBReceived)"
   max(ApplianceMBSent) as "max(ApplianceMBSent)"
   splitrow
      HostName
| eval "Max. network data volume (MB)" = round('max(ApplianceMBReceived)' + 'max(ApplianceMBSent)',1)
| where 'Max. network data volume (MB)' > 900
<!--NeedCopy-->

Too Many HA Status Changes

A Citrix ADC high-availability (HA) pair consists of a primary and a secondary node. If the primary fails the secondary takes over. Too many status changes indicate that there is something wrong. Maybe your hypervisor does live migration of VMs? It should not for Citrix ADC VPX machines!

| pivot `uA_DM_CitrixADC_ApplianceInventory` CitrixADC_ApplianceInventory
   dc(LastHAStatusChange) as "#Status changes"
   splitrow
      HostName
| where '#Status changes' > 5
<!--NeedCopy-->

The current status is collected once per day by default consequently let that search run for at least a few days.

Get Notified When SSL Cards Are Down

In a hardware Citrix ADC appliance, SSL cards handle the encryption and decryption of SSL traffic. If an SSL card is down the encryption/decryption rate drops which possibly affects user experience.

| pivot `uA_DM_CitrixADC_ApplianceInventory` CitrixADC_ApplianceInventory
   latest(SSLCards) as "#SSL cards"
   latest(SSLCardsUp) as "#SSL cards up"
   splitrow
      HostName
| eval "#SSL cards down" = '#SSL cards' - '#SSL cards up'
| where '#SSL cards down' != 0
<!--NeedCopy-->

Get Notified When the Configuration Was Not Saved Within 24 Hours After a Change

Citrix ADC configuration changes are active immediately, even if they have not been specifically saved. However, unsaved changes are lost when the appliance is rebooted. With this search, you will be notified after 24 hours if someone forgot to save the configuration.

| pivot `uA_DM_CitrixADC_ApplianceInventory` CitrixADC_ApplianceInventory
   latest(LastConfigSaveTime) as LastConfigSaveTime
   latest(LastConfigChangedTime) as LastConfigChangedTime
   splitrow
      HostName
| eval LastConfigSaveTime_epoch = strptime(LastConfigSaveTime, "%Y-%m-%d %H:%S:%M")
| eval LastConfigChangedTime_epoch = strptime(LastConfigChangedTime, "%Y-%m-%d %H:%S:%M")
| eval Offset =  LastConfigChangedTime_epoch - LastConfigSaveTime_epoch
| where Offset > 86400
<!--NeedCopy-->

Get Notified When a Certificate Expires Within Four Weeks

The all-time favorite. A certificate is installed but the team managing it forgot to update it. Never let that happen again with this search!

| pivot `uA_DM_CitrixADC_Certificate` CitrixADC_Certificate
    latest(CertExpirationDate) as CertExpirationDate
    splitrow
       CertName as "Certificate name"
| eval CertExpirationDate_epoch = strptime(CertExpirationDate, "%Y-%m-%d %H:%S:%M")
| eval CurrentTime_epoch = now()
| convert ctime(now)
| eval thirty_days_from_now = CurrentTime_epoch + 2592000
| eval Expires_within_next_30_days = if(CertExpirationDate_epoch >= CurrentTime_epoch AND CertExpirationDate_epoch <= thirty_days_from_now, "yes", "no")
| where Expires_within_next_30_days = "yes"
| table
   "Certificate name"
   CertExpirationDate
   Expires_within_next_30_days
<!--NeedCopy-->

List All IPs Where Management Access Is Enabled but It Is Not an NSIP

It is a common best practice to enable management only on the NetScaler IP (NSIP). Anyway, managing it through the Subnet IP (SNIP) is much more easy, right? Just don’t! Keep safe with this search:

| pivot `uA_DM_CitrixADC_Ip` CitrixADC_Ip
   latest(IpNetmask) as Netmask
   latest(IpType) as Type
   latest(IpvServer) as "Bound to vServer"
   latest(IpMgmtAccess) as "Mgmt. access"
   splitrow
      IpAddress as "IP address"
   filter HostName is "ns1"
| eval "Mgmt. access" = if('Mgmt. access'=1,"Yes","No")
| where Type != "NSIP" and 'Mgmt. access' = "Yes"
| table
   "IP address"
   Netmask
   Type
   "Bound to vServer"
   "Mgmt. access"
<!--NeedCopy-->
Citrix ADC Monitoring & Alerting