Network Troubleshooting

Customers can view logs of all the network appliances from a single pane of glass, enabling quick troubleshooting. You can view audit and device logs.

Audit logs

Audit logs capture the action, time, and result of the action performed by users in the customer network.

Network audit logs

Device logs

Customers can view the device logs that are specific to sites.

You can select specific device logs, download it, and share it with site admins if necessary.

Network device logs

Security Logs

In the Citrix SD-WAN appliance, the Edge Security events are logged in the SDWAN_advanced_firewall.log file. The log file is periodically rotated based on the size, with up to 23 archives or a day’s worth of logs remaining, whichever is less. For example, consider the following two use cases:

  • If the log file fills up at a rate of 1 GB per 20 transactions (log entries), logs for approximately 8 hours is available, in the appliance, at any given time.
  • If the log file fills up at a rate of 1 GB per hour or slower, logs for exactly a day are available in the appliance.

Note

  • The size threshold for log rotation depends on the appliance. For the Citrix SD-WAN 1100 appliance the log rotation size threshold is 1 GB.
  • The log file SDWAN_advanced_firewall.log is available for Citrix SD-WAN 1100 appliance only, it is not available on Citrix SD-WAN 210 SE and Citrix SD-WAN 210 SE LTE appliances.

To receive the security logs from the Citrix SD-WAN appliance, in the appliance UI, navigate to Configuration > Appliance Settings > Logging/Monitoring > Syslog Server and ensure that the Firewall Logs to Syslog option is enabled.

Retrieval from orchestrator

Similar to other appliance log files, you can retrieve the Edge Security firewall logs from the Citrix SD-WAN Orchestrator. At the network level, navigate to Troubleshooting > Device Logs, select a site with Edge Security enabled, select the Advanced Firewall logs you want to download and click Download.

Network troubleshooting

Exporting to external syslog server

If an external syslog server is configured on the appliance UI (Appliance Settings > Logging/Monitoring > Syslog server) Edge Security logs are generated and offloaded to this server.

Log entries

Edge Security log entries follow the Common Event Format (CEF). CEF is a standard that defines the syntax of log messages and therefore allows the interoperability of multiple devices generating log messages in a solution.

CEF consists of a standard header and a variable extension. The header format is as follows:

Timestamp host CEF:Version|Device Vendor|Device Product|Device Version|Device Event Class ID|Name|Severity|[Extension]

Example:

Jan 18 11:07:53 sd-wan CEF:0|Citrix|SD-WAN|11.2.2.7|EdgeSec|Name|Severity|Extension

The following fields are common to all edge security logs generated by SD-WAN:

  • Timestamp: The time when the log message is generated.

    Note

    This time can be different from the time of the event the message corresponds to.

  • Host: The name of the host that generates the log file. For example, mybranch.

  • CEF: A fixed string, indicating that the remainder of the message follows the CEF format.

    Note

    No other format is supported.

  • Version: Identifies the version of the CEF format. The current CEF version is 0.

  • Device Vendor: The vendor of the instance or appliance generating the CEF message. The field is always Citrix.

  • Device Product: The product, SD-WAN.

  • Device Version: The software version of the SD-WAN appliance in major.minor.patch.buildnumber format. For example, 11.2.0.88.

  • Device Event Class ID: A unique identifier for every event-type. For Edge security logs, it is always EdgeSec.

  • Name: A human-readable description of the Edge Security event-type. For example, HTTP, FTP, and so on.

  • Severity: Reflects the security criticality of the event. The severity levels defined in the CEF standard are as follows:
    • 0-3=Low
    • 4-6=Medium
    • 7-8=High
    • 9-10=Very-High

    For Citrix SD-WAN Edge Security events, the following criteria are used to determine the severity level:

    Severity level Description
    0 All SESSION events and all allowed HTTP Web filtering events
    3 All clean (allowed) FTP, SMTP, and HTTP Anti-Malware events
    4 Logged IPS events
    6 Blocked HTTP Web filtering events
    7 Blocked IPS events
    8 Blocked (infected) FTP, SMTP, and HTTP Anti-Malware events
  • Extensions: A collection of key-value pairs that provide more details on the event. For example, “rt=Aug 13 2020 11:46:55”, where “rt” is the key for the event timestamp and “Aug 13 2020 11:46:55” is the value. The key-value pairs used depend on the event type the CEF message corresponds to. The detailed descriptions of the event types are provided in the following sections.

HTTP(S)

HTTP(S) log entries capture events related to HTTP or HTTPS traffic. Such events can be generated by either the web-filtering component, which performs URL categorization on HTTP(S) requests, or the Anti-Malware component, which scans HTTP responses for malware infections. The following table describes the various extensions in an HTTP log entry:

Field Description
rt The time of the event, without time zone. The time is expressed in UTC.
cn1 The session identifier, allows correlation with session events
cn1Label Descriptive text for the cn1 field. The value is session_Id.
cs1 The configured security policy
cs1Label Descriptive text for the cs1 field. The value is Security profile.
src The source IP address (client-side)
spt The source port (client-side)
dst The destination IP address (server-side)
dpt The destination port address (server-side)
requestMethod The HTTP request (for example, GET, POST)
request The HTTP URL
dhost The HTTP host name
act The action for the HTTP event. For an allowed HTTP event, the value is allowed and for a blocked HTTP event, the value is blocked.
reason The component that generated the event. Valid values are web_filter and anti_malware.
cs2 The category name of the URL category matched
cs2Label Descriptive text for the cs2 field. The value is URL Category.
cs3 The name of the malware identified in the payload (if any)
cs3Label Descriptive text for the cs3 field. The value is Malware name.

HTTP events can be generated by either the web-filtering or Anti-Malware components. For web-filtering events, the value for cs3 key is blank, whereas for Anti-Malware events, the value for cs2 key is blank.

A typical log entry for an allowed HTTP web-filtering event is as follows:

Oct 8 2020 09:51:01 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|HTTP|0|rt=Oct 8 2020 09:51:01 cn1=104946811893306 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=54749 dst=192.168.1.2 dpt=80 requestMethod=GET request=http://192.168.1.2/eicar.exe dhost=192.168.1.2 act=allowed reason=web_filter cs2=Uncategorized cs2Label=URL Category cs3= cs3Label=Malware name

A typical log entry for a Blocked HTTP web-filtering event is as follows:

Oct 8 2020 09:46:57 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|HTTP|6|rt=Oct 8 2020 09:46:57 cn1=104946811893249 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=59543 dst=192.168.1.2 dpt=443 requestMethod=GET request=http://www.randomadultsite.com/ dhost=www.randomadultsite.com act=blocked reason=web_filter cs2=Adult and Pornography cs2Label=URL Category cs3= cs3Label=Malware name

A typical log entry for an allowed HTTP Anti-Malware event is as follows:

Oct 8 2020 11:49:09 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|HTTP|3|rt=Oct 8 2020 11:49:08 cn1=104946811893527 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=34143 dst=192.168.1.2 dpt=80 requestMethod=GET request=http://192.168.1.2/harmless.exe dhost=192.168.1.2 act=allowed reason=anti_malware cs2= cs2Label=URL Category cs3= cs3Label=Malware name

A typical log entry for a blocked HTTP Anti-Malware event is as follows:

Oct 8 2020 11:45:43 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|HTTP|8|rt=Oct 8 2020 11:45:43 cn1=104946811893520 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=37702 dst=192.168.1.2 dpt=80 requestMethod=GET request=http://192.168.1.2/eicar.exe dhost=192.168.1.2 act=blocked reason=anti_malware cs2= cs2Label=URL Category cs3=EICAR-Test-File cs3Label=Malware name

FTP events

The FTP log entries capture user activity related to FTP requests. The following table describes the various fields in an FTP log entry:

Field Description
rt The time of the event, without Time Zone
cn1 The session identifier, allows correlation with session events
cn1Label Descriptive text for the cn1 field. The value is session_Id.
cs1 The configured security policy
cs1Label Descriptive text for the cs1 field. The value is Security profile.
src The source IP address (client-side)
dst The destination IP address (server-side)
request The FTP URI
act The action for the FTP event. For an allowed FTP event, the value is allowed and for a blocked FTP event, the value is blocked.
reason The component that generated the event. The only component generating FTP events are currently anti_malware.
cs3 The name of the malware identified in the payload (if any)
cs3Label Descriptive text for the cs3 field. The value is Malware name.

A typical log entry for an allowed FTP Anti-Malware event is as follows:

Oct 8 2020 09:49:56 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|FTP|3|rt=Oct 8 2020 09:49:56 cn1=104946811893256 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 dst=192.168.1.2 request=harmless.exe act=allowed reason=anti_malware cs3= cs3Label=Malware name

A typical log entry for a blocked FTP Anti-Malware event is as follows:

Oct 8 2020 09:50:06 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|FTP|8|rt=Oct 8 2020 09:50:06 cn1=104946811893276 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 dst=192.168.1.2 request=eicar.exe act=blocked reason=anti_malware cs3=EICAR-Test-File cs3Label=Malware name

SMTP events

The SMTP log entries capture user activity related to unencrypted email, sent using the SMTP protocol. The following table describes the various fields in an SMTP log entry:

Field Description
rt The time of the event, without Time Zone
cn1 The session identifier, allows correlation with session events
cn1Label Descriptive text for the cn1 field. The value is session_Id.
cs1 The configured security policy
cs1Label Descriptive text for the cs1 field. The value is Security profile.
src The source IP address (client-side)
spt The source port (client-side)
dst The destination IP address (server-side)
dpt The destination port address (server-side)
cn2 The message identifier
cn2Label Descriptive text for the cn2 field. The value is Message identifier.
cs4 The email subject
cs4Label Descriptive text for the cs4 field. The value is Message subject.
suser The address of the sender
duser The address of the receiver
act The action for the SMTP event. The value is allowed, for an allowed SMTP email message, blocked, for a blocked SMTP email message, and remove for an SMTP email message that was allowed after a malware payload was removed.
reason The reason for the action. The value is anti_malware.
cs3 The name of the malware identified in the payload (if any)
cs3Label Descriptive text for the cs3 field. The value is Malware name.

A typical log entry for an SMTP event with virus is as follows:

Oct 8 2020 11:51:31 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|SMTP|8|rt=Oct 8 2020 11:51:31 cn1=104946811893617 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=36097 dst=192.168.1.2 dpt=25 cn2=104946811893546 cn2Label=message identifier cs4=Test email cs4Label=subject suser=sender@sender.com suserLabel=sender duser=receiver@receiver.com duserLabel=receiver act=remove reason=anti_malware cs3=EICAR-Test-File cs3Label=Malware name

A typical log entry for an SMTP event without virus is as follows:

Oct 8 2020 11:50:50 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|SMTP|3|rt=Oct 8 2020 11:50:50 cn1=104946811893573 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=52737 dst=192.168.1.2 dpt=25 cn2=104946811893537 cn2Label=message identifier cs4=Test email cs4Label=subject suser=sender@sender.com suserLabel=sender duser=receiver@receiver.com duserLabel=receiver act=allowed reason=anti_malware cs3= cs3Label=Malware name

IPS/IDS

Field Description
rt The time of the event, without Time Zone
cn3 The IPS/IDS signature identifier that triggered the event
cn3Label Descriptive text for cn3. The value is signature_id.
src The source IP address of the packet
spt The source port of the packet (if applicable)
dst The destination IP address of the packet
dpt The destination port of the packet (if applicable)
proto The protocol of the packet (TCP, UDP)
act The action for the IPS/IDS event. For an allowed IPS/IDS event the value is logged, whereas for a blocked IPS/IDS event, the value is blocked.
cs5 The class-type of the IPS/IDS signature that triggered the event
cs5Label Descriptive text for cs5. The value is class-type
msg The IPS message associated with the event

A typical log entry for a blocked IPS event is as follows:

Aug 14 2020 14:58:59 mybranch CEF:0|Citrix|SD-WAN|11.2.2.53|EdgeSec|HTTP|7|rt=Aug 14 2020 14:58:59 cn3=2210051 cn3Label=signature_id src=192.168.0.2 spt=1944 dst=192.168.1.2 dpt=22 proto=TCP act=blocked cs5=protocol-command-decode cs5Label=class-type msg=SURICATA STREAM Packet with broken ack

A typical log entry for a logged IPS event is as follows:

Oct 8 2020 12:57:36 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|IPS|4|rt=Oct 8 2020 12:57:36 cn3=2210051 cn3Label=signature_id src=192.168.0.2 spt=1076 dst=192.168.1.2 dpt=22 proto=TCP act=logged cs5=protocol-command-decode cs5Label=class-type msg=SURICATA STREAM Packet with broken ack

Session Events

The Session log entries capture user activity at the TCP layer. They complement HTTP, FTP, and SMTP events by providing insight with regards to TCP session duration, and start and stop timestamps. Session log entries can refer to session start, session end, or update events.

Field Description
rt The time of the event, without Time Zone. For session start events, the timestamp is when the session was established.
cn1 The session identifier
cn1Label Descriptive text for the cn1 field. The value is session_Id.
cs1 The configured security policy
cs1Label Descriptive text for the cs1 field. The value is Security profile.
src The source IP address (client-side)
spt The source port (client-side)
dst The destination IP address (server-side)
dpt The destination port address (server-side)
act The type of the session event. It can be new_session for events referring to a newly established session, session_update, for events referring to existing long-lived sessions, and session_closed, for events referring to sessions closing.
end The time the session closed without time zone. The field is only applicable to session_closed events.

A typical log entry for a session start event is as follows:

Oct 7 2020 23:46:44 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|Session|0|rt=Oct 7 2020 23:46:44 cn1=104946811892916 cn1Label=session_id cs1=Test_Prof1 cs1Label=Security profile src=192.168.0.2 spt=43838 dst=10.78.242.11 dpt=53 act=new_session

A typical log entry for a session end event is as follows:

Oct 7 2020 23:46:46 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|Session|0|rt=Oct 7 2020 23:46:45 cn1=104946811892917 cn1Label=session_id cs1=1 cs1Label=Security profile end=1602114405989 src=192.168.0.2 spt=42253 dst=10.78.242.11 dpt=53 act=session_closed

Session Updates

The Session updates log entries capture user activity at the TCP layer for long-running sessions on a per-minute basis. Session updates help identify existing log entries (HTTP, SMTP, FTP, and session events) that correspond to still open sessions. The corresponding events can either be ignored, or treated as tentative, since session closure might update certain attributes (that is, session end time). Contrary to session_start and session_closed events, the session_update events only have a limited subset of the fields.

Field Description
rt The time of the event, without Time Zone
cn1 The session identifier
cn1Label Descriptive text for the cn1 field. The value is session_Id.
act The action for the IPS/IDS event. For a session update, the value is session_update.

A typical log entry for a session update event is as follows:

Oct 7 2020 23:47:00 mybranch CEF:0|Citrix|SD-WAN|11.2.2.2|EdgeSec|Session|0|rt=Oct 7 2020 23:47:00 cn1=104946811892912 cn1Label=session_id act=session_update

Network Troubleshooting