Citrix SD-WAN

AppFlow and IPFIX

AppFlow and IPFIX are flow export standards used to identify and collect application and transaction data in the network infrastructure. This data gives better visibility into application traffic utilization and performance.

The collected data, called flow records are transmitted to one or more IPv4 or IPv6 collectors. The collectors aggregate the flow records and generate real-time or historical reports.

AppFlow

AppFlow exports flow level data for HDX / ICA connections only. You can enable either the TCP only for HDX dataset template or the HDX dataset template. The TCP only for HDX dataset provides multi-hop data. The HDX dataset provides HDX insight data.

AppFlow Collectors like Splunk and Citrix ADM have dashboards to interpret and present these templates.

IPFIX

IPFIX is a collector export protocol used for exporting flow level data for all connections. For any connection, you can view information such as packet count, byte count, type of service, flow direction, routing domain, application name and so on. IPFIX flows are transmitted through the management interface. Most collectors can receive IPFIX flow records, but may need to build a custom dashboard to interpret IPFIX template.

The IPFIX template defines the order in which the data stream is to be interpreted. The collector receives a template record, followed by the data records. Citrix SD-WAN uses templates 611 and 613 to export IPv4 IPFIX flow data, 615 and 616 to export IPv6 IPFIX flow data along with Options template 612.

Application Flow Info (IPFIX) exports data sets as per templates 611 for IPv4 flows, 615 for IPv6 flows and 612 options Template with Application info.

Basic Properties (IPFIX) exports data sets as per templates 613 for IPv4 flows and 616 for IPv6 flows.

The following tables provide the detailed list of flow data associated with each IPFIX template.

Application Flow Info (IPFIX) - V10 templates

Template ID - 611

Info Element (IE) IE name & ID Type and len Description
Observation point ID observationPointId, 138 Unsigned32, 4  
Export process ID exportingProcessId, 144 Unsigned32, 4  
Flow ID flowId, 148 Unsigned64, 8  
Ipv4 SRC IP sourceIPv4Address, 8 Ipv4address, 4  
Ipv4 DST IP destinationIpv4Addres, 12 Ipv4address, 4  
Ipversion ipVersion, 60 Unsigned8, 1 Set to 4.
IP protocol number protocoldentifier,4 Unsigned8, 1  
Padding N/A Unsigned16, 2  
SRC Port sourceTransportPort, 7 Unsigned16, 2  
DST Port destinationTransportPort,11 Unsigned16, 2  
Pkt Count packetDeltaCount, 2 Unsigned64, 8  
Byte Count octetDeltaCount, 1 Unsigned64, 8  
Time for first pkt in microseconds flowStartMicroseconds, 154 dateTimeMicroseconds, 8  
Time for lastpkt in microseconds flowEndMicroseconds, 155 dateTimeMicroseconds, 8  
IP ToS ipClassOfService, 5 Unsigned8, 1  
Flow Flags tcpControlBits, 6 Unsigned8, 2 Currently set to 0.
Flow Direction flowDirection, 61 Unsigned8, 1 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN
Input Interface ingressInterface, 10 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.
Output Interface egressInterface, 14 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.
Input Vlan ID vlanId, 58 Unsigned16, 2  
Output Vlan ID postVlanId, 59 Unsigned16, 2  
VRF ID ingressVRFID, 234 Unsigned32, 4  
Flow Key Indicator flowKeyIndicator, 173 Unsigned64, 8 Set to 0x1E037F.
Application ID applicationId, 95 octetArray, variable The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update.

Template ID – 615 (IPv6 flows)

Info Element (IE) IE name & ID Type and len Comment  
Observation point ID observationPointId, 138 Unsigned32, 4  
Export process ID exportingProcessId, 144 Unsigned32, 4    
Flow ID flowId, 148 Unsigned64, 8    
Ipv6 SRC IP sourceIPv6Address, 27 Ipv6address, 16    
Ipv6 DST IP destinationIpv6Addres, 28 Ipv6address, 16    
Ipversion ipVersion, 60 Unsigned8, 1 Set to 6  
IP protocol number protocoldentifier, 4 Unsigned8, 1    
Padding N/A Unsigned16, 2    
SRC Port sourceTransportPort, 7 Unsigned16, 2    
DST Port destinationTransportPort, 11 Unsigned16, 2    
Pkt Count packetDeltaCount, 2 Unsigned64, 8    
Byte Count octetDeltaCount, 1 Unsigned64, 8    
Time for first pkt in microseconds flowStartMicroseconds, 154 dateTimeMicroseconds, 8    
Time for lastpkt in microseconds flowEndMicroseconds, 155 dateTimeMicroseconds, 8    
IP ToS ipClassOfService, 5 Unsigned8, 1    
Flow Flags tcpControlBits, 6 Unsigned8, 2 Currently set to 0.  
Flow Direction flowDirection, 61 Unsigned8, 1 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN  
Input Interface ingressInterface, 10 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.  
Output Interface egressInterface, 14 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.  
Input Vlan ID vlanId, 58 Unsigned16, 2    
Output Vlan ID postVlanId, 59 Unsigned16, 2    
VRF ID ingressVRFID, 234 Unsigned32, 4    
Flow Key Indicator flowKeyIndicator, 173 Unsigned64, 8 Set to 0x1E037F.  
Application ID applicationId, 95 octetArray, variable The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update.  

Template 612 (Options Template)

Info Element (IE) IE name & ID Type Comment
Application ID applicationId, 95 octetArray The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update.
Application Name applicationName, 96 string Specifies the name of the Citrix SDWAN specific proprietary application.
Application Description applicationDescription, 94 string Specifies the description of the application.

Basic Properties (IPFIX) – V9 compliant template - Template 613 (IPv4 flows)

Info Element (IE) IE name & ID Type and len Comment
Ipv4 SRC IP sourceIPv4Address, 8 Ipv4address, 4  
Ipv4 DST IP destinationIpv4Addres, 12 Ipv4address, 4  
Ipversion ipVersion, 60 Unsigned8, 1  
IP protocol number protocoldentifier, 4 Unsigned8, 1  
IP ToS ipClassOfService, 5 Unsigned8, 1  
Flow Direction flowDirection, 61 Unsigned8, 1 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN
SRC Port sourceTransportPort, 7 Unsigned16, 2  
DST Port destinationTransportPort, 11 Unsigned16, 2  
Pkt Count packetDeltaCount, 2 Unsigned64, 8  
Byte Count octetDeltaCount, 1 Unsigned64, 8  
Input Interface ingressInterface, 10 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.
Output Interface egressInterface, 14 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.
Input Vlan ID vlanId, 58 Unsigned16, 2  
Output Vlan ID postVlanId, 59 Unsigned16, 2  

Template ID – 616 (IPv6 flows)

Info Element (IE) IE name & ID Type and len Comment  
Ipv6 SRC IP sourceIPv6Address, 27 Ipv6address, 16    
Ipv6 DST IP destinationIpv6Addres, 28 Ipv6address, 16    
Ipversion ipVersion, 60 Unsigned8, 1 Set to 6  
IP protocol number protocoldentifier,4 Unsigned8, 1    
IP ToS ipClassOfService, 5 Unsigned8, 1    
Flow Direction flowDirection, 61 Unsigned8, 1 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN  
SRC Port sourceTransportPort, 7 Unsigned16, 2    
DST Port destinationTransportPort, 11 Unsigned16, 2    
Pkt Count packetDeltaCount, 2 Unsigned64, 8    
Byte Count octetDeltaCount, 1 Unsigned64, 8    
Input Interface ingressInterface, 10 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.  
Output Interface egressInterface, 14 Unsigned32, 4 Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations.  
Input Vlan ID vlanId, 58 Unsigned16, 2    
Output Vlan ID postVlanId, 59 Unsigned16, 2    

Limitations

  • AppFlow does not support IPv6 collector and flow records.
  • The export interval for Net Flow is increased from 15 seconds to 60 seconds.
  • AppFlow/IPFIX flows are transmitted over UDP, on connection loss not all data is retransmitted. If the export interval is set to X minutes, the appliance stores X minutes of data only. Which is retransmitted after X minutes of connection loss.
  • In Citrix SD-WAN, release 10 version 2 the AppFlow settings are made local to every appliance, while in the previous releases it was a global setting. If the SD-WAN software release is downgraded to any of the previous releases and if AppFlow is configured on any one of the appliances, it will be applied globally to all alliances.

Configuring AppFlow/IPFIX

You can configure AppFlow / IPFIX only through Citrix SD-WAN Orchestrator service. For more informtion, see AppFlow and IPFIX.

Log files

For troubleshooting issues related to AppFlow / IPFIX export protocols, you can view and download the SDWAN_export.log files. Navigate to Configuration > Logging / Monitoring and select the SDWAN_export.log files.

Logging and monitoring

AppFlow and IPFIX