Citrix Virtual Apps and Desktops

Architecture Overview

August 25, 2025

Contributed by:

Architecture Diagram

The Always On Tracing (AOT) architecture is designed to provide continuous, real-time trace collection across the Citrix Virtual Apps and Desktops (CVAD) environment. It enables IT administrators and Citrix Support to troubleshoot user and infrastructure issues without requiring manual trace initiation or issue reproduction.

The AOT system is built on several integrated components working in sequence to capture, transfer, and store telemetry logs from multiple Citrix infrastructure layers into a centralized repository.

Key Components and Workflow

AOT Log Generators (Citrix Core Components): Citrix components such as the Virtual Delivery Agent (VDA), Delivery Controller (DDC), StoreFront, and others act as log generators. These components are equipped with the Telemetry Service, which uses the AOT API to capture trace data, record pre-defined step traces, and log error events relevant to user sessions and transactions.
Enable Log Collection: When an administrator initiates a request to collect logs (typically through a powershell command), instructions are sent to the broker, which orchestrates the log collection process across the relevant components, including broker and VDA; the administrator initiates the request to collect logs (through a powershell command) from Storefront Server separately.
Telemetry Service: After configuring the Log Server address and port in Storefront and DDC, the Telemetry Service then activates the new real-time AOT listener, which collects logs based on predefined events, failures, or trigger points.
Log Transfer to Centralized Log Server: Once the logs are collected, the Telemetry Service transfers them securely to a centralized log server directly. In environments where endpoints are connected from external networks through a Citrix Gateway, the transfer typically occurs over a SOCKS tunnel to ensure secure and seamless transfer.
Log Storage and Organization: The Centralized Log Server receives, parses, and stores the AOT logs in a structured and searchable format using an indexed database backend (OpenSearch by default). Logs are tagged by session, component, and timestamp, enabling easy access and efficient troubleshooting.

Logs in the Citrix AOT system are stored in a structured and searchable format, tagged with various fields to enable easy access and efficient troubleshooting. These tags include:

MachineName: The name of the machine where the log originated.
MachineIP: The IP address of the machine.
Role: The role of the Citrix component (e.g., VDA, DDC, StoreFront).
TimeStamp: The UTC timestamp when the log event occurred.
Message: The actual log message content. Could be searched/filtered by words.
Level: The severity level of the log (e.g., info, warning, error).
Module: The specific software module that generated the log.
ProcessName: The name of the process that generated the log.
ProcessId: The ID of the process.
Thread: The thread ID within the process.
Cpu: Information related to CPU id at the time of the log.
SessionId: The ID of the user session associated with the log.
Class: The class or component within the module.

These detailed tags allow administrators to quickly filter, search, and analyze logs based on specific criteria, facilitating the identification and resolution of issues.

In summary, the AOT workflow begins with Citrix components generating diagnostic logs. A centralized request is initiated to collect these logs, which the Telemetry Service gathers from the relevant components. The collected logs are then securely transferred to a centralized log server for indexing, storage, and later analysis.

How AOT Works

Citrix Always On Tracing (AOT) continuously captures diagnostic data from key components in your environment. When triggered, logs are collected automatically, securely transferred to a centralized log server, and retained for analysis. This eliminates the need for manual tracing and simplifies troubleshooting.

Tracing Triggers and Log Collection: Citrix components such as VDA, DDC, StoreFront and other components are equipped with the AOT API to track critical steps and errors. These components serve as AOT log generators. When an administrator initiates a log collection request from the Delivery Controller:

The DDC sends the instruction to relevant Citrix components. Each component forwards the request to its local Telemetry Service.

The Telemetry Service starts the real-time AOT listening program to collectAOT logs, and forwards the logs to the Centralized Log Server.

Log Storage and Retention: Once the AOT logs are received, the Centralized Log Server formats and indexes them into a structured, searchable database. Logs are retained based on a defined retention policy to optimize storage usage. By default, AOT logs are retained for 7 days, after which they are automatically purged to conserve space.

AOT vs Traditional Logging

Aspect	AOT	Traditional CDF Tracing
Activation	Automatic, continuous	Manual, issue reproduction required
Complexity	User-friendly, readable logs	Engineering-level parsing
Diagnostic Speed	Immediate logs available	Delayed by reproduction effort
Resource Usage	Low overhead with optimized buffers	Moderate to high if used incorrectly

Traditional tracing tools require manual effort and only work if you catch the problem while it’s happening. With Always On Tracing (AOT), logging runs continuously in the background, so issues are captured as they happen — even if you’re not watching. This saves time and makes troubleshooting much easier.

AOT Highlights

Always On: Tracing runs all the time, no need to turn it on manually.
Auto Capture: Logs are collected always when issues occur.
Central Log Server: All logs are stored in one place for easy access.
Easier Troubleshooting: Helps quickly pinpoint common failures.
No Reproduction Needed: No need to recreate the issue, it’s already captured.

The official version of this content is in English. Some of the Cloud Software Group documentation content is machine translated for your convenience only. Cloud Software Group has no control over machine-translated content, which may contain errors, inaccuracies or unsuitable language. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, suitability, or correctness of any translations made from the English original into any other language, or that your Cloud Software Group product or service conforms to any machine translated content, and any warranty provided under the applicable end user license agreement or terms of service, or any other agreement with Cloud Software Group, that the product or service conforms with any documentation shall not apply to the extent that such documentation has been machine translated. Cloud Software Group will not be held responsible for any damage or issues that may arise from using machine-translated content.

Was this helpful

Architecture Overview

August 25, 2025

Contributed by:

August 25, 2025

Contributed by:

Architecture Overview

Key Components and Workflow

How AOT Works

AOT vs Traditional Logging

AOT Highlights

In this article