Troubleshoot authentication issues in Citrix ADC and Citrix Gateway with aaad.debug module

This section describes how to troubleshoot authentication issues in Citrix ADC and Citrix Gateway with aaad.debug module.

Background

Authentication in Citrix Gateway is handled by the Authentication, authorization, and auditing (AAA) daemon. The raw authentication events that AAA daemon processes can be monitored by viewing the output of the aaad.debug module and serves as a valuable troubleshooting tool. The aaad.debug is a pipe as opposed to a flat file and does not display the results or log them. Therefore, the cat command can be used to view the output of aaad.debug. The process of using nsaaad.debug to troubleshoot an authentication problem is typically referred to as “debugging aaad”.

Debug process using aaad.debug module

This process is useful for troubleshooting authentication issues such as:

  • General authentication errors
  • Username/password failures
  • Authentication policy configuration errors
  • Group extraction discrepancies

Note: This process applies to Citrix Gateway and Citrix ADC appliance.

Troubleshooting authentication issues

To troubleshoot authentication with aaad.debug module, complete the following procedure:

  1. Connect to the Citrix Gateway command line interface with a Secure Shell (SSH) client such as PuTTY.

  2. Run the following command to switch to the shell prompt: shell
  3. Run the following command to change to the /tmp directory: cd /tmp
  4. Run the following command to start the debugging process: cat aaad.debug
  5. Perform the authentication process that requires troubleshooting, such as a user logon attempt.
  6. Monitor the output of the cat aaad.debug command to interpret and troubleshoot the authentication process.
  7. Stop the debugging process by pressing Ctrl+Z.
  8. Run the following command to record the output of aaad.debug to a file: cat aaad.debug | tee /var/tmp/<debuglogname>, where /var/tmp is the required directory path and <debuglogname.log> is the required log name.

The following section provides examples of how aaad.debug module can be used to troubleshoot and interpret an authentication error.

Incorrect password

In the following example, the user enters an incorrect RADIUS password.

process_radius Got RADIUS event
process_radius Received BAD_ACCESS_REJECT for: <username>
process_radius Sending reject.
send_reject_with_code Rejecting with error code 4001

Invalid username

In the following example, the user enters an incorrect LDAP username.

/home/build/rs_121/usr.src/netscaler/aaad/ldap_drv.c[450]: receive_ldap_user_search_event 1-140: Admin authentication(Bind) succeeded, now attempting to search the user testusernew

/home/build/rs_121/usr.src/netscaler/aaad/ldap_drv.c[453]: receive_ldap_user_search_event 1-140: Number of entires in LDAP server response = 0

/home/build/rs_121/usr.src/netscaler/aaad/ldap_drv.c[459]: receive_ldap_user_search_event 1-140: ldap_first_entry returned null, user testusernew not found

/home/build/rs_121/usr.src/netscaler/aaad/naaad.c[4781]: send_reject_with_code 1-140: Not trying cascade again 4009

/home/build/rs_121/usr.src/netscaler/aaad/naaad.c[4783]: send_reject_with_code 1-140: sending reject to kernel for : testusernew

/home/build/rs_121/usr.src/netscaler/aaad/naaad.c[4801]: send_reject_with_code 1-140: Rejecting with error code 4009

Determining group extraction results

In the following example, the group extraction results can be determined. Many issues with AAA group access involve the user not picking up the correct session policies for their assigned group in a Citrix Gateway appliance. Some common reasons for this include an incorrect spelling of AD or the Radius group name in the appliance and users not being a member of the security group in AD or the Radius server.

/usr/home/build/rs_80_48/usr.src/usr.bin/nsaaad/../../netscaler/aaad/ldap_drv.c[40]:
start_ldap_auth attempting to auth scottli @ 10.12.33.216

/usr/home/build/rs_80_48/usr.src/usr.bin/nsaaad/../../netscaler/aaad/ldap_drv.c[291]:

/usr/home/build/rs_80_48/usr.src/usr.bin/nsaaad/../../netscaler/aaad/ldap_drv.c[551]: recieve_ldap_user_search_event built group string for scottli of:Domain Admins

aaad.debug module error codes

The following table lists the various aaad.debug module error codes, cause for the error, and the resolution.

aaad.debug module error codes Error message Cause of error Resolution
4001 Incorrect credentials/Password. Try again. Incorrect credentials are supplied Enter the correct credentials
4002 Not Permitted This is a catch all error. Occurs when ldapbind operation fails for reasons other than incorrect user credentials. Make sure bind operation is permitted
4003 Cannot connect to server. Try connecting again in a few minutes. Server times out Increase the LDAP/Radius server timeout value on Citrix ADC (Authentication > LDAP/Radius > Server > Timeout value). Default timeout value is 3 secs.
4004 System Error Citrix ADC/Citrix Gateway internal error or a runtime error in the appliance library Check for the cause of the system error and reslove the same
4005 Socket Error Socket error while talking to the authentication server Make sure that LDAP/RADIUS server or any other authentication servers can listen on the ports mentioned in the authentication action configured on Citrix ADC. For example, a common error scenario can be that an ldapprofile on Citrix ADC is configured to use port 636 /SSL, however the same port is not open on the AD.
4006 Incorrect username Bad (format) username passed to nsaaad, like empty username Enter the correct username
4007 Incorrect password Bad (format) password passed to nsaaad Enter the correct password
4008 Passwords do not match Password mismatch Enter the correct password
4009 User not found No such user Login using a valid user present in AD
4010 You do not have permission to log on at this time Restricted log on hours Log on outside the restricted hours
4011 Your AD account is disabled Account disabled Get your AD account enabled
4012 Your password has expired Password expired Reset your password
4013 You do not have permission to log on No dial-in permission (RADIUS specific). This usually happens if a user is not authorized to authenticate at a server. Network access permission setting needs to be changed
4014 Could not change your password Error in changing password. This can happen due to many reasons. One such reason could be trying to change the password using the non-ssl port provided in ldapprofile on Citrix ADC. Ensure secure port and sec type is used for changing password
4015 Your account is temporarily locked User AD account is locked Get your AD account unlocked
4016 Could not update your password. The password must meet the length, complexity, and history requirements of the domain. User password requirements not met while changing the password Meet the needed requirements while changing the password
4017 NAC process Microsoft Intune Specific. Citrix Gateway is unable to verify device, either due to API failure or connectivity failure. Ensure Microsoft Intune managed devices are reachable to Citrix Gateway
4018 NAC Noncompliance Microsoft Intune returns a status saying that this device is not compliant device Ensure Microsoft Intune managed devices are compliant with Citrix Gateway
4019 NAC Unmanaged Microsoft Intune returns a status saying that this is not a managed device Ensure Microsoft Intune specific configurations are in place
4020 Authentication not Supported This error is seen in case of misconfiguration. For example, auth type is not supported by Citrix ADC or if the Authentciationprofile config is incorrect on Citrix ADC appliance or if accounting action (radius) is attempted for authentication. Check the Authentication checkbox for the Authentication server on Citrix ADC if its unchecked. Use appropriate Authentication action on Citrix ADC.
4021 User Account Expired The user account has expired Get your user account renewed
4022 User account is locked by Citrix ADC The user account is locked by Citrix ADC Unlock the account using unlock aaa user <> command
4023 Max OTP Device limit reached Device limit for receiving OTP reached Either try unregistering the non-required OTP devices or continue with the ones already registered

Troubleshoot authentication issues in Citrix ADC and Citrix Gateway with aaad.debug module