How to Recover Inaccessible NetScaler MAS Servers

The maximum size of the NetScaler MAS database depends on the system database model, and is not the same on all systems. If the size of your database exceeds the threshold, the database becomes inaccessible, which might cause your NetScaler MAS server to not respond. In such cases, you can use the following procedures to connect to the file system, delete a few components, and make the database accessible. You can then perform recovery operations. This operation takes approximately an hour to 90-minutes.

The NetScaler MAS recovery script helps you delete some data from the database by prompting you to navigate from one component to another, without affecting the core system files, or the settings of the NetScaler MAS. Files such as the upgrade images, backup files (instance and NetScaler MAS server), Analytics reports, perf reports can be deleted using this operation.

How to Use NetScaler MAS Database Recovery Script for a NetScaler MAS Standalone Deployment

Use the following procedure in a single server NetScaler MAS deployment to connect to the file system, delete a few components, and make the database accessible, and then perform the recovery operations.

  1. Using a SSH client or your hypervisor’s console, log on to NetScaler MAS and  type the following command:

    /mps/mas\_recovery/mas\_recovery.py
    

    localized image

  2. When the screen displays a caution message for stopping a few MAS processes, press the y key. The following screen appears while the system determines which components of the database you can delete without affecting the system’s core files.

    localized image

  3. When a screen similar to the following example appears, decide which components to delete, and then press the y key.”

    localized image

  4. When prompted to delete database components, specify the components as described on the screen. For example:

    localized image

  5. When prompted to delete file-system components, specify the components as decribed on the screen. For example:

    localized image

  6. When confirmation prompt appears, press the y key to stop all NetScaler MAS processes and initiate cleanup of the selected components, or press n to return to the Filesystem components selection screen.

    localized image

  7. Wait for about 30 minutes for internal DB operations to complete after NetScaler MAS has restarted. You should then be able to connect to NetScaler MAS database. If not, run the recovery script again to free up more space. When NetScaler MAS is up and running, it should work as expected.

How to Use NetScaler MAS Database Recovery Script for a NetScaler MAS High Availability Deployment

The database system for NetScaler MAS servers in a high availability deployment will be in continuous synchronization mode. You need to replicate the procedure to use the recovery script individually on both the NetScaler MAS servers. You also need to break the high availability pair and reconfigure it once the recovery operations are done. This operation approximately takes two hours.

Use the following procedure to connect to the file system, delete a few components, and make the database accessible, and then perform recovery operations.

  1. Using a SSH client or your hypervisor’s console, log on to NetScaler MAS.

  2. Run the NetScaler MAS recovery script (/mps/mas_recovery/mas_recovery.py) individually on node 1 and node 2.

    localized image

    Note

    Ensure that you Restart both nodes.

  3. When the screen displays a caution message for stopping a few MAS processes, press the y key. The following screen appears while the system determines which components of the database you can delete without affecting the system’s core files.

    localized image

  4. When a screen similar to the following example appears, decide which components to delete, and then press the y key.”

    localized image

  5. When prompted to delete database components, do not specify any of the listed components, select Continue without selecting. For example:

    localized image

  6. When prompted to delete file-system components, specify the components as decribed on the screen. For example:

    localized image

  7. Once the nodes are up, wait for about 30 minutes for internal database operations to complete.

  8. The database should now be up on both nodes.

  9. Log on to any one of the NetScaler MAS servers using a supported web browser.

  10. Navigate to System > Deployment, and click Break HA.

  11. Using a SSH client or your hypervisor’s console, log on to NetScaler MAS.

  12. Run the NetScaler MAS recovery script (/mps/mas_recovery/mas_recovery.py) individually on node 1 and node 2, and delete the database ** components. For example:

    localized image

    Note

    Make sure that you delete the same database components on both the NetScaler MAS servers, and restart both the nodes.

  13. Let the two nodes run for at least 45 minutes, and then recreate the HA pair by running the deployment script (deployment_type.py) on the console of each node.