Bot

A bot is a software program that automatically performs certain actions over and over at a much faster rate than a human. Over 35 percent of your web traffic comprises bots and 80 percent of organizations suffer from bot attacks. They can interact with a webpage, submit forms, click links, scan text, or download content. Bots can access videos, post comments, and tweet on social media platforms. Some bots can even hold basic conversations with human users. These are known as chatbots.

A bot that performs a needful or helpful service such as customer service, chatbots, search engine crawlers are known as good bots. Some malicious bots can scrape or download content from a website, steal user credentials, spread spam content, and perform various other kinds of cyberattacks. These malicious bots are known as bad bots. It is essential to identify bad bots and protect your appliance from advanced security attacks. You can achieve this using a bot management system. For more information on Bot, see Bot Management.

Configure Bot detection techniques in Citrix ADC

In Citrix ADC, you can configure bot detection techniques to detect the incoming bot traffic. The following are the bot techniques that you configure in Citrix ADC instance:

  • WhiteList. This rule has a list of URLs and policy expressions to evaluate if a specific set of good bots that can access to your web resource.

  • BlackList. This rule has a list of URLs and policy expressions to evaluate if a specific set of bad bots can access your website.

  • IP reputation. This rule detects if the incoming bot traffic is a malicious IP address.

  • Device fingerprinting. This rule detects if the incoming bot traffic has device fingerprint ID in the incoming request header and browser attributes of an incoming client bot traffic.

  • Rate limiting. This rule rate limits multiple requests coming from the same client.

  • Signatures. This rule detects and blocks bots based on signature detection. It also prevents unauthorized URLs that scrape websites, brute forcing logins, and bots that probe for vulnerabilities.

For more information on configuring Bot management, see Configure Bot management.

Using Bot Insight in Citrix ADM

After you configure the bot management in Citrix ADC, you must enable Bot Insight on virtual servers to view insights in Citrix ADM.

To enable Bot Insight:

  1. Navigate to Networks > Instances > Citrix ADC and select the instance type. For example, VPX.

  2. Select the instance and from the Select Action list, select Configure Analytics.

  3. Select the virtual server and click Enable Analytics.

  4. On the Enable Analytics window:

    1. Select Bot Insight

    2. Under Advanced Option, select Logstream.

      Bot-insight

    3. Click OK.

After enabling Bot Insight, navigate to Analytics > Bot Insight.

Bot-insight

1 - Time list to view bot details

2 – Drag the slider to select a specific time range and click Go to display the customized results

3 – Total instances affected from bots

4 – Virtual server for the selected instance with total bot attacks

  • Total Bots – Indicates the total bot attacks (inclusive of all bot categories) found for the virtual server.

  • Total Human Browsers – Indicates the total human users accessing the virtual server.

  • Bot Human Ratio – Indicates the ratio between human users and bots accessing the virtual server.

  • Signature Bots, Fingerprinted Bot, Rate Based Bots, IP Reputation Bots, Whitelist Bots, and Blacklist Bots – Indicates the total bot attacks occurred based on the configured bot category. For more information about bot category, see Configure Bot detection techniques in Citrix ADC.

5 - Click > to view bot details in a graph format.

Bot-graph

Click the virtual server to view the Application Summary

Bot-application-summary

1 – Provides the Application Summary details such as:

  • Average RPS – Indicates the average bot transaction requests per second (rps) received on virtual servers.

  • Bots by Severity – Indicates the highest bot transactions occurred based on the severity. The severity is categorized based on Critical, High, Medium, and Low.

    For example, if the virtual servers have 11770 high severity bots and 1550 critical severity bots, then Citrix ADM displays Critical 1.55 K under Bots by Severity.

  • Largest Bot Category – Indicates the highest bot attacks occurred based on the bot category.

    For example, if the virtual servers have 8000 Blacklisted bots, 5000 Whitelisted bots, and 10000 Rate Limit Exceeded bots, then Citrix ADM displays Rate Limit Exceeded 10 K under Largest Bot Category.

  • Largest Geo Source – Indicates the highest bot attacks occurred based on a region.

    For example, if the virtual servers have 5000 bot attacks in Santa Clara, 7000 bot attacks in London, and 9000 bot attacks in Bangalore, then Citrix ADM displays Bangalore 9 K under Largest Geo Source.

  • Average % Bot Traffic – Indicates the human bot ratio.

2 – Displays severity of the bot attacks based on locations in map view

3 – Displays the types of bot attacks (Good, Bad, and All)

4 – Displays the total bot attacks along with the corresponding configured actions. For example, if you have configured:

  • IP address range (192.140.14.9 to 192.140.14.254) as blacklist bots and selected Drop as an action for these IP address ranges

  • IP range (192.140.15.4 to 192.140.15.254) as blacklist bots and selected to create Log message as an action for these IP ranges

    In this scenario, Citrix ADM displays:

    • Total blacklisted bots

    • Total bots under Dropped

    • Total bots under Log

To further drill-down for details, click the bot attack type under Bot Category. For example, if you want to view details for Blacklisted bot attacks, click BlackList under Bot Category.

The details such as attack time and total number of bot attacks are displayed.

Bot-drill-down

You can also drag the bar graph to select the specific time range to be displayed with bot attacks.

Bot-time

To get additional information of the bot attack, click to expand.

Bot-expand

  • Instance IP – Indicates the Citrix ADC instance IP address

  • Total Bots – Indicates the total bot attacks occurred for that particular time

  • HTTP Request URL – Indicates the URL that is configured to be blacklisted

  • Country Code – Indicates the country where the bot attack occurred

  • Region – Indicates the region where the bot attack occurred

  • Profile Name – Indicates the profile name that you provided during the configuration

You can also use the search text box and time duration list, where you can view bot details as per your requirement. When you click the search box, the search box gives you the following list of search suggestions.

  • Instance-IP – Citrix ADC instance IP address

  • Client-IP – Client IP address

  • Bot-Type – Bot type such as Good or Bad

  • Severity – Severity of the bot attack

  • Action-Taken – Action taken after the bot attack such as Drop, No action, Redirect

  • Bot-Category – Category of the bot attack such as blacklist, whitelist, fingerprint, and so on. Based on a category, you can associate a bot action to it

  • Bot-Detection – Bot detection types (blacklist, whitelist, and so on) that you have configured on Citrix ADC instance

  • Location – Region/country where the bot attack has occurred

  • Request-URL – URL that has the possible bot attacks

You can also use operators in your search queries to narrow the focus of your search. For example, if you want to view all bad bots:

  1. Click the search box and select Bot-Type

  2. Click the search box again and select the operator =

  3. Click the search box again and select Bad

  4. Click Search to display the results

    Bot-search