Product Documentation

IP Reputation

Jan 26, 2017

IP reputation is an extremely effective tool in identifying the IP address that is sending unwanted requests. You can use the IP reputation list to preemptively reject requests that are coming from the IP with the bad reputation. For example, you can use this feature to optimize application firewall performance by filtering out the requests that you do not want to process. You can reset or drop the connection, or you can configure a responder policy to take a specific responder action.

Following are some examples of attacks that you can prevent by using IP Reputation:

  • Virus Infected personal computers (home PCs) are the single biggest source of Spam on the internet. IP Reputation can identify the IP address that is sending unwanted requests. IP reputation can be especially useful for blocking large scale DDoS, DoS, or anomalous syn flood attacks from known infected sources.
  • Centrally managed and automated botnet attacks have gained popularity for stealing passwords, because it doesn’t take long when hundreds of computers work together to crack your password. It is easy to launch botnet attacks to figure out passwords that use commonly used dictionary words.
  • Compromised web-server attacks are not as common as they used to be, because awareness and server security have increased, so hackers and spammers look for easier targets. There are still web servers and online forms that hackers can compromise and use to send spam (often the more vicious types, such as viruses and porn) but usually this type of activity is easier to detect and and quickly shut down, or block with a reputation list such as SpamRats.
  • Windows Exploits (such as Active IPs offering/distributing malware, shell code, rootkits, worms or viruses).
  • Known spammers and hackers.
  • Mass e-mail marketing campaigns.
  • Phishing Proxies (IP addresses hosting phishing sites, and other fraud such as ad click fraud or gaming fraud).
  • Anonymous proxies (IPs providing proxy and anonymization services including The Onion Router aka TOR).

A NetScaler appliance uses Webroot as the service provider for the dynamically generated malicious IP database and the metadata for those IP addresses. Metadata might include geolocation details, threat category, threat count, and so on. The Webroot threat Intelligence engine receives real-time data from millions of sensors. It automatically and continuously captures, scans, analyses and scores the data, using advanced machine learning and behavioral analysis.  Intelligence about a threat is continually updated.

As soon as a threat is detected anywhere in the network, the IP address is flagged as malicious and all appliances connected to the network are immediately protected. The dynamic changes in the IP addresses are processed with high speed and accuracy by leveraging advanced machine learning. 

As stated in the datasheet from Webroot, the Webroot’s sensor network identifies many key IP threat types, including spam sources, Windows exploits, botnets, scanners, and others. (See the flow diagram on the datasheet)

The NetScaler appliance uses an iprep client process to get the database from Webroot. The iprep client uses the HTTP GET method to get the absolute IP list from Webroot for the first time. Subsequently, it checks delta changes once every 5 minutes. 

Important

Make sure the NetScaler appliance has Internet access and DNS is configured before you use the IP Reputation feature.

To access the webroot database, the NetScaler appliance should be able to connect to api.bcti.brightcloud.com on port 443. Each node in the High Availability (HA) or Cluster deployment gets the database directly from the webroot and should be able to access this FQDN (Fully Qualified Domain Name).

Webroot hosts it's reputation database in AWS currently. Therefore, NetScaler should be able to resolve AWS domains for downloading the reputation db.  Also, firewall should be open for AWS domains.

NetScaler applaince should be able to connect to wiprep-rtu.s3-us-west-2.amazonaws.com on port 443 to obtain IP data from AWS.

Note

NSPPE might fail with high CPU and insufficient memory due to the MEM_NETWORK and MEM_IPREPUTATION modules. These modules consume high CPU and memory. When this happens, it is recommended to test the NetScaler appliance by disabling IP reputation feature using the following command:

- disable ns feature Rep

Note that each packet engine requires at least 4GB to function properly when IP Reputation feature is enabled.

 

 

PI Expressions: The IP Reputation feature can be configured by using PI expressions (NetScaler default syntax expressions) in the policies bound to supported modules such as application firewall and responder. Following are two examples showing expressions that can be used to detect whether the client IP address is malicious

  1. CLIENT.IP.SRC.IPREP_IS_MALICIOUS: This expression evaluates to TRUE if the client is included in the malicious IP list.
  2. CLIENT.IP.SRC.IPREP_THREAT_CATEGORY (CATEGORY): This expression evaluates to TRUE if the client IP is malicious IP and is in the specified threat category. 

Following are the possible values for the threat category:

SPAM_SOURCES,

WINDOWS_EXPLOITS,

WEB_ATTACKS,

BOTNETS,

SCANNERS,

DOS,

REPUTATION,

PHISHING,

PROXY,

NETWORK,

CLOUD_PROVIDERS

MOBILE_THREATS.

Note

The IP reputation feature can check both source and destination IP addresses. It can also detect malicious IPs in the header. If the PI Expression in a policy can identify the IP address, the IP reputation check can determine whether it is malicious.  

IPRep log message. The /var/log/iprep.log file contains useful messages that capture information about communication with the Webroot database. The information can be about the credentials used during Webroot communication, failure to connect with Webroot, what is included in an update (such as number of IP addresses in the database), and so on. 

Creating a blacklist or whitelist of IPs using policy data set.  You can maintain a whitelist to allow access to specific IP addresses that are blacklisted in the Webroot database. You can also create a customized blacklist of IP addresses to supplement the Webroot reputation check. These lists can be created by using policy data set. A data set is a specialized form of pattern set that is ideally suited for IPv4 address matching. To use data sets, first create the data set and bind IPv4 addresses to it. Then, when you configure a policy for comparing a string in a packet, use an appropriate operator and pass the name of the pattern set or data set as an argument.

To use the dataset to create a customized whitelist of addresses to treat as exceptions during IP reputation evaluation, configure the policy so that the PI expression evaluates to False even if an address in the whitelist is listed as malicious by Webroot (or any service provider).

Enabling or disabling IP reputation. IP reputation is a part of the general reputation feature, which is license based.  When you enable or disable the reputation feature, it enables or disables IP Reputation.

General procedure. Deploying IP reputation involves the following tasks

  • Verify that the license installed on the NetScaler appliance has IP reputation support. Platinum and standalone application firewall licenses support the IP reputation feature.
  • Enable the IP reputation and application firewall features.
  • Add an application firewall profile.
  • Add an application firewall policy using the PI expressions to identify the malicious IP addresses in the IP Reputation database. 
  • Bind the application firewall policy to an appropriate bind point.
  • Verify that any request received from a malicious address gets logged in the ns.log file to show that the request was processed as specified in the profile. 

Using the Command Line to Configure the IP Reputation Feature

To enable IP reputation using CLI, you can use the following commands:

> enable feature [rep | reputation]

> disable feature [rep | reputation]

The following examples show how you can add an application firewall policy using the PI expression to identify malicious addresses. You can use the built-in profiles, or add a new profile, or configure an existing profile to invoke the desired action when a request matches a policy match.

Examples 3 and 4 show how to create a policy dataset to generate a Black (to be blocked) or White (to be allowed) list of IP addresses.  

Example 1

The following command creates a policy that identifies malicious IP addresses and block the request if a match is triggered:

> add appfw policy pol1 CLIENT.IP.SRC.IPREP_IS_MALICIOUS APPFW_BLOCK

Example 2

The following command creates a policy that uses the reputation service to check the client IP address in a specific header (X-Forwarded-For) and reset the connection if a match is triggered:

> add appfw policy pol1 "HTTP.REQ.HEADER(\"X-Forwarded-For\").TYPECAST_IP_ADDRESS_AT.IPREP_IS_MALICIOUS" APPFW_RESET

Example 3

The following example shows how to add a list to add exceptions that allow specified IP addresses:

> add policy dataset Allow_list ipv4

> bind policy dataset Allow_list 10.217.25.17 -index 1

> bind policy dataset Allow_list 10.217.25.18 -index 2

Example 4

The following example shows how to add the customized list to flag specified IP addresses as malicious:

> add policy dataset Block_List ipv4

> bind policy dataset Block_List 10.217.31.48 -index 1

> bind policy dataset Block_List 10.217.25.19 -index 2

Example 5

The following example shows a policy expression to block the client IP if it matches an IP address configured in the customized Block_list (example 4) or if it matches an IP address listed in the Webroot database unless relaxed by inclusion in the Allow_list (example 3).

> add appfw policy "Ip_Rep_Policy" "((CLIENT.IP.SRC.IPREP_IS_MALICIOUS || CLIENT.IP.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Block_List\")) && ! (CLIENT.IP.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Allow_List\")))" APPFW_BLOCK

Using Proxy server

If the NetScaler appliance does not have direct access to the Internet and is connected to proxy, use the following command to configure the iprep client to send requests to the proxy.

> set reputation settings –proxyServer <proxy server ip> -proxyPort <proxy server port> 

Example

> set reputation settings proxyServer 10.102.30.112 proxyPort 3128

> set reputation settings –proxyServer testproxy.citrite.net  –proxyPort 3128

> unset reputation settings –proxyserver –proxyport

> sh reputation settings

Note

The Proxy Server IP can be an IP address or a fully qualified domain name (FQDN). 

Using the GUI to Configure the IP Reputation Feature

To enable reputation (which enables IP reputation) feature in GUI

Navigate to the System -> Settings. In the Modes and Features section, click the link to access the Configure Advanced Features pane and enable the Reputation check box. Click OK

localized image

To configure a proxy server by using the configuration utility

1. On the configuration tab, navigate to Security > Reputation. Under Settings, click Change Reputation Settings to configure a proxy server. You can also enable or disable the reputation feature.  Proxy Server can be an IP address or a fully qualified domain name (FQDN). Proxy port accepts values between [1 – 65535].

localized image

To use a dataset to create a whitelist (list of safe IPs) and blacklist (list of unsafe IPs) of Client IP addresses

  • On the Configuration tab, navigate to AppExpert > Data Sets.
  • Click Add
localized image
  • In the Create Data Set (or Configure Data set) pane, provide a meaningful name for the list of the IP addresses. The name should reflect the purpose of the list. For example, you could use a name such as whitelist or Allow_list when creating a list of IP addresses you would like to exempt from the action taken when the IP Reputation check determines that the source IP matches an IP address designated as a malicious IP in the webroot database.  Similarly, use names such as Block_list or Malicious_ip_list if you are adding a list of IP addresses you want to flag as malicious to supplement the webroot database. 
  • Select *Type as IPv4.
  • Click Insert to add a new entry.
localized image
  • In the Configure Policy dataset binding pane, add an IPv4 format IP address in the *Value input box.
  • Provide an index.
  • Add a comment that explains the purpose of the list. This step is optional, but is recommended because a descriptive comment is quite helpful in managing the list. 

Similarly, you can create a Block_list and add the IP addresses that are to be considered malicious. 

Also see, http://docs.citrix.com/en-us/netscaler/11/appexpert/pattern-sets-data-seta.html for additional details regarding using data sets and configuring default syntax policy expressions. 

To configure an application firewall policy by using the configuration utility

1. On the Configuration tab, navigate to Security > Application Firewall > Policies > Firewall. Click Add to add a new policy using the PI expressions to use IP reputation. 

localized image

You can also use the Expression editor to build your own policy expression. The drop-down menu shows preconfigured options that are quite useful for configuring an expression using the threat categories.

localized image

Highlights:

  • Quickly and accurately stop bad traffic at the network’s edge from known malicious IP addresses posing different types of threats. You can block the request without parsing the body.
  • Dynamically configure IP reputation functionality for multiple applications.  
  • Secure your network against data breach without a performance penalty, and consolidate protections onto a single services fabric using fast and easy deployments.
  • You can do IP Reputation checks on source as well as destination IPs.
  • You can also inspect the headers to detect malicious IPs.
  • IP reputation check is supported in both forward proxy and reverse proxy deployments.  
  • The iprep process connects with Webroot and updates the database every 5 minutes.
  • Each node in the High Availability (HA) or Cluster deployment gets the database from Webroot.
  • The IP reputation data is shared across all partitions in admin-partition deployments.
  • You can use an AppExpert data set to create lists of IP addresses to add exceptions for IPs blacklisted in the Webroot database. You can also create your own customized blacklist to designate specific IPs as malicious.
  • The iprep.db file is created in the /var/nslog/iprep folder. Once created, it is not deleted even if the feature is disabled. 
  • When the reputation feature is enabled, the NetScaler Webroot database is downloaded. After that, it is updated every 5 minutes.
  • The Webroot database major version is currently version: 1.
  • The minor version gets updated every day.  The update version is incremented after every 5 minutes and is reset back to 1 when the minor version is incremented.
  • PI expressions enable you to use IP reputation with other features, such as responder and rewrite.
  • The IP addresses in the database are in decimal notation.  

Debugging Tips:

  • If you cannot see the reputation feature in GUI, verify that you have the right license
  • Monitor the messages in /var/log/iprep.log for debugging.
  • Webroot connectivity: If you see the "ns iprep: Not able to connect/resolve WebRoot" message, make sure that the appliance has internet access and DNS is configured. 
  • Proxy server: If you see the "ns iprep: iprep_curl_download:88 curl_easy_perform failed. Error code: 5 Err msg:couldnt resolve proxy name" message, make sure that the proxy server configuration is accurate.
  • IP Reputation feature not working: The iprep process takes about five minutes to start after you enable the reputation feature. The IP reputation feature might not work for that duration.