Product Documentation

URL Categorization

URL Categorization restricts user access to specific websites and website categories. As a subscribed service offered by Citrix Secure Web Gateway (SWG), the feature enables enterprise customers to filter web traffic by using a commercial categorization database. The database has a vast number (billions) of URLs classified into different categories, such as social networking, gambling, adult content, new media, and shopping. In addition to categorization, each URL has a reputation score kept up to date based on the site’s historical risk profile. To filter your traffic, you can configure advanced policies based on categories, category groups (such as Terrorism, Illegal drugs), or site-reputation scores.

For example, you might block access to dangerous sites, such as sites known to be infected with malware, and selectively restrict access to content such as adult content or entertainment streaming media for enterprise users. You can also capture the user’s transactional details and outbound traffic details for monitoring web traffic analytics on the Citrix ADM server.

How URL Categorization Works

The following figure shows how Citrix SWG URL categorization service is integrated with a commercial URL Categorization database and cloud services for frequent updates.

localized image

The components interact as follows:

  1. A client sends internet bound URL request.

  2. The Citrix SWG proxy applies a policy enforcement to the request based on the category details (such as, category, category group, and site-reputation score) retrieved from the URL categorization database. If the database returns the category details, the process jumps to step 5.

  3. If the database misses the categorization details, the request is sent to a cloud-based lookup service maintained by a URL categorization vendor.  However, the appliance does not wait for a response, instead, the URL is marked as uncategorized and a policy enforcement is performed (jump to step 5). The appliance continues to monitor the cloud query feedback and updates the cache so that future requests can benefit from the cloud lookup.

  4. The SWG appliance receives the URL category details (category, category group, and reputation score) from the cloud-based service and stores it in the categorization database.

  5. The policy allows the URL and the request is sent to the origin server. Otherwise, the appliance drops, redirects, or responds with a custom HTML page.

  6. The origin server responds with the requested data to the SWG appliance.

  7. The appliance sends the response to the client.

Use Case: Internet Usage under Corporate Compliance for Enterprises

You can use the URL Filtering feature to detect and implement compliance policies to block sites that violate corporate compliance. These could be sites such as adult, streaming media, social networking which could be deemed nonproductive or consume excess internet bandwidth in an enterprise network. Blocking access to these web sites can improve employee productivity, reduce operating costs for bandwidth usage, and reduce the overhead of network consumption.

Prerequisites

The URL Categorization feature works on a Citrix SWG platform only if it has an optional subscription service with URL filtering capabilities and threat intelligence for Citrix Secure Web Gateway.  The subscription allows customers to download the latest threat categorizations for websites and then enforce those categories on the Secure Web Gateway. The subscription is available for both hardware appliances and software (VPX) versions of Secure Web Gateway. Before enabling and configuring the feature, you must install the following licenses: CNS_WEBF_SSERVER_Retail.lic

CNS_XXXXX_SERVER_SWG_Retail.lic.

Where, XXXXX is the platform type, for example: V25000

Responder Policy Expressions

The following table lists the different policy expressions that you can use to verify if an incoming URL must be allowed, redirected, or blocked.

  1. <text>. URL_CATEGORIZE (<min_reputation>, <max_reputation>) - Returns a URL_CATEGORY object. If is greater than 0, the returned object does not contain a category with a reputation lower than . If is greater than 0, the returned object does not contain a category with a reputation higher than . If the category fails to resolve in a timely manner, the undef value is returned.
  2. <url_category>. CATEGORY() - Returns the category string for this object. If the URL does not have a category, or if the URL is malformed, the returned value is “Unknown.”
  3. <url_category>. CATEGORY_GROUP() - Returns a string identifying the object’s category group. This is a higher level grouping of categories, which is useful in operations that require less detailed information about the URL category. If the URL does not have a category, or if the URL is malformed, the returned value is “Unknown.”
  4. <url_category>. REPUTATION() - Returns the reputation score as a number from 0 to 5, where 5 indicates the riskiest reputation. If there is the category is “Unknown”, the reputation value is 1.

Policy Types:

  1. Policy to select requests for URLs that are in the Search Engine category - add responder policy p1 ‘HTTP.REQ.HOSTNAME.APPEND(HTTP.REQ.URL).URL_CATEGORIZE(0,0). CATEGORY().EQ(“Search Engine”)
  2. Policy to select requests for URLs that are in the Adult category group - add responder policy p1 ‘HTTP.REQ.HOSTNAME.APPEND(HTTP.REQ.URL).URL_CATEGORIZE(0,0). CATEGORY_GROUP().EQ(“Adult”)’
  3. Policy to select requests for Search Engine URLs with a reputation score lower than 4 - add responder policy p2 ‘HTTP.REQ.HOSTNAME.APPEND (HTTP.REQ.URL).URL_CATEGORIZE(4,0).HAS_CATEGORY(“Search Engine”)
  4. Policy to select requests for Search Engine and Shopping URLs - add responder policy p3 ‘HTTP.REQ.HOSTNAME.APPEND(HTTP.REQ.URL).URL_CATEGORIZE(0,0).CATEGORY().EQ (“good_categories”)
  5. Policy to select requests for Search Engine URLs with a reputation score equal to or greater than 4 - add responder policy p5 ‘CLIENT.SSL.DETECTED_DOMAIN.URL_CATEGORIZE(4,0). CATEGORY().EQ(“Search Engines”)
  6. Policy to select requests for URLs that are in the Search Engine category and compare them with a URL Set - ‘HTTP.REQ.HOSTNAME.APPEND(HTTP.REQ.URL).URL_CATEGORIZE(0,0). CATEGORY().EQ(“Search Engine”) && HTTP.REQ.HOSTNAME.APPEND(HTTP.REQ.URL).URLSET_MATCHES_ANY(“u1”)’

Responder Policy Types

There are two types of policies used in URL Categorization feature and each of these policy types is explained below:

Policy Type Description
URL Category Categorize web traffic and based on evaluation result blocks, allows or redirects traffic.
URL Reputation Score Determines the reputation score of the website and allows you to control access based on the reputation score threshold level set by the administrator.

Configuring URL Categorization

To configure URL Categorization on a Citrix SWG appliance, do the following:

  1. Enable URL filtering.
  2. Configure a proxy server for Web traffic.
  3. Configure SSL interception for Web traffic in explicit mode.
  4. Configure shared memory to limit cache memory.
  5. Configure URL categorization parameters.
  6. Configure URL categorization by using the Citrix SWG wizard.
  7. Configuring URL categorization parameters by using the SWG wizard.

Step 1: Enabling URL Filtering

To enable URL categorization, enable the URL filtering feature and enable modes for URL categorization.

To enable URL Categorization by using the Citrix SWG: CLI

At the command prompt, type:

enable ns feature URLFiltering

disable ns feature URLFiltering

Step 2: Configuring a Proxy Server for Web traffic in Explicit Mode

The Citrix SWG appliance supports transparent and explicit proxy virtual servers. To configure a proxy virtual server for SSL traffic in explicit mode, do the following:

  1. Add a proxy server.
  2. Bind an SSL policy to the proxy server.

To add a proxy server by using the Citrix SWG CLI

At the command prompt, type:

add cs vserver \<name\> \[-td \<positive\_integer\>\] \<serviceType\>  \[-cltTimeout \<secs\>\]

Example:

add cs vserver starcs PROXY 10.102.107.121 80 -cltTimeout 180

To bind an SSL policy to a proxy virtual server by using the Citrix SWG CLI

bind ssl vserver <vServerName> -policyName <string> [-priority <positive_integer>]

Step 3: Configuring SSL Interception for HTTPS Traffic

To configure SSL interception for HTTPS traffic, do the following:

  1. Bind a CA certificate-key pair to the proxy virtual server.
  2. Configure the default SSL profile with SSL parameters.
  3. Bind a front-end SSL profile to the proxy virtual server and enable SSL interception in the front-end SSL profile.

To bind a CA certificate-key pair to the proxy virtual server by using the Citrix SWG CLI

At the command prompt, type:

bind ssl vserver <vServerName> -certkeyName <certificate-KeyPairName> -CA –skipCAName

To configure the default SSL profile by using the Citrix SWG CLI

At the command prompt, type:

set ssl profile <name> -denySSLReneg <denySSLReneg> -sslInterception (ENABLED | DISABLED) -ssliMaxSessPerServer <positive_integer>

To bind a front-end SSL profile to a proxy virtual server by using the Citrix SWG CLI

At the command prompt, type:

set ssl vserver <vServer name>  -sslProfile ssl_profile_interception

Step 4: Configuring Shared Memory to Limit Cache Memory

To configure shared memory to limit cache memory by using the Citrix SWG CLI

At the command prompt, type:

set cache parameter [-memLimit <megaBytes>]

Where, the memory limit configured for caching is set as 10 MB.

Step 5: Configuring URL Categorization Parameters

To configure the URL categorization parameters by using the Citrix SWG CLI

At the command prompt, type:

set urlfiltering parameter [-HoursBetweenDBUpdates <positive_integer>] [-TimeOfDayToUpdateDB <HH:MM>]

Example:

    Set urlfiltering parameter –urlfilt_hours_betweenDB_updates 20

Step 6: Configuring URL Categorization by Using the Citrix SWG Wizard

To configure URL Categorization by using the Citrix SWG GUI

  1. Log on to the Citrix SWG appliance and navigate to Secured Web Gateway page.
  2. In the details pane, do one of the following: 1. Click Secured Web Gateway Wizard to create a new configuration. 2. Select an existing configuration and click Edit.
  3. In the URL Filtering section, click Edit.
  4. Select the URL Categorization checkbox to enable the feature.
  5. Select a URL Categorization policy and Click Bind.
  6. Click Continue and then Done.

For more information about URL Categorization policy, see How to Create a URL Categorization Policy.

Step 7: Configuring URL Categorization Parameters by Using SWG Wizard

To configure URL Categorization parameters by using the Citrix SWG GUI

  1. Log on to Citrix SWG appliance and navigate to Secured Web Gateway > URL Filtering.
  2. In the URL Filtering page, click Change URL filtering settings link.
  3. In the Configuring URL Filtering Params page, specify the following parameters.
  4. Hours Between DB Updates. URL Filtering hours between database updates. Minimum value: 0 and Maximum value: 720.
  5. Time of Day to Update DB. URL Filtering time of day to update database.
  6. Click OK and Close.

Sample Configuration:

    enable ns feature LB CS SSL IC RESPONDER AppFlow URLFiltering

    enable ns mode FR L3 Edge USNIP PMTUD

    set ssl profile ns_default_ssl_profile_frontend -denySSLReneg NONSECURE -sslInterception ENABLED -ssliMaxSessPerServer 100

    add ssl certKey swg_ca_cert -cert ns_swg_ca.crt -key ns_swg_ca.key

    set cache parameter -memLimit 100

    add cs vserver starcs PROXY 10.102.107.121 80 -cltTimeout 180

    add responder action act1 respondwith "\"HTTP/1.1 200 OK\r\n\r\n\" + http.req.url.url_categorize(0,0).reputation + \"\n\""

    add responder policy p1 "HTTP.REQ.URL.URL_CATEGORIZE(0,0).CATEGORY.eq(\"Shopping/Retail\") || HTTP.REQ.URL.URL_CATEGORIZE(0,0).CATEGORY.eq(\"Search Engines & Portals

    \")" act1

    bind cs vserver starcs_PROXY -policyName p1 -priority 10 -gotoPriorityExpression END -type REQUEST

    add dns nameServer 10.140.50.5

    set ssl parameter -denySSLReneg NONSECURE -defaultProfile ENABLED -sigDigestType RSA-MD5 RSA-SHA1 RSA-SHA224 RSA-SHA256 RSA-SHA384 RSA-SHA512 -ssliErrorCache ENABLED

    -ssliMaxErrorCacheMem 100000000

    add ssl policy pol1 -rule "client.ssl.origin_server_cert.subject.  URL_CATEGORIZE(0,0).CATEGORY.eq(\"Search Engines & Portals\")"" -action INTERCEPT

    add ssl policy pol3 -rule "client.ssl.origin_server_cert.subject.ne(\"citrix\")" -action INTERCEPT

    add ssl policy swg_pol -rule "client.ssl.client_hello.SNI.URL_CATEGORIZE(0,0).CATEGORY.ne(\"Uncategorized\")" -action INTERCEPT

    set urlfiltering parameter -HoursBetweenDBUpdates 3 -TimeOfDayToUpdateDB 03:00

Configuring Audit Log Messaging

Audit logging enables you to review a condition or a situation in any phase of URL Categorization process. When a Citrix ADC appliance receives an incoming URL, if the responder policy has an URL Filtering expression, the audit log feature collects URL Set information in the URL and stores it as log messages for any target allowed by audit logging.

  • Source IP address (the IP address of the client that made the request).

  • Destination IP address (the IP address of the requested server).

  • Requested URL containing the schema, the host and the domain name (http://www.example.com).

  • URL category that the URL filtering framework returns.

  • URL category group that the URL filtering framework returned.

  • URL reputation number that the URL filtering framework returned.
  • Audit log action taken by the policy.

To configure audit logging for URL List feature, you must complete the following tasks:

  1. Enable Audit Log.
  2. Create Audit Log message action.
  3. Set URL List responder policy with Audit Log message action.

For more information, see Audit Logging topic.

Storing Failure Errors Using SYSLOG Messaging

At any stage of the URL Filtering process, if there is a system-level failure, the Citrix ADC appliance uses the audit log mechanism to store logs in the ns.log file. The errors are stored as text messages in SYSLOG format so that, an administrator can view it later in a chronological order of event occurrence. These logs are also sent to an external SYSLOG server for archival. For more information, see article CTX229399.

For example, if a failure occurs when you initialize the URL Filtering SDK, the error message is stored in the following messaging format.

Oct  3 15:43:40 <local0.err> ns URLFiltering[1349]: Error initializing NetStar SDK (SDK error=-1). (status=1).

The Citrix ADC appliance stores the error messages under four different failure categories:

  • Download failure. If an error occurs when you try to download the categorization database.
  • Integration failure. If an error occurs when you integrate an update into the existing categorization database.
  • Initialization failure. If an error occurs when you initialize the URL Categorization feature, set categorization parameters, or end a categorization service.
  • Retrieval failure. If an error occurs when the appliance retrieves the categorization details of the request.