Product Documentation

Deploying SD-WAN in PBR mode (Virtual Inline Mode)

Aug 17, 2017

In virtual inline mode, the router uses policy based routing rules to redirect incoming and outgoing WAN traffic to the appliance, and the appliance forwards the processed packets back to the router. 

The following article describes the step-by-step procedure to configure two SD-WAN (SD-WAN SE) appliances:

  • Data Center Appliance in PBR mode (Virtual Inline Mode)  
  • Branch Appliance in Inline mode 
  • PBR needs to be configured either at the core switch or further upstream at the router. The router must monitor the health of the SD-WAN appliance so that the appliance can be bypassed if it fails.   
  • Virtual Inline Mode places the SD-WAN appliance physically out of path (one-arm deployment) i.e. only a single Ethernet interface to be used (Example: Interface 1/1) with bypass mode set to fail-to-block (FTB). 

NetScaler SD-WAN appliance needs to be configured to pass traffic to the proper gateway.  Traffic intended for the Virtual Path is directed towards the SD-WAN appliance and then encapsulated and directed to the appropriate WAN link.

Gathering Information for Configuration

  • Accurate network diagram (example diagram show below) of your local and remote site(s) including:   

         -  Local and Remote WAN links and their bandwidths in both directions, their subnets, Virtual IP Addresses and Gateways from each link, Routes, and VLANs.

  • Deployment Table (example diagram shown below) 

 

Data Center Topology – PBR mode (Virtual Inline Mode)

localized image

Branch Topology – Inline Mode

localized image
Site Name DataCenter Site Branch Site

Appliance Name

A_DC1 

A_BR1 

Management IP

172.30.2.10/24 

172.30.2.20/24 

Security Key

If any

If any

Model/Edition

4000 

2000

Mode

PBR mode (Virtual Inline Mode) 

Inline

Topology

2 x WAN Path 

2 x WAN Path 

VIP Address

192.168.1.10/24 – MPLS

192.168.1.11/24 – Internet*Public IP w.x.y.z 

10.17.0.9/24 - MPLS

10.18.0.9/24 – Internet *Public IP a.b.c.d 

Gateway MPLS

10.20.0.1 

10.17.0.1 

Gateway Internet

10.19.0.1

10.18.0.1 

Link Speed

MPLS – 100 Mbps

Internet – 20 Mbps 

MPLS – 10 Mbps

Internet – 2 Mbps 

Route

Need to add a route on the SD-WAN SE Appliance on how to reach the LAN Subnets (10.10.11.0/24, 10.10.12.0/24, 10.10.13.0/24, etc) through any of the physical interfaces:

Gi0/1 - 192.168.1.1

Configuration > Virtual WAN > Configuration Editor > SJC_DC > Routes.

In this example interface 192.168.1.1 was used:

  • n/w address: 10.10.13.0/24, 10.10.12.0/24, 10.10.11.0/24
  • service type: local
  • gateway IP address: 192.168.1.1

 

 

No additional routes were added 

VLANs

None (default 0)

None (default 0)

Steps to configure a site in Virtual Inline Mode:

  • Enable the MCN functionality.
  • Create a New site.
  • Create an Interface Group and Virtual Interfaces.
  • Assign Virtual IP Address to Virtual Interfaces.
  • Create WAN Links and assign IP address.
  • Add Routes.
  • Troubleshooting.
  • Policy Based Routing configuration on the PBR Router.

Configuration Pre-requisites

  • Enable SD-WAN appliance as a Master Control Node.
  • Configuration is done only on the Master Control Node (MCN) of the SD-WAN appliance. 

To enable an appliance as a Master Control Node:

1. In the NetScaler SD-WAN web management interface, navigate to Configuration > Appliance Settings > Administrator Interface > Miscellaneous tab > Switch Console.

Note

If  “Switch to Client Console” is displayed, then the appliance is already in MCN mode. There should only be one active MCN in a SD-WAN network.

2. Enable Virtual WAN Service.

localized image

     3. Start Configuration by navigating to Configuration > Virtual WAN Configuration Editor. Click New to begin configuration. 

localized image

This operation will create an Untitled_1 initial configuration file which can be renamed [optional] later using the Save As button. 

Following are the high-level configuration steps to configure Datacenter site in PBR deployment mode:

1.  Create a new DC site.

2.  Configure Interface Groups based on connected Ethernet interfaces.

3.  Configure Virtual IP address for each virtual interface.

4.  Populate WAN links based on physical rate and not burst speeds using Internet and MPLS Links.  

5.  Populate Routes if there are additional subnets in the LAN infrastructure.

Datacenter Site PBR Mode Configuration

To create a new DC site

  1. Navigate to Configuration Editor - > Sites, and click the "+" Add button.
  2. Populate the fields as shown below.
  3. Keep default settings unless instructed to change.
localized image
localized image

To configure interface groups based on connected Ethernet interfaces

  1. In the Configuration Editor, navigate to Sites → [Site Name] → Interface Groups. Click “+”to add interfaces intended to be used. In PBR mode, configuration on only a single Ethernet interface is used i.e. interface connecting the upstream router providing PBR policy implications (Example- Interface 1/1).
  2. Bypass mode is set to fail-to-block since only one Ethernet/physical interface is used per virtual interface.  There are also no Bridge Pairs. 
  3. In this example, expand Virtual Interfaces + option and configure the Virtual Interfaces.
localized image

To create Virtual IP (VIP) address for each virtual interface

  1. Create a Virtual IP Address on the appropriate subnet for each WAN Link.  VIPs are used for communication between two SD-WAN appliances in the Virtual WAN environment.
localized image
localized image

To populate WAN links based on physical rate and not on burst speeds using Internet and MPLS link:

  1. Navigate to WAN Links, click the “+” button to add a WAN Link for the Internet link.
  2. Populate Internet link details, including the supplied Public IP address as shown below. Note that AutoDetect Public IP cannot be selected for SD-WAN appliance configured as MCN.
  3. Navigate to Access Interfaces, click the “+” button to add interface details specific for the Internet link.
  4. Populate Access Interface for IP and gateway addresses as shown below. The Proxy ARP is not checked for less than two Ethernet interfaces. 
localized image

To create MPLS Link

  1.  Navigate to WAN Links, click the “+” button to add a WAN Link for the MPLS link.
  2.  Populate MPLS link details as shown below.
  3.  Navigate to Access Interfaces, click the “+” button to add interface detail specific for the MPLS link.
  4.  Populate Access Interface for MPLS Virtual IP and gateway addresses as shown below.
localized image

Note

The Proxy ARP is not checked for less than two Ethernet interfaces. 

To populate Routes 

 

On the Data center site,  add a route on the SD-WAN SEE appliance to reach the LAN Subnets (10.10.11.0/24, 10.10.12.0/24, 10.10.13.0/24, etc) through any of the physical interfaces:

0/1/0.1 – 192.168.1.1 on VLAN 10

0/1/0.2 – 192.168.2.1 on VLAN 20 

localized image
localized image
localized image

Branch Site Inline Deployment Configuration

Following are the high-level configuration steps to configure Branch site for Inline deployment:

1.  Create a new Branch site.  

2.  Populate Interface Groups based on connected Ethernet interfaces. 

3.  Create Virtual IP address for each virtual interface.

4.  Populate WAN links based on physical rate and not burst speeds using Internet and MPLS Links.  

  • Virtual Interface “INTERNET” configured on Bridge pair 1/3 and 1/4 
  • Virtual Interface “MPLS” configured con Bridge Pair 1/1 and 1/2 

5.  Populate Routes if there are additional subnets in the LAN infrastructure.

localized image
localized image

To populate interface groups based on connected Ethernet interfaces

  1. In the Configuration Editor, navigate to Sites → [Client Site Name] → Interface Groups. Click “+”to add interfaces intended to be used.  For Inline mode configuration, four Ethernet interface are used; interface pair 1/3, 1/4 and interface pair 1/1 and 1/2. 
  2. Bypass mode is set to fail-to-wire since two Ethernet/physical interfaces are used per virtual interface. There are two bridge Pairs. 
  3. Populate WAN links based on physical rate and not burst speeds using Internet and MPLS Links.  
  • Virtual Interface “INTERNET” configured on Bridge pair 1/3 and 1/4 
  • Virtual Interface “MPLS” configured con Bridge Pair 1/1 and 1/2.

       4. Refer to the sample “Remote Site Inline Mode” topology above and populate the Interface Groups fields as shown below.

localized image

To create Virtual IP (VIP) address for each virtual interface

  1. Create a Virtual IP address on the appropriate subnet for each WAN Link.  VIPs are used for communication between two SD-WAN appliances in the Virtual WAN environment.
localized image

To populate WAN links based on physical rate and not on burst speeds using Internet link

  1. Navigate to WAN Links, click the “+” button to add a WAN Link for the Internet link.
  2. Populate Internet link details, including the AutoDetect Public IP address as shown below. 
  3. Navigate to Access Interfaces, click the “+” button to add interface details specific for the Internet link.
  4. Populate Access Interface for Virtual IP address and gateway as shown below.
localized image
localized image

To create MPLS Link

  1.  Navigate to WAN Links, click the “+” button to add a WAN Link for the MPLS link.
  2.  Populate MPLS link details as shown below.
  3.  Navigate to Access Interfaces, click the “+” button to add interface details specific for the MPLS link.
  4.  Populate Access Interface for Virtual IP address and gateway as shown below.
localized image
localized image

To populate Routes 

Routes are auto-created based on above configuration. In case there are additional subnets specific to this remote branch office, then specific routes need to be added identifying which gateway to direct traffic to in order to reach those backend subnets.  

localized image

Resolving Audit Errors

After completing configuration for DC and Branch sites, you will be alerted to resolve audit error on both DC and BR sites.  In this example,  we will resolve the Audit Error related to Private Intranet WAN Link [SJC_DC-MPLS].

Note: By default the system will generate paths for WAN Links defined as access type Public Internet (highlighted).

 

localized image
localized image
localized image
localized image

By default, the system will generate paths for WAN Links defined as access type Public Internet. You would be required to use the auto-path group function or enable paths manually for WAN Links with an access type of Private Internet. Paths for MPLS links can be enabled by clicking on the Add operator (in the green rectangle).  

localized image

Create an Autopath Group

  1.  Click on the [+] sign next to Autopath Groups. 
  2.  Configure the Autopath Group created as per requirement and click Apply. 
localized image
localized image

          3. Rename the Autopath Group [Optional].

localized image

          4.  Map the Autopath Group to the Virtual Paths of Intranet WAN links at respective sites.

No two Autopath Groups can be marked as default. If marked would lead to an Audit Error. 

After mapping the Autopath Group to the Virtual Paths of Intranet WAN, the paths should be automatically populated (highlighted).

 

localized image

Manually add WAN links with access type Private Intranet

  1. Select the Virtual Paths under WAN Links for respective sites and no Autopath Group would be mapped. 
  2. Click the [+] sign next to Paths to add Virtual Paths manually.  
localized image
localized image

                   3. Select the Virtual Paths WAN Links for each site. 

localized image

After manually adding the virtual paths for WAN links with access type Private Intranet, it gets populated under Paths (highlighted). 

localized image

After completing all the above steps, proceed to Preparing the SD-WAN Appliance Packages on the MCN topic.

 

Policy Based Routing configuration on the PBR Router

Interface connected to the LAN

  • Router# configure terminal
  • Router(config)# interface FastEthernet0/1
  • Router(config-if)# description ToLAN
  • Router(config-if)# ip address 10.10.11.1 255.255.255.0
  • Router(config-if)# duplex auto
  • Router(config-if)# speed auto

Interface connect to the MPLS WAN Link

  • Router# configure terminal
  • Router(config)# interface GigabitEthernet0/0
  • Router(config-if)# description To-MPLS-WAN
  • Router(config-if)# ip address 10.20.0.2 255.255.255.0
  • Router(config-if)# duplex auto
  • Router(config-if)# speed auto

Interface connected to the INET WAN Link

  • Router# configure terminal
  • Router(config)# interface GigabitEthernet0/2/0
  • Router(config-if)# description To-INET-WAN
  • Router(config-if)# ip address 10.19.0.2 255.255.255.0
  • Router(config-if)# duplex auto
  • Router(config-if)# speed auto

Note: Interface GigabitEthernet0/1 on the PBR router is connected to the SD-WAN port 1/1, it is in 1-arm mode and this one port will serve traffic for MPLS and INET links.

  • Router# configure terminal
  • Router(config)# interface GigabitEthernet0/1
  • Router(config-if)# description To-SDWAN-link
  • Router(config-if)# ip address 192.168.1.1 255.255.255.0

Static Route Configuration (Route to the client/remote subnets):

  • MPLS 10.17.0.0/24 via next hop WAN router MPLS 10.20.0.1
  • INET 10.18.0.0/24 via next hop WAN router/FW INET 10.19.0.1
 
  • Router# configure terminal
  • Router(config)# ip route 10.17.0.0 255.255.255.0 10.20.0.1
  • Router(config)# ip route 10.18.0.0 255.255.255.0 10.19.0.1

Route Map Definition

Access Control List Configuration:

Configure ACL’s to define the traffic to be sent to and from the SD-WAN appliance.

1- From LAN to SD-WAN Appliance

As per topology, the LAN subnets are 10.10.11.0/24, 10.10.12.0/24, 10.10.13.0/24, etc. To send traffic from LAN to the SD-WAN, configure a unidirectional ACL (from LAN to any).

  • Router# configure terminal
  • Router(config)# ip access-list extended server_side
  • Router(config)# permit ip 10.10.0.0 0.0.255.255 any

 

2- From SD-WAN Appliance to physical WAN Links

  • Router# configure terminal
  • Router(config)# ip access-list extended MPLS_Link
  • Router(config)# permit ip 192.168.1.10 0.0.0.0 any
  • Router# configure terminal
  • Router(config)# ip access-list extended INET_Link
  • Router(config)# permit ip 192.168.1.11 0.0.0.0 any

Route Map Configuration:

Define the route-map matching the ACL’s.

Route map for LAN traffic:

Next hop will be any of SD-WAN Virtual IP’s (VIP).

MPLS VIP 192.168.1.10

INET VIP 192.168.1.11

In this case, we chose MPLS VIP 192.168.1.10 as next hop and also added a health check to make sure if the SD-WAN fails, traffic is not routed to it.

  • Router# configure terminal
  • Router(config)# route-map server_side_VW_PBR permit 10
  • Router(config-route-map)# match ip address server_side
  • Router(config-route-map)# set ip next-hop verify-availability 192.168.1.10 10 track 123

Note: The above command configures the route map to verify the reachability of the tracked object. The tracking process provides the ability to track individual objects, such as ICMP ping reachability, routing adjacency, an application running on a remote device, a route in the Routing Information Base (RIB) or to track the state of an interface line protocol.

Route map for WAN traffic:

Next hop will be MPLS Router and Firewall for respective WAN links.

  • Router# configure terminal
  • Router(config)# route-map WAN_VW_PBR permit 20
  • Router(config-route-map)# match ip address MPLS_Link
  • Router(config-route-map)# set ip next-hop verify-availability 10.20.0.1 20 track 124
  • Router# configure terminal
  • Router(config)# route-map WAN_VW_PBR permit 30
  • Router(config-route-map)# match ip address INET_Link
  • Router(config-route-map)# set ip next-hop verify-availability 10.19.0.1 30 track 125

Apply the Route Map to the interface:

Router# configure terminal

  • Router(config)# interface FastEthernet0/1
  • Router(config-if)# ip policy route-map server_side_VW_PBR
  • Router(config-if)# duplex auto
  • Router(config-if)# speed auto
  • Router# configure terminal
  • Router(config)# interface GigabitEthernet0/1
  • Router(config-if)# ip policy route-map WAN_VW_PBR
  • Router(config-if)# duplex auto
  • Router(config-if)# speed auto

MPLS Router Configuration (Gateway 10.20.0.1)

  • Add route on MPLS router to reach MPLS VWAN VIP on the Data Center.
  • MPLS VIP subnet 192.168.1.0/24 via next hop PBR router MPLS link 10.20.0.2
  • Router# configure terminal
  • Router(config)# ip route 192.168.1.0 255.255.255.0 10.20.0.2

Firewall Configuration (Gateway 10.19.0.1)

Add route on Firewall to reach INET VWAN VIP on the Data Center.

INET VIP subnet 192.168.1.0/24 via next hop PBR router INET link 10.19.0.2

  • Router# configure terminal
  • Router(config)# ip route 192.168.1.0 255.255.255.0 10.19.0.2