Fragmentation in Controller Architectures

*Updated Dec. 2010 - Added CAPWAP information.


Why IP Fragmentation is a Concern on Wireless Networks

A big concern with controller-based architectures is reduced performance due to IP fragmentation due to tunnel overhead between the AP and controller. Typically, most lightweight APs tunnel all client traffic back to the controller through a layer 3 tunnel. This tunnel adds overhead, as the AP has to encapsulate the original client frame within a new IP packet.

Almost all vendors now offer options to locally switch (or bridge) the client frames into the wired network from the AP, without tunneling the frame back to the controller. Lookup Cisco H-REAP, Aruba RAP, Ruckus does this by default on all APs, Aerohive doesn’t even have a physical controller, you get my point.

Tunnel overhead reduces the amount of data available in the packet payload that can carry client data. If the client sends frames that are above a certain threshold, then adding the additional tunnel headers increases the packet size beyond the Ethernet MTU (1500 bytes unless using jumbo frames), resulting in IP fragmentation. Ultimately, this results in increased network latency and jitter, and decreased throughput.

Bad for data, worse for voice!

LWAPP Overhead

In Layer 3 LWAPP mode, additional IP, UDP, and LWAPP headers are added to encapsulate the original client 802.11 frame between the access point and controller.

LWAPP overhead for tunneled client frames is 52 bytes due to these additional frame headers which encapsulate the original frame taken out of the air:

§  18 – Ethernet header and checksum

§  20 – IP header (without IP options)

§  8 – UDP header

§  6 – LWAPP header (CAPWAP header is 8 bytes)

The contents of the LWAPP frame also contain the original 802.11 client data frame, with a few notable portions that are stripped out. Let’s add these into the tunneled frame contents:

§  24 – Original 802.11 MAC header, without QoS, encryption, or checksum fields

§  8 – 802.2 LLC and SNAP headers (3 LLC, 5 SNAP)

§  20 – Original IP header

§  Variable – Transport layer header (20 TCP, 8 UDP, 8 ICMP, etc.) and data payload

Notice that I said the whole 802.11 frame, not just the client’s data payload, get’s tunneled. The 802.11 MAC header is tunneled in order for the controller to be able to de-multiplex the original client frame, determine source and destination MAC addresses, forward the frame to appropriate destination host, and perform the non-real-time 802.11 MAC functions in the Split-MAC architecture.

Split-MAC Responsibilities

In Cisco’s LWAPP architecture, split-MAC is the term coined to differentiate what MAC functions get handled by the controller versus the access point.

Since the controller needs to handle some MAC functions, it needs the client’s 802.11 MAC header tunneled back to it to perform its responsibilities. Since the AP handles real-time MAC functions, such as QoS and encryption/decryption, those fields are stripped out of the header before tunneling the packet to the controller.

Controller Responsibilities:

§  Security management (policy enforcement, rogue detection, etc.)

§  Configuration and firmware management

§  Northbound management interfaces

§  Non real-time 802.11 MAC functions

o   Association, dis-association, re-association

o   802.11e/WMM resource reservation (CAC, TSPEC, etc.)

o   802.1x/EAP authentication

o   Key management

§  802.11 distribution services

§  Wired and wireless integration services

Lightweight Access Point Responsibilities:

§  Real-time 802.11 MAC Functions

o   Beacon generation

o   Probe responses

o   Informs WLC of probe requests via LWAPP control channel

o   Power management and packet buffering

o   802.11e/WMM scheduling and queuing

o   MAC layer data encryption and decryption

o   802.11 control messages (ACK, RTS/CTS)

§  Data encapsulation and de-capsulation to LWAPP

§  Fragmentation and re-assembly

§  RF spectral analysis

§  WLAN IDS signature analysis

LWAPP Fragmentation

Let’s take a look at how Cisco’s LWAPP protocol handles IP fragmentation in code versions prior to 5.2, when the switch to CAPWAP is made.

Layer 3 LWAPP is able to perform full fragmentation and reassembly of tunnel packets using standard IP fragmentation, thereby allowing client traffic to make use of a full 1500 byte MTU and not to have to adjust for any tunnel overhead. This simplifies administration of wireless clients, since you don’t have to touch them to modify MTU settings for the wireless adapter. Clients simply assume their IP packets have an MTU of 1500 bytes and move happily along. It’s not until the frame reaches the AP that fragmentation needs to occur because of the tunnel overhead.


* Note - CAPWAP includes fragmentation and re-assembly support and does not rely on the standard IP fragmentation. Therefore, in fragmented CAPWAP tunnel packets, fragmentation will be performed on the CAPWAP data, with each fragment duplicating the outer IP, UDP, and CAPWAP headers.

Adding up the tunnel overhead, we find that the threshold for the original client’s IP packet that causes fragmentation of the LWAPP IP packet on an Ethernet segment with a 1500 byte MTU is at a size of 1434 bytes. We get this by adding the tunnel headers (IP, UDP, LWAPP) and the tunneled 802.11 frame headers (802.11, LLC, SNAP), and subtracting those amounts from the 1500 byte MTU. This leaves only the original client IP packet which must be under a 1434 byte threshold to prevent fragmentation. The outer Ethernet header is not included because the MTU size does not consider the layer 2 MAC header, which is appended later.


(Threshold is 1432 bytes for CAPWAP encapsulated frames)

Here is a packet capture to illustrate this fragmentation. Remember, the outer IP header for the tunnel will handle the fragmentation, so the UDP and LWAPP headers will only be found in the first fragment.

Testing LWAPP Fragmentation

An easy way to test LWAPP fragmentation is to connect to the wireless network and ping the subnet gateway above and below the maximum ICMP payload size of 1406 bytes assuming a 1500 byte MTU. How we get to 1406 bytes for this test:

1500 byte MTU

     Subtract 34 bytes for LWAPP encapsulation (subtract 36 bytes for CAPWAP)

     Subtract 24 bytes for the tunneled 802.11 header

     Subtract 8 bytes for the tunneled 802.2 LLC/SNAP header

     Subtract 20 bytes for the tunneled IP header (without options)

     Subtract 8 bytes for the ICMP header

= 1406 bytes remaining for ICMP payload (use 1404 bytes for CAPWAP)

Setup a tap or port mirror on the switch port attached to the access point. Connect a workstation running a protocol analyzer to the mirrored port or tap, filter on port UDP 12,222 (LWAPP Data) or UDP 5247 (CAPWAP Data), and start capturing packets. 

From the wireless client, try out these commands and watch the results in the protocol analyzer.

Test 1

ping –f –l 1406 ip-address

Result will be successful, no fragmentation.

Test 2

ping –f –l 1407 ip-address

Result will be successful, but LWAPP tunnel packets will be fragmented. LWAPP tunnel packets do not carry over the original packet’s don’t fragment flag.

Test 3

ping –f –l 1473 ip-address

Result will fail, as original IP packet needs to be fragmented by the client but the don’t fragment flag is set.

Note

Many VPN client software packages will change the default interface MTU to a lower value. Verify the interface MTU to adjust test payload size properly. For example the Cisco Systems VPN client software for Windows XP modifies the MTU from 1500 bytes to 1300 bytes.

To view the current interface MTU:

§  Windows XP regedit
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\interface-id\MTU

(if the key does not exist, it uses the default MTU of 1500 bytes)

§  Windows Vista/7 view interface MTU command is “netsh interfaces ipv4 show subinterfaces”

Preventing Fragmentation

Unfortunately, there is no method to prevent LWAPP fragmentation in the controller or access points until code version 6.0 with the CAPWAP protocol.

To avoid fragmentation of client packets enable TCP Maximum Segment Size (MSS) control on the controller and lightweight access points in code versions 6.0 and later.

This feature allows connected CAPWAP access points to re-write client TCP SYN packets with the configured maximum segment size to optimize large data transfers and prevent fragmentation when CAPWAP encapsulated. The valid range is from 536 to 1363 bytes, and it is recommended to use a 1363 byte value.

Configure it from the CLI using this command:

config ap tcp-adjust-mss { enable | disable } { all | ap-name } tcp_mss_value

- Andrew