Microsoft Lync QoS

This week I'm back to my favorite topic, quality of service. Engineers across multiple teams at my current employer have been working on a project to enable wireless VoIP using softphones running Microsoft Lync on both Windows 7 and XP. This project has provided an opportunity to review how our organization handles multi-function wireless devices and network performance, with particular focus on quality of service mechanisms.

We've found some interesting things...

Wireless QoS - Not a "One-Size Fits All" Policy (Anymore)
Since Wi-Fi is a shared medium with control distributed to all active nodes in the system, proper network arbitration and performance is heavily dependent on client behavior. When dealing with QoS using 802.11e/WMM, this means accurate application traffic marking and queuing by client devices themselves. 802.11e prescribes 8 user priorities and 4 priority queues to provide a base level of differentiated services for traffic (you can read more about this in my Wireless QoS 5-part series).

Many wireless LAN vendors have only provided basic support for wireless QoS by reading the QoS values within frames and queuing traffic for downstream transmission to clients. Some vendors have begun going beyond basics to provide customers with a feature called "Airtime Fairness", which are proprietary extensions to ensure more in-depth control of clients by the infrastructure. I particularly like how Devin Akin at Aerohive highlights that this feature is really manipulation of the environment to suit a specific policy, whether it be equitable between clients or not. However, these are still mainly downlink traffic mechanisms, and the uplink flow is still controlled by the client. (There are some exceptions to this that involve infrastructure vendors monkeying with client TCP windows, acknowledgments, etc., but let's not get into that now.)

Device Convergence
A Swiss-Army Knife of Capabilities

Traditionally, vendors have implemented QoS on a per-SSID basis. This worked decent once-upon-a-time when all devices had only a single purpose in life. It was fairly easy for administrators to segment data-only devices such as laptops from voice-capable devices such as IP phones. No problem. Everything in the SSID gets slapped with a QoS template and we're done.

But what happens now when we have veritable Swiss-Army devices that perform multiple functions. What SSID do we put those in? How do we differentiate network performance and QoS based on application flows rather than device or SSID?

The answer is that wireless clients and the network must both have more intelligence to handle dynamic QoS requirements. Device convergence and the use of multi-purpose devices eliminates the ability to effectively use static 1:1 QoS policies tied to wireless SSIDs. We need something better!

Microsoft Lync QoS - A Case Study
Microsoft Lync is a software platform for unified communications providing data, voice, and video collaboration on Windows workstations. Many organizations are exploring device convergence to expand capabilities available to all employees while controlling capital and recurring costs.

As part of our lab verification of Lync prior to production deployment, we tested Wi-Fi quality of service integration. Microsoft Windows Vista, Windows 7, and Server 2008 platforms support policy-based QoS, while older Windows XP systems only support two service levels with the QoS Packet Scheduler. I'll focus on Windows 7 workstations with the more robust policy-based QoS capabilities. We setup an Active Directory GPO to classify and mark all voice traffic coming from Lync with a DSCP value of 46 (expedited forwarding), which is a general best-practice on IP-based networks (see RFC 3246 section 2.7) and conforms with Cisco's QoS Baseline recommendations.

The resulting traffic analysis verified that Lync correctly marked DSCP in the packets, but we also noticed that the layer 2 Class of Service (CoS) marking for 802.11e/WMM is set to 5. We expected to see 6, since the 802.11-2007 standard clearly states in table 9-1 that user priorities 6 and 7 are reserved for voice, while 4 and 5 are reserved for video (note that the Wikipedia entry on 802.11e contains incorrect data). This is distinctly in conflict with the 802.1p CoS values for wired LANs (using the revised 802.1Q-2005 values) which places voice traffic in priority 5.

Microsoft Lync marks layer 2 CoS values based on mapping 
IP Precedence values (3 most-significant bits in IP ToS header field)

Moreover, the mappings between layer 2 and layer 3 markings are distinctly different between vendors. Microsoft maps layer 2 CoS values, whether 802.1p or 802.11e, to the same set of layer 3 values based on IP Precedence (not DSCP) and does not differentiate between wired and wireless networks. Mappings in the opposite direction (DSCP to layer 2) also rely on only the IP Precedence values and not on the full DSCP codepoint. Therefore, a DSCP value of 46 is translated as an IP Precedence of 5 and a layer 2 priority of 5. Cisco, on the other hand, maps values differently for wired and wireless networks (see Table 2-7) to accommodate the variations between standards.

Differences in layer 2 Class of Service (CoS) values between IEEE standards
and layer 2 to layer 3 mapping implementations by vendors create complexity

The root of the issue is two-fold. First, the 802.1p and 802.11e QoS values are clearly in conflict. This makes accurate QoS implementation difficult to achieve because variations need to be dealt with correctly by each and every solution implementation. This introduces plenty of room for error. Second, Microsoft's QoS implementation in Windows maps DSCP to layer 2 CoS based on legacy IP Precedence values. It does not differentiate between wired and wireless network connections and adjust markings appropriately.

Network-Wide Ramifications
The standard practice of marking voice with DSCP 46 will result in the improper classification, marking and queuing of the traffic throughout the network.

First, wireless client transmisions (upstream) by the Windows workstations will get placed into the video queue and will not receive the appropriate contention window values. This will reduce the statistical advantage that voice frames receive for transmission over the air and could impact voice latency and jitter, especially in video rich networks. As video adoption in the enterprise increases, especially with increasing mobile device usage, this could have serious effects on voice quality.

Second, many wireless network infrastructure vendors to do not provide the ability to inspect and re-classify traffic and are forced to trust the client markings. For example, the Cisco Unified Wireless Network simply enforces a maximum QoS value for each SSID that clients cannot exceed. If the SSID is configured for Platinum QoS, then no maximum can be exceeded and traffic will be translated based on the client marking. The CAPWAP tunnel will map the client's layer 2 value of 5 (video) to a DSCP value of 34 for the outer tunnel IP header. Once the traffic is de-encapsulated by the controller, it applies an 802.1p value of 4 based on the CAPWAP packet DSCP value 34 mapped back to a layer 2 wired value, while leaving the client packet's original DSCP value of 46 untouched. Given the best practice of configuring switches to trust DSCP from CAPWAP APs and trust CoS from controllers, this traffic will be mishandled by intermediate switches and routers as well. In fact, the default Cisco switch behavior (when QoS is enabled) is for DSCP transparency to be disabled, and will result in the trusted layer 2 value coming from the WLC overriding the original client DSCP marking and being re-written to 32 based on the default switch CoS-to-DSCP mapping.

Third, advanced wireless voice control features will be ineffective and broken since the voice traffic is not properly identified within the voice queue. This includes TSPEC, call admission control (CAC), traffic bandwidth reservation, expedited bandwidth requests to facilitate emergency 911 calls, and voice stream metrics collection and reporting. Many of these features are only available to traffic streams within the voice queue. Additionally, off-channel scanning used by RRM and rogue detection are configured to defer if traffic in user priorities 4, 5, or 6 have been received in the last 100ms. If this policy has been changed to only defer for priority 6 traffic it could negatively impact Lync traffic in priority 5 by increasing network re-transmissions and network latency.

Finally, WAN bandwidth across the network could be negatively impacted. Typically, network administrators will design WAN circuits to reserve bandwidth for a specific amount of voice calls (based on Erlang calculations) as well as enforce a per-call bandwidth limitation based concurrent call volume. Since the voice calls will not be placed into the voice queue, no traffic admission or policing can occur. Should the environment grow to a point where concurrent voice calls exceeds the design, then WAN bandwidth may not be sufficient to handle the additional calls but will not be able to restrict call admission, resulting in poor voice performance.

Let's also not forget the risk of human error somewhere down the road. Assuming documentation is in proper order, policy accommodations have been made throughout the network, and engineer transitions include adequate knowledge transfer, a clear risk still exists that future changes will inadvertently affect voice traffic since it isn't supposed to be within the video queue.

Integrating Lync Into a Broader Network Architecture
Our networks aren't created in a vacuum; it's a shared resource and engineers must design networks to handle varying capabilities and demands of all connected endpoints and traffic flows. So, what options are available to mitigate this issue and effectively handle Microsoft Lync voice traffic on a converged wireless network?

We could ignore the problem and apply the standard voice QoS value of DSCP 46 (EF) in our Windows policy. This will result in accurate marking and traffic handling for wired clients, but suffers the ramifications previously described for wireless clients and broader network resource impacts. Hardly an optimal solution.

Instead, consider applying a non-standard DSCP marking. Using this approach we use policy-based QoS on Windows platforms to mark voice traffic as DSCP 48 (CS6), which allows it to be mapped to the correct wireless layer 2 CoS value of 6 and be queued correctly for transmission by the client.

The problem is that this DSCP value is reserved for Internetwork Control traffic by most networking vendors (including Cisco). Network administrators will need to ensure that wired and wireless integration is configured correctly, otherwise incorrect classification could propogate throughout the network for these traffic flows. For Cisco wireless networks this means that switch ports should be configured to trust DSCP from APs and CoS from controllers. This trust coupled with the Cisco wireless mappings between CoS and DSCP ensure that the correct DSCP value of 46 (EF) will be used both in the CAPWAP tunnel and re-written by the switch (based on WLC CoS trust) on the client packet once forwarded upstream out of the controller.

If using another wireless vendor, engineers should review the QoS capabilities of the solution and integration options to ensure accurate handling of this traffic. For example, many vendors support robust inspection and classification of traffic flows to match configured QoS policy directly in the access point. In these instances, configure APs to identify voice traffic and apply QoS based on defined policy.

This configuration will also cause incorrect markings when the workstations are connected to a wired network, which is still a likely occurrence for laptops in most environments. Network administrators should ensure that switches are configured to strip the client QoS markings on switch ports throughout the network, which is standard practice. This way the switch will ignore the client markings and re-apply QoS policy using traffic inspection techniques. Ethernet switching largely eliminates medium contention concerns on wired networks, so having an incorrect policy on the first-hop upstream is not a large concern.

I prefer to implement this solution. With proper end-to-end network QoS implemented, accurate traffic handling can be accomplished on both wired and wireless networks.

Revolution or Evolution? - Andrew's Take
Clearly something went wrong with standards-based QoS development. Divergent layer 2 QoS definitions exist from the IEEE standards. Furthermore, no standard mappings exist between layer 2 and layer 3 QoS values, as evidenced by differences in vendor implementations.

As mobile device convergence proliferates and wired and wireless networks continue to become more closely integrated, consistent end-to-end QoS policy definition and traffic handling will be critical in supporting increasing network demands.

Microsoft Lync highlights these challenges. Increasingly, organizations are looking for converged solutions to provide improved business capabilities and service. And many client platforms won't have as robust control or policy definition options as Windows. How will your organization handle voice, video and data over iOS, Android, Blackberry, Windows Phone 7, and other platforms?

Cheers,
Andrew


Additional Links for Microsoft Lync and Windows QoS:

• Microsoft TechNet Quality of Service Content Section
• Microsoft Lync Quality of Service (QoS) Overview
• Microsoft Policy-Based Quality of Service
• Overview of Policy-Based QoS
• Creating a Client Quality of Service (QoS) Policy
• QoS Packet Scheduler (Windows Server 2003 and Windows XP)
• Enabling Quality of Service on Windows Server 2003 and Windows XP
• MSDN Blog - Introduction to 802.1p
• MSDN Blog - WiFi QoS Support in Windows Vista: WMM
• MSDN Blog - WiFi QoS Support in Windows Vista: WMM part 2 (contains 802.1p to DSCP mappings)
• MSDN Blog - Using Policy-Based QoS to Enable and Manage WMM in Enterprise Wireless Deployments
• MSDN Blog - Content Tagged with QoS
• Integrated Research whitepaper titled "QoS: The Missing Link" on the Webtorials website (free registration required)

[QoS] It's Tricky, Tricky, Tricky, Tricky...

You may have noticed somewhat of a recurring theme across several of my posts - Quality of Service. Since wireless networks are inherently a shared medium, and with Wi-Fi in particular using distributed contention protocols (DCF, EDCA), it stands to reason that implementing QoS controls and having some form of differentiated access to the network is just a bit more critical than on a switched LAN.

Most of the time, Wi-Fi engineers such as ourselves focus on over-the-air QoS since that is typically more critical to performance than wired QoS. However, a recent support incident highlighted the need for careful wired QoS policy definition, especially when supporting an LWAPP/CAPWAP wireless environment.

Shiny New Equipment
A recent project at our organization involved the deployment of several hundred new Cisco 3502 CleanAir access points which run on the Cisco Unified Wireless Network using the CAPWAP protocol. (For an overview of the CAPWAP protocol, see my previous blog posts here, here, here, here, and here.)

This project involved replacing existing 802.11a/b/g "legacy" access points with new 802.11n access points, as well as installation of a few hundred net-new additional APs. The replacement APs were to be installed and patched into the existing AP switch ports, while the new APs were to be patched into open switch ports in the existing data VLAN which provides DHCP services. This would allow the new APs to be deployed with zero-touch configuration, simply taken out of the box and installed by the contractor, minimizing operational expense. After the net-new APs were installed and registered to the controller, an administrator would then move them to the management VLAN and apply the standard port configuration settings for APs in our environment.

Help Desk, "We Have a Problem"
However, almost immediately after the new APs began to be installed, support tickets started rolling in. Users were reporting horribly slow wireless network performance, severe enough to the point of making the network unusable.

A quick trip to the affected building (only 5 min. away) confirmed the issue. A simple ping from a wireless client to the default gateway would drop numerous packets, sometimes as bad as 10% packet loss. And that was when the client was otherwise idle without other applications running. The issue would get even worse when attempting to load more traffic over the connection, such as pulling down a file over an SMB share or browsing a webpage with video content, spiking upwards of 25-30% packet loss. Clearly something was going on.

Sample pings from the distribution switch (housing the WiSM controller) to the wireless client showed the same symptoms in the reverse direction as well:

CAT6K#ping        
Protocol [ip]:
Target IP address: 172.16.10.20
Repeat count [5]: 100
Datagram size [100]: 1400
Timeout in seconds [2]:
Extended commands [n]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 100, 1400-byte ICMP Echos to 172.16.10.20, timeout is 2 seconds:
!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!!!!!.
!!!!!!.!!!!!!.!!!!!!.!!!!!!.!!
Success rate is 86 percent (86/100), round-trip min/avg/max = 1/7/84 ms
CAT6K#

Initial suspicions fell on the wireless network, after all with changes come the opportunity for new problems. However, a parallel deployment of the new CleanAir APs in several other office buildings as well as a previous warehouse deployment had gone off without a hitch and no issues were experienced in those locations. Numerous wireless packet captures were performed that showed little to no issues over-the-air. Issues were experienced on both 2.4 GHz and 5 GHz frequency bands, very few retransmissions were present, and no interference was observed. Configuration changes were backed-out and testing was performed with the legacy APs, but the issue persisted.

Additionally, the packet loss experienced to/from wireless clients was not observed when communicating directly with the AP (ping, SSH, etc.). It appeared that the wired network was performing normally.

Even stranger, we had one brand-new 3502 access point that performed great, without issue. So we tried moving this good AP to a switch port where another AP experiencing problems had been connected. Still no issue with this single AP.

How Wired QoS Killed Our CAPWAP Tunnels
Reviewing the gathered data, we began investigating switches between the APs and controller for interface errors and packet drops. All counters appeared normal, and no dropped packets were seen in the queues. However, given the predictable pattern of packet loss (as shown above) the issue smelled of rate-limiting of some sort.

Our support case with Cisco was raised to TAC Escalation engineers (aka the Big Dogs in house), who proceeded to run through numerous hidden commands on our Cat6500 switches, looking at ASIC performance and low-level debugs.

Still finding nothing, we took a shot in the dark. We disabled QoS globally on one of the access layer switches that had APs attached with issues.

no mls qos

Immediately... multi-megabit performance! SMB file transfers took seconds where they took close to an hour previously. No packet loss. We've found our culprit! 

(Some of you may be thinking, if they suspected QoS issues before, why wasn't this tested earlier? In a large enterprise, testing changes to an existing configuration in a production environment is risky business. Established processes governing change management don't exactly allow for this type of activity by administrators. It's one thing to have a back-out plan for new changes, but an entirely different scenario when changing established configuration baselines.)

Questions still remained. What QoS settings were causing the problems and why were the QoS queue counters not showing any dropped packets?

Analysis of the data also revealed that only CAPWAP encapsulated packets were being dropped, not packets or traffic destined directly to the access point outside of the tunnel (pings or SSH, as mentioned). So what is unique to the CAPWAP tunnel? Well, we know that CAPWAP uses UDP packets with destination ports 5246 and 5247 to the controller. But the source ports used by the APs are randomly selected ephemeral (high-numbered) ports above 1,024.

What other application traffic uses ephemeral UDP ports? ... Voice Bearer Traffic (RTP)! A quick review of the QoS policy revealed a fairly typical looking configuration:

ip access-list extended QOS_VOICE_BEARER
 permit udp any range 16384 32767 any
 permit udp any any range 16384 32767


class-map match-all MARK_EF
  match access-group name QOS_VOICE_BEARER


policy-map MARK_VLAN_TRAFFIC
  class MARK_EF
     police flow 128000 8000 conform-action set-dscp-transmit ef exceed-action drop


interface Vlan589
 description **Wireless Management**
 service-policy input MARK_VLAN_TRAFFIC

In the policy, voice bearer traffic is identified conforming to typical design guide best practices through an ACL matching UDP ports from 16,384 through 32,767. Additionally, matching voice traffic was then marked with the EF classification and policed to 128kbps.

A quick verification of CAPWAP traffic from previous wired packet captures taken during troubleshooting efforts revealed overlapping port usage between the two applications.

And that one AP that never had an issue just chose a source port that was below the lower-bound of the ACL entry. There were likely other APs unaffected as well, but it would be highly variable based on the port chosen during the CAPWAP join process.

The workaround was to re-enable QoS globally on the switch and to prevent the switch from re-classifying traffic from the AP ports. This is accomplished by trusting the DSCP value of packets being received on switch ports connected wireless access points using the following command:

mls qos trust dscp


Additionally, the packet drops were eventually found by looking at the policed packet counter through the QoS netflow statistics on the switch.

A Cautionary Example
This incident highlights the importance of complete end-to-end QoS configuration for wireless networks. Although not directly a wireless issue, the wireless network relies on other network components to deliver traffic in a fashion congruent with unique application and traffic characteristics found in various wireless architectures.

Having a thorough understanding of both wireless and wired QoS configuration and best practices is critical for network engineers designing and deploying these systems. In addition, best practices don't always work in every environment. Sure, they are a rule-of-thumb, but engineers should examine their unique requirements and adjust accordingly.

For wireless networks, this means at a minimum, that wired QoS policies should have explicit provisions to handle wireless traffic appropriately. This may be as simple as trusting QoS values coming out of the APs, as implemented with our workaround. Or this may mean re-writing of the base QoS policy to ensure correct identification and classification of traffic. ACLs and port-matching are a broad brush which can snag victim application traffic easily. We will be reviewing the method(s) by which voice traffic is identified within our organization.

My recommendation: be fluent in end-to-end QoS, best practices, and know the applications flowing across  your network so you can make sound design decisions.

QoS is a tricky trickster. Put your game face on when dealing with him!

Cheers,
Andrew

Cisco WLC QoS Profile Bugs

There is a pretty devious little bug ID in the Cisco WLC QoS profile settings that can lead to problematic traffic forwarding for users and a pretty severe disruption to user experience.

Recently, during a guest wireless deployment, our wireless engineering team designed a solution to minimize potential impact of guest Wi-Fi devices on our production network by implementing QoS bandwidth contracts to guest users. This is accomplished through the Wireless > QoS > Profiles section of the wireless controller configuration, as shown below.

When this QoS profile is applied to a WLAN, each user's bandwidth is limited to the specified average and burst data rates (in Kbps). Additionally, the wired QoS protocol tagging feature uses 802.1p for layer 2 class of service integration with the wired network switches.

Once implemented, we began experiencing random TCP sessions dropping as well as random UDP packet loss. Scratching our heads, we started digging into what changed recently. Besides upgrading to 7.0.98.0 code, the only other change was enforcement of QoS bandwidth contracts using the Bronze QoS profile applied to the guest wireless WLAN. It also happened that the guest user base were the only individuals reporting problems. They were having IPSec and SSL VPN sessions dropping numerous times throughout each day. Definitely not a productive environment to support many corporate business partners whom we rely on to help us get work done.

Many packet traces at *multiple* points throughout the network later, we found return packets to clients missing, and apparently being dropped by the local controller. We expected that tracing packets in both CAPWAP and Mobility Ether-IP tunnels would be fairly complex, but were pleasantly surprised to see that Wireshark had protocol dissectors for both protocols! Yeah Wireshark!

Also, having a network of remote sniffers deployed throughout various points in your network is a godsend for remote configuration and packet capture capabilities. Thank you to our internal performance services team for having this capability! I can only imagine how painful it would have been to have to physically visit each point in the network to setup a SPAN port or other manual network sniffer solution.

So, knowing the local controller (as opposed to the DMZ anchor, or any other network equipment) was dropping the packets, it was a fairly simple matter of figuring out what changed to cause the issue. Rather than downgrading code, we first reverted the configuration to it's state prior to the problems by removing the QoS bandwidth contracts. Voila, problem magically disappeared.

Searching the Cisco documentation of this feature revealed no clues or warnings around best practice use of this feature (normally, if Cisco does not want a customer to modify a default value without contacting TAC or Advanced Services they will explicitly state that in the configuration guide). No warnings, okay - what is going on here, why is this feature not working correctly. Hmmm.... BUG?!

How about that, numerous bugs exist for the QoS profile settings for 802.1p tagging and bandwidth contracts. Here's one interesting one:

CSCsz20162 - WLC5500: QoS rate limiting feature is not accurate
This bug was resolved in 7.0.98.0 code, or was it?

Additionally, here are some other bugs not fixed until code versions after 7.0.98.0 (all fixes apply to a later release of code):

CSCth94887 - 5500: 802.1p markings not working for pings, EoIP and FastCache mode
CSCth90962 - Document QoS 802.1p tagging blocks traffic on untagged interfaces
CSCte64638 - Clients cannot ping or get IP address with 802.1p QoS profile
CSCti62070 - Per-User bandwidth contract blocks all traffic when set to 0
CSCte53175 - Per-User bandwidth contract blocks all traffic when set to 0

None of these directly relate to our issue, but there are enough bugs on the feature to make me think I may have found a new one. Additionally, since our issue cleared right up when removing the bandwidth contracts, a bug seems to be the likely bet here.

I also find it interesting that Cisco's Bug Toolkit does not include lookups for their 5508 series wireless controllers. So I'm stuck looking up bugs for the 4400 series, hoping that everything affecting 5508's also affects 4400's.

As Ethan Banks posted over in his blog, engineers are constantly opening support cases with vendors for bugs. "All the bugs. All the time, bugs. Bugs and bugs and bugs. Buggy bug bugs. AHHH!" I can relate to that.


So, what have we learned from this experience?

1. Document all changes to systems. There is a reason that change management practices and processes exist in most organizations. Use it, live by it, it will save your butt.
   
2. Test your changes prior to production implementation. We did test this change and still failed to catch the issue. But we will update our testing procedures. We catch most issues with testing, but some still fall through the cracks. Learn from them and update your testing plans accordingly.
   
3. Prepare troubleshooting tools ahead of time. Without remote packet capture capability, it might have taken us 10x longer to figure out where the issue was occurring. We had new addressing space, routing, firewalls, proxies, etc. all involved that I didn't discuss for brevity, but it could have been anywhere. Take a pro-active approach to deploying troubleshooting and monitoring tools. This way, when a problem comes up you can react quickly and execute on an emergency response plan. They do this for fire drills and emergency responders, take the time to do it for your network!
   
4. Admit mistakes, take responsibility, and fix the issue. I've seen too many individuals in IT attempt to deny issues and gloss over relevant information for fear of looking bad, incompetent, or just plain "not perfect" at what they do. Some even go so far as to lie about log data (or selectively focus on data that reinforces their position while discounting data that does not). This leads to longer issue resolution times, worse business impact, and once the root cause is determined, which it will be, they end up losing credibility and trust of everyone involved. Instead, present all data that could possibly be involved in the issue up front, and if it is your issue take responsibility. Everyone makes mistakes or simply cannot do a *perfect* job. Even the best engineers, including CCIEs (yes, you're fallible too)!

For the time being, I would recommend avoiding per-user bandwidth contracts in your wireless LAN controller QoS profiles. 


Best of luck out there with your bugs ;)
Andrew

Wireless QoS Part 5 - Contention Window and Final Thoughts

Read the Entire Wi-Fi Quality of Service 5-Part Series:

1. Part 1 - Background Information
2. Part 2 - IEEE 802.11e Principles
3. Part 3 - User Priorities, Access Categories and Queues
4. Part 4 - Arbitration Interframe Spacing
5. Part 5 - Contention Window and Final Thoughts

Part 5 - Contention Window and Final Thoughts
The third major QoS design change implemented as part of the 802.11e amendment and WMM certification is replacement of the PHY-specific contention window scaling with PHY / QoS Access Category specific values. Just as with AC dependent AIFS values, contention window values are now AC dependent and will provide a statistical advantage for higher priority frames.

The Contention Window
Once the station has waited the appropriate arbitration inter-frame space time, it randomly selects a value for its random backoff timer. The timer value must be within the Contention Window values defined for the priority queue. Each of the 4 priority queues has a defined Contention Window range, initially defined as ranging from 0 to CWmin, where CWmin varies between each of the queues.

Once the appropriate AIFS time has been waited, each station begins decrementing the random backoff timer by one for every slot time that passes. If another station begins transmitting before its timer has reached zero, the station defers access until the medium is available again, at which time it continues decrementing the timer from where it previously left off. Once the timer reaches zero, the station is allowed to transmit the frame over the air.

If a collision occurs where two stations transmit at the same time, no acknowledgment of the frame will be received and the station will increment its retry counter and increase its contention window according to the binary exponential backoff algorithm, up to a maximum contention window size of CWmax. The stations must then wait the appropriate AIFS time, select a new random backoff timer using the new contention window range, and proceed as before.

Similar to AIFS, the differences in the contention window values serve to prioritize traffic in higher priority queues by allowing them to wait shorter time intervals before being allowed to transmit over the air. The CWmin and CWmax values vary based on the PHY and the AC queue in use.

For review, the DCF (non-QoS) contention window values are:

• 802.11b    aCWmin 31    aCWmax 1023
• 802.11g    aCWmin 31    aCWmax 1023 (when 802.11b stations are present)
• 802.11g    aCWmin 15    aCWmax 1023
• 802.11a    aCWmin 15    aCWmax 1023
• 802.11n    aCWmin 15    aCWmax 1023

Notice how the contention window range is the same across all OFDM PHYs, with legacy CCK PHY being the only dissimilar value. Traffic prioritization is therefore very coarse, and is based not on application traffic but on the PHY used for transmission. Effectively, all frames in a legacy DCF Basic Service Set (BSS) have the same priority and access to the medium. This can lead to problems, especially for latency sensitive applications such as voice and videoconferencing.

The new EDCA contention window values vary based on the Access Category (AC) and are derived from the DCF base values shown above. These values are administrator configurable, with default values defined as:

• AC_VO (Voice)       CWmin = (aCWmin+1)/4 – 1   CWmax = (aCWmin+1)/2 – 1
• AC_VI (Video)       CWmin = (aCWmin+1)/2 – 1   CWmax = aCWmin
• AC_BE (Best Effort) CWmin = aCWmin             CWmax = aCWmax
• AC_BK (Background)  CWmin = aCWmin             CWmax = aCWmax

Note – Default values for non-AP QoS stations are defined in IEEE 802.11e amendment section 7.3.2.27 and the current IEEE 802.11-2007 standard section 7.3.2.29.

The default EDCA contention window values for the 802.11b PHY in a QoS BSS are defined as:

• Voice Queue        CWmin = 7      CWmax = 15
• Video Queue        CWmin = 15     CWmax = 31
• Best Effort Queue  CWmin = 31     CWmax = 1023
• Background Queue   CWmin = 31     CWmax = 1023

The default EDCA contention window values for the 802.11g/a/n PHY in a QoS BSS are defined as:

• Voice Queue        CWmin = 3      CWmax = 7
• Video Queue        CWmin = 7      CWmax = 15
• Best Effort Queue  CWmin = 15     CWmax = 1023
• Background Queue   CWmin = 15     CWmax = 1023

Notice the differences from legacy DCF contention window ranges. In a QoS BSS, each queue clearly has differentiated access to the medium. For instance frames in the voice queue will initially select a random backoff timer between 0 - 3, versus frames in the video queue which will initially select values between 0 - 7. In this manner, frames in the voice queue have a statistically greater chance of selecting a random timer value that is lower than frames in the video, best effort, and background queues. It is still possible that a frame from a lower priority queue will select a lower random backoff timer, but most of the time they will not. 

Also, notice how the maximum contention window range for voice and video are still relatively small compared to the other queues. On a heavily utilized network, as retransmission attempts increase, the statistical advantage for voice and video frames gets even better. 

The CWmin and CWmax values are encoded in exponent form, base 2, then decremented by 1 in the EDCA Parameter Set information element, and each field is 4 bits long. Therefore, the minimum contention window values is 0 and the maximum value is 32,767. However, in practice the typical maximum value is never set above 1,023.

Contention Window Scaling

Contention window scaling between CWmin and CWmax is easier to understand when illustrated. For the first transmission attempt, the random backoff timer is set to a value between 0 – CWmin. Only when a retransmission is required due to the lack of a returned frame acknowledgement will the possible range grow. For the first and each subsequent retransmission attempt, the contention window will double by a power of 2. This is called binary exponential backoff. Once the window grows to CWmax, it will grow no further. Subsequent retransmission attempts will use the largest contention window range when selecting a random backoff timer value until the frame is either successfully transmitted (and acknowledged) or the maximum number of retransmission attempts is reached (typically somewhere between 7 and 64 attempts; Cisco APs default to 64 attempts for example).

As an interesting side note, when optimizing networks for VoWLAN, many vendors have a feature to quickly age out latency-sensitive voice frames after fewer retransmission attempts. This is done because a voice packet that is delayed too long will be useless to the receiver since it cannot insert it back into the voice stream if it processed and sent the analog stream to the human recipient past the point where the frame was delayed. On Cisco equipment, this feature is called Low Latency MAC, it applies only to voice frames in the Voice Access Category, and it only attempts 3 retransmissions for voice frames before dropping them.

(Figure 2-5 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-7, by Cisco Systems)

Putting it all Together

Therefore, the design of priority schemes for IEEE 802.11 wireless networks are dictated by the definition 8 user priorities mapped to 4 access categories, 4 priority queues, Arbitrated Inter-Frame Spacing (AIFS) values for each queue, and Contention Window values for each queue. Higher priority traffic waits less time statistically before being allowed to transmit a frame over the network. 

Since wireless is shared medium, and medium contention is distributed among all clients under DCF and EDCA, higher priority traffic cannot be guaranteed to be able to transmit before lower priority traffic from other stations, or even internally within a station between its internal queues. 802.11e and WMM only provide for a statistical advantage for higher priority traffic. On average, higher priority frames will wait less time prior to transmission and have greater access to the medium than will lower priority frames. In a shared medium, there is no such thing as a guarantee!

The IEEE 802.11e AIFS and CWmin values are illustrated in the following figure, based on the 802.11b PHY.

(Figure 2-8 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-10, by Cisco Systems)

External Resources

Be sure to check out these additional references for more information on 802.11 arbitration (medium contention), 802.11e, and WMM:

IEEE 802.11-2007 Standard

Wi-Fi Alliance WMM Articles and Whitepapers

CWNP Whitepaper on 802.11 Arbitration

There are many other features in the 802.11e and WMM certification relating to QoS, such as Transmit Opportunities (TXOP), (Un)Scheduled Automatic Power Save Delivery (APSD), WMM Power Save, etc.

As a wireless network administrator / engineer / architect / consultant, understanding the fundamentals of 802.11 wireless network medium arbitration and frame prioritization is essential to planning, designing, implementing, and supporting a production network. Wi-Fi networks are mission-critical for many organizations these days, and skilled wireless IT staff is typically hard to find. Arm yourself with the knowledge and skills and you'll be sure to find success following you!

-Andrew

Wireless QoS Part 4 - Arbitration Interframe Spacing

Read the Entire Wi-Fi Quality of Service 5-Part Series:

1. Part 1 - Background Information
2. Part 2 - IEEE 802.11e Principles
3. Part 3 - User Priorities, Access Categories and Queues
4. Part 4 - Arbitration Interframe Spacing
5. Part 5 - Contention Window and Final Thoughts

Part 4 - Arbitration Interframe Spacing
The second QoS design change implemented as part of the 802.11e amendment and WMM certification is replacement of the one-size-fits all Distributed Inter-Frame Spacing (DIFS) for all data and management frames, with Arbitration Inter-Frame Spacing (AIFS) which is dependent on the access category and queue of the frame waiting transmission.


Arbitration Inter-Frame Spacing (AIFS) defines different inter-frame gaps for traffic from each of the 4 priority queues. This replaces the original DCF Inter-Frame Spacing (DIFS) which defined only a single inter-frame gap value for all data frames. Using AIFS, each frame awaiting transmission must wait until the medium is declared to be available through Clear Channel Assessment (CCA) and the Network Allocation Vector (NAV), not discussed here for brevity. Once the medium is available, each logical station (one for each priority queue) must wait the defined inter-frame space time based on the queue to which the traffic is assigned.

Each of the 4 priority queues has a defined inter-frame space value corresponding to the priority assigned to the queue. For example, the Voice queue is the highest priority and as such has the lowest inter-frame space timer. The AIFS timers assigned by IEEE 802.11e are all defined as 1 Short Inter-Frame Spacing (SIFS) value plus a variable number of slots times (AIFSN) which are defined by the physical layer encoding method in-use (CCK, DSSS, OFDM).

The AIFS Number (AIFSN) values are administrator configurable, with default values defined as the following:

• Voice Queue                 1 SIFS + 2 * slot time (AIFSN = 2)
• Video Queue                 1 SIFS + 2 * slot time (AIFSN = 2)
• Best Effort Queue          1 SIFS + 3 * slot time (AIFSN = 3)
• Background Queue        1 SIFS + 7 * slot time (AIFSN = 7)

Note – Default values for non-AP QoS stations are defined in IEEE 802.11e amendment section 7.3.2.27 and the current IEEE 802.11-2007 standard section 7.3.2.29.

Therefore, the default AIFS values for each PHY are as follows:

Note – 802.11g and 802.11n when operating in the 2.4 GHz spectrum use long slot times when legacy 802.11b is present, and use the short slot times when no legacy stations are present. This is done for backwards compatibility.

Here is an illustration of AIFS in action, with [X] representing sample User Priorities for differentiated inter-frame spacing between access categories. Clearly, frames with UP = 0 (AC_BE) will wait a longer IFS than frames with UP = 6 (AC_VO), thereby prioritizing voice AC traffic.

(Figure 2-9 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-11, by Cisco Systems)

The AIFSN field within the EDCA Parameter Set information element is 4 bits long, with a minimum value of 2 defined in the standard and a maximum value of 15 based on the field length limitation.

In this manner, arbitration inter-frame spacing allows a statistical advantage for traffic in higher priority queues because those frames are not required to wait as long prior to decrementing their random backoff timers.

We're almost, but not quite, finished with the technical details of wireless QoS. In my final post, I'll detail enhancements to the contention window value selection, which determines the random backoff range used by stations prior to transmitting. I'll also wrap it all up together in a nice bow, and provide a few very useful external links for further reading.

-Andrew

Wireless QoS Part 3 - User Priorities, Access Categories, and Queues

Read the Entire Wi-Fi Quality of Service 5-Part Series:

1. Part 1 - Background Information
2. Part 2 - IEEE 802.11e Principles
3. Part 3 - User Priorities, Access Categories and Queues
4. Part 4 - Arbitration Interframe Spacing
5. Part 5 - Contention Window and Final Thoughts

Part 3 - User Priorities, Access Categories and Queues
The first QoS design change implemented as part of the 802.11e amendment and WMM certification, are User Priorities, Access Categories, and Queuing Structures.


The IEEE 802.11e amendment defines 8 user priorities (UP) for class of service (CoS) definition. These user priorities were established for layer 2 data link frame prioritization in alignment with earlier CoS standards including 802.1D (based on the work within the 802.1p task group).

These 8 user priorities are grouped into 4 access categories, containing two user priorities each. User priority 0 is placed into the Best Effort AC instead of the Background AC for backwards compatibility with non-QoS stations. This was done to preserve compatibility, as the IEEE deemed QoS functionality as an optional component for certification citing the lack of need for QoS by many legacy devices and applications. The mapping between 802.1D, 802.11e UP, and Access Category is show below.

(Table 9-1 courtesy of the IEEE Std. 802.11-2007 section 9.1.3.1 on page 253)

As an example implementation, the relationship between User Priority, Access Category, and Cisco’s AVVID architecture standard (which provides different guidelines for voice and video than the IEEE standards) is detailed in the table below. Also referenced are the default markings applied by the Cisco Unified wireless network.

(Table 10-6 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 10-33, by Cisco Systems)


Queuing Structure

4 priority queues are established to prioritize data frames within each station. These 4 priority queues align with the 4 access categories mentioned previously. Each frame that a station wishes to transmit is classified and placed into one of the appropriate queues.

The inter-frame spacing and random backoff timers are calculated independently and decremented in parallel for each queue. If an internal collision occurs, whereby frames from two or more queues are ready to transmit at the same time, the higher priority queue is granted access to transmit and the other queues act as if a physical collision occurred during transmission, increment their retry counter, and increase their contention window values according to binary exponential backoff. In this manner, one physical station emulates four logical stations, one for each traffic queue. The following figure illustrates these priority queues.

(Figure 2-7 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-10, by Cisco Systems)


Through user priorities, access categories, and queuing structures, the 802.11e amendment and WMM certification provides differentiated classification of frames and the framework for frame prioritization. In the next two posts, I'll describe how frame prioritization occurs through Adaptive Inter-Frame Spacing and Contention Window scaling.


-Andrew

Wireless QoS Part 2 - IEEE 802.11e Principles

Read the Entire Wi-Fi Quality of Service 5-Part Series:

1. Part 1 - Background Information
2. Part 2 - IEEE 802.11e Principles
3. Part 3 - User Priorities, Access Categories and Queues
4. Part 4 - Arbitration Interframe Spacing
5. Part 5 - Contention Window and Final Thoughts

Part 2 - IEEE 802.11e Principles
Now, let's dig into the guiding principles behind wireless QoS design and the 802.11e amendment. This is part 2 of this series on wireless QoS, building on the background information from the first post in this series, which described the original Distributed Coordination Function (DCF) 802.11 medium contention protocol.


802.11e and WMM QoS Overview

The amended IEEE 802.11 wireless protocol supports a priority scheme that can provide up to 8 priority classes for traffic. This work was done as part of the IEEE 802.11e working group and was added as an amendment to the standard. Certifications of compliance with portions of this amendment are done by the WiFi Alliance through the WiFi Multimedia (WMM) certification process.

802.11e defines 8 user priorities for Quality of Service (QoS) of wireless traffic. These 8 user priorities are grouped into 4 Access Categories (AC) defined as Voice, Video, Best Effort, and Background. Each access category contains 2 different user priorities.

The design of this priority scheme is based on three major changes to the operation of the original 802.11 Distributed Coordination Function (DCF), which is now called Enhanced Distributed Coordination Access (EDCA). The three major changes are:

1. Establishment of 4 priority queues for traffic (Access Categories), implemented on a per-station basis
   
   
2. Arbitrated Inter-Frame Spacing (AIFS) values for each of the 4 priority queues to replace the single Distributed Coordination Inter-Frame Spacing (DIFS) value previously used for all data and management frames
   
   
3. Random Backoff timers defining Contention Window minimum (CWmin) and maximum (CWmax) values for each of the 4 priority queues

Further information on WMM certification can be found the Wi-Fi Alliance website. In addition, the IEEE 802.11e amendment and current 802.11-2007 standard can be found on the IEEE website.


In future posts, I'll detail each of these design changes and how they're implemented. We'll also see why wireless QoS provides a statistical advantage for higher priority traffic, but cannot go so far as to perform or guarantee strict priority queuing due to the use of a shared medium and distributed contention among multiple stations. This is a fundamental difference for those familiar with QoS on wired switches, which have the ability to implement strict priority-queues over a contention-free full-duplex link.


-Andrew

Wireless QoS Part 1 - Background Information

Read the Entire Wi-Fi Quality of Service 5-Part Series:

1. Part 1 - Background Information
2. Part 2 - IEEE 802.11e Principles
3. Part 3 - User Priorities, Access Categories and Queues
4. Part 4 - Arbitration Interframe Spacing
5. Part 5 - Contention Window and Final Thoughts

Part 1 - Background Information
Prior to the 802.11e amendment, legacy 802.11 wireless network operated under the Distributed Coordinate Function (DCF). DCF is built on the CSMA/CA medium contention protocol to regulate access to a shared medium, wireless airtime. On multiple access networks, a method is required to determine an orderly method for stations to determine whose turn it is to transmit a frame, how to avoid collisions, how to detect collisions, and how to gracefully recover from failed transmissions due to collisions or other errors such as interference. The CSMA/CA protocol fills this need.

The basic principles of CSMA/CA are:

1. Carrier Sense to detect incoming transmissions
2. Clear Channel Assessment (CCA) and Network Allocation Vector (NAV) to determine if the medium is idle and it is acceptable for the station to transmit a frame
3. Positive acknowledgement of frames to pro-actively avoid collisions since wireless receivers cannot passively determine if a collision has occurred as the RF energy is dispersed. This is in contrast to wired Ethernet’s use of CSMA/CD and collision detection where the receiver is able to detect collisions since the energy is reflected through the wire back to the transmitter.

802.11 networks are further built upon two principles to regulate access to the medium:

1. Inter-frame Spacing (IFS)
2. Random Backoff Contention Window

First, inter-frame spacing establishes baseline intervals that certain types of frames are required to wait prior to being able to transmit. These basic IFS values serve as a coarse method of frame prioritization. However, this prioritization is not based on the application priority but rather on frame priority for basic functionality within the 802.11 protocol. Three IFS values are defined in a non-QoS legacy 802.11 basic service set:

1. Short Inter-frame Space (SIFS) is used for frames that need to immediately follow the preceding frame, generally for control purposes. Examples include control frames that are required to positively acknowledge receipt of an immediately pre-ceding frame (ACK), to reserve the medium when an explicit request to send (RTS) has been sent by a station and are responded to with a clear-to-send frame (CTS) frame, data frames immediately following a CTS, and the second or greater fragment of a fragmented frame.
   
   
2. Point-Coordinated Inter-frame Space (PIFS) is used when the BSS is placed into a contention-free period by the access point. This mode of operation uses the AP as a point coordinator to poll capable stations for data frames, rather than let stations contend for access in a distributed fashion. This IFS is dependent on the Point-Coordinated Function (PCF) of the standard being implemented by both the AP and stations. To date, no product implements PCF, so this IFS remains un-used in the marketplace.
   
   
3. Distributed-Coordinated Inter-Frame Space (DIFS) is used when the BSS is operating in DCF mode, where stations contend with one another for access to the medium using the CSMA/CA protocol. DIFS is used for all frames for which SIFS is not applicable, typically data and management frames.

All IFS values are dependent on the physical layer (PHY) implementation. DIFS values are defined with the formula: 1 x SIFS + (2 x Slot Time). The SIFS and DIFS values are as follows:

Note – 802.11g and 802.11n values are specified for both legacy interoperability with 802.11b (long) and greenfield operation (short) in the 2.4 GHz frequency band.

(Figure 2-3 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-5, by Cisco Systems)

Second, the random backoff contention window further specifies how long a station must continue to wait if attempting to transmit a frame after detecting a busy medium. If the medium was busy when the station deferred access and queued a frame for transmission, there is a high likelihood that other stations also deferred access and have frames to transmit as well. Without a random backoff timer, multiple stations would then attempt to transmit frames at the exact same time, leading to a very high probability of collisions and degradation in network performance. The deferral of access due a busy medium serves to align the transmission of subsequent frames and increases the probability of collisions on the network.

By implementing a random backoff contention window on a per-station basis, frame transmissions are no longer aligned in most instances and allow for proper access by only one station to reserve the medium and transmit a frame; all other stations will recognize the beginning of a new transmission and pause their backoff timers and defer access to the new transmission. Additionally, should a collision still occur if two stations pick the same random backoff timer and no positive acknowledgment of the transmitted frame is received, the stations will increase the contention window range allowing for more possible random values to be selected and decrease the likelihood of subsequent collisions.

The contention window ranges from zero up to CWmin initially, with the potential to grow and range from zero to CWmax. Each subsequent collision (detected through a lack of acknowledgment in return) results in the station doubling the contention window size, up to the maximum value. Contention window values are decremented by one for every slot time that passes without detecting another station transmitting.

The DCF (non-QoS) contention window values are:

• 802.11b – aCWmin 31, aCWmax 1023
• 802.11g – aCWmin 31, aCWmax 1023 (when 802.11b stations are present)
• 802.11g – aCWmin 15, aCWmax 1023
• 802.11a – aCWmin 15, aCWmax 1023
• 802.11n – aCWmin 15, aCWmax 1023

(Figure 2-4 courtesy of “Voice over Wireless LAN 4.1 Design Guide” page 2-6, by Cisco Systems)

So to recap:

• DCF defines a distributed coordination function whereby all stations contend for access to the medium.
• DCF is governed by the underlying CSMA/CA protocol which requires positive acknowledgment of frames and pro-active collision avoidance.
• Each station defers access to the medium if busy.
• Once the medium is determined to be idle through both CCA and NAV:
• The station waits the appropriate inter-frame space (IFS) value based on the type of frame being sent.
• If the medium was idle prior to the frame being queued, the random backoff timer is skipped.
• If the medium was busy prior to the frame being queued, a random backoff value is selected in the range of 0 to CWmin (for the first transmission attempt), possibly up to 0 to CWmax (after multiple collisions).
• The random backoff value is decremented by one for every slot time that passes without detecting another station transmitting.
• Each slot time that passes, the stations checks CCA and NAV to determine if the medium is still idle, or if it is now busy with a transmission from another station.
• If another station does begin transmitting, the backoff timer is paused until the end of transmission, the appropriate IFS value is waited, then the backoff timer resumes countdown from where it left off.
• The random backoff timer expires (countdown to zero).
• The station transmits the queued frame.

In subsequent posts, I will detail what changes are made as part of the 802.11e amendment to implement Quality of Service (QoS) over wireless networks.

- Andrew