Bluetooth That Will Make You Cry

Question: What happens when you put 40 Bluetooth devices in simultaneous operation within 800 sq. feet of each other?

Answer: This...

Spectrum Analysis Capture of 40 Simultaneous Bluetooth Devices

Now think about this on your own: Can you spot the Wi-Fi going on at the same time? No, you say that you can't? Why not? (Please provide your answers in the comments section for this post)

Here's an image of the "baseline" Wi-Fi activity prior to the Bluetooth activity in the same environment. I bet you can spot the Wi-Fi in this one. There's a typical enterprise deployment with APs on channels 1, 6, and 11, plus an iperf performance measurement currently going on across channel 6.

What's Going On Here?
This is a capture of 40 Honeywell Xenon 1902 cordless Bluetooth area-imaging barcode scanners operating at the same time. These units are used in retail environments at checkout registers to provide faster scan rates, ease of mobility, and overall a faster checkout process for customers. The "area-imaging" implies reading of 2D barcodes such as QR codes and such.

Honeywell Xenon 1920 Area-Imaging Cordless Barcode Scanners

Bluetooth is becoming the default communication method for cordless barcode scanners by almost every manufacturer. I did a bit of research on it, and only 2 out of 8 manufacturers of cordless barcode scanners support an alternative to Bluetooth (one used Wi-Fi and another used narrowband at 433 or 910 MHz). But every single one of them provide a Bluetooth option, and it is typically the more prominently displayed option on their websites. I also received information from a good friend, Joel Barrett, that indicates manufacturers are all moving to Bluetooth scanners and support for other options will be phased out. So, if I wanted to choose a different option I could probably get one now, but support would be short-lived and I'd end up having to switch to Bluetooth anyways. Might as well bite the bullet now and figure out how to deploy these in my environment to co-exist [relatively] peacefully with my Wi-Fi network.

Performance Impact
These units are rated as a Class 2 Bluetooth transmitter, meaning they should have a maximum power output of 2.5mW and an estimated range of 10 meters. Sounds nice and low, and one would expect minimal impact to Wi-Fi. But the reality can be far different!

It's important to understand the different impact that Bluetooth can have in an enterprise environment than in a consumer environment. The deployment scenarios can be dramatically different, and a high concentration of Bluetooth devices in a small area directly correlates to decreased performance. Sure, the typical duty cycle of a single Bluetooth device is small. But as Bluetooth device density increases so does Wi-Fi performance impact due to increased CCA busy detection by Wi-Fi devices and increased frame corruption when Bluetooth can't avoid APs on multiple channels. Even if Bluetooth version 1.2 and later capable devices are used that implement adaptive frequency hopping, they cannot avoid interfering with Wi-Fi access points spread out across the entire 2.4GHz frequency band.

In executing Wi-Fi performance testing with these Bluetooth devices our team ran multiple scenarios, changing Bluetooth power levels, pairing status, and scan rates. What we came away with also varied dramatically based on these settings. Our baseline was an 802.11g network with 20 Mbps throughput. The environment is an open-air retail setting at the front register checkout lanes.

Clearly, despite being rated as a Class 2 Bluetooth device, the RF signal was carrying quite far. Luckily, Honeywell has done a good job providing management tools to customize the radio performance of their barcode scanners. By adjusting the power level down we were able to minimize the impacted area as well as the impact to the Wi-Fi network.

What made our situation even more challenging was the desire to deploy VoWiFi around the same time as the cordless barcode scanners in the same environment. Our preference is to use voice handsets that support 5GHz frequency bands, but that may not be possible due to other business considerations on device capabilities and application support (we're still evaluating solutions). So, we ran 2.4GHz voice tests that showed an average 20% frame loss rate when the Bluetooth scanners operated at 10% (0.25mW) and an unacceptable user experience. When the power level was reduced to 1% (0.025mW) the frame loss was much lower and no perceptible voice quality issues could be observed by end-users.

Ultimately, we were able to find a compromise that allowed the use of these cordless barcode scanners while minimizing impact to the Wi-Fi network.

Deployment Considerations
Here are some considerations when deploying Bluetooth in an enterprise environment:

• Device Selection
  Select Bluetooth devices that are configurable and easy to provision. The device should support modification of all of the settings listed below, and keep those configuration settings across reboots. If a device is factory-reset or the battery dies, it should be able easy to re-apply the custom configuration settings by staff in the field with minimal training and effort.
  
  Recommendation - Purchase "enterprise-class" Bluetooth devices that allow custom configuration.
  
  
• Device Density
  In general, the more Bluetooth devices operating in a confined area, the more impact to the Wi-Fi network. Pretty simple. Each individual Bluetooth device has minimal impact due to very low duty cycle (airtime used), but as more and more devices are added it linearly increases interference and decreases Wi-Fi performance.
  
  Recommendation - Minimize Bluetooth device density as much as possible.
  
  
• Power Level
  The Bluetooth transmission power level, especially in dense deployments, can have a dramatic effect on the impact to a Wi-Fi network. In our testing, reducing power levels from 100% (2.5mW) down to 1% (0.025mW) significantly reduced the impact to the Wi-Fi network, and the range provided was still adequate to meet our business needs.
  
  Recommendation - Reduce Bluetooth transmission power to the lowest setting that still allows reliable functionality for a given deployment scenario.
  
  
• Bluetooth Pairing
  The pairing status of a Bluetooth device can determine how actively the device transmits. A paired device usually transmits much less frequently than an unpaired one. Unpaired devices may constantly search for a base station or partner, often times transmitting very frequently in what many manufacturers call "distress mode". Honeywell also provides a configurable scan timer that adjusts how long an unpaired device will search for its partner. We adjusted this setting down to 3 cycles instead of infinite. It will also scan whenever the trigger is pulled. This minimizes interference in the worst-case scenario that the device gets unpaired.
  
  Recommendation - Establish sound operational practices to ensure Bluetooth devices remain paired at all times. Additionally, adjust scanning timers down to a reasonable level from defaults.
  
  
• Know Your Environment
  Bluetooth impact will also vary based on the environmental characteristics in which it is deployed. In my situation the impact was significant because an "open-air" environment. But that may not be the case in an office with many more walls and obstacles that prevent RF signal propagation. Also, know your Wi-Fi client device capabilities and applications. If you only use data applications like web surfing and file transfer, Bluetooth may not be a big risk. But if you use real-time applications like voice or streaming video, then it could cause usability issues.
  
  Recommendation - Understand how Bluetooth impact will vary based on the facility characteristics and applications deployed on the Wi-Fi network.
  
  
• Migrate Wi-Fi to 5GHz
  If you can't mitigate the performance issues with Bluetooth or any other source of interference in the 2.4GHz spectrum, move your clients over to 5GHz. This one is easy to understand, but can be difficult to achieve in practice. Consider the influx of mobile devices that only operate using a single-radio 2.4GHz chipset. What applications will be used on those devices, and what is the implied or defined service level agreement between the network team and business teams?
  
  Recommendation - Use band steering techniques or different WLAN configurations on the Wi-Fi network to move 5GHz capable clients over to this band.

Andrew's Take
The delivery of business capabilities will always trump non-functional technology requirements (unless your business is technology). As IT professionals we must understand and accept this. Instead of saying "no" to solutions that go against best practices, work to understand the business request and develop a solution that delivers what the business needs. This requires compromise on your part, not every solution can be 100% the best technology. Often times, the best technical solution is NOT the best business solution.

Network administrators should be aware of initiatives to use Bluetooth client devices within their environments. They do not need to block use of these devices outright, but do need to perform proper performance analysis and modify Bluetooth configuration settings to minimize impact to the Wi-Fi network.

This type of scenario also highlights why I prefer to deploy wireless access points with integrated spectrum analysis in my environment. Day-to-day operation of this network requires "always-on" visibility into non-Wi-Fi sources of interference so that I can baseline, trend, and report on network performance. It's also why I prefer a dedicated SpecAn chipset in APs, and am skeptical of 1st-generation Atheros-based solutions that cannot perform concurrent spectrum analysis while serving Wi-Fi clients. They require a dedicated RF / Spectrum mode of operation and have to be taken out of service. I hear that's changing and Atheros solutions can now provide that "always-on" capability, but have yet to see one hands-on or test vendor claims.

Cheers,
Andrew