Spectrum Deception

When using Wi-Fi spectrum analyzers, it's good to remember an old Russian proverb: Trust, but verify.

Recently, I was doing some work for a company that needs BYOD Wi-Fi at several office spaces in multi-tenant buildings (insert: lame excuse for not blogging more) and we ran into what seemed to be an interference problem.

Why did I think it was an interference problem? I had already completed the following checklist:

1. Cisco AP transmit power set to level 2 or 3 (that's 20 dBm to 17 dBm if you're using 3600/3700/3800 APs)? Check.

2. RRM channels 2, 3, 4, 5, 7, 8, 9 and 10 disabled? Check.

3. Excess 2.4 GHz radios disabled based on a survey done using an iPhone 4s? (What can I say? I'm a big softy for users who over-extend the life of their smartphones. They're the real MVPs of climate change.) Check.

4. OmniPeek captures, done from potential "neighbor" trouble areas, to look for channels occupied by large amounts of Retry frames? (This one has become necessary because of all of the Bad Wi-Fi that's been created by people disabling low data rates.) Check.

5. High throughput test? (I know! I shouldn't be doing this! But the heart wants what it wants, and the hearts of this company's IT crew wanted to do throughput tests.) NO.

At this point, I knew it was interference. Actually, I didn't KNOW that the problem was interference. But I knew it was one of four things (network problem, AP hardware problem, interference problem or multipath problem) and the APs were Cisco (not that Cisco is the ONLY reliable AP vendor; there are others), the 5 GHz Wi-Fi was solid and the office was carpeted, which usually neutralizes multipath.

So, I asked the IT crew to look at their spectrum analyzer, and they told me that it was seeing interference from cordless phones and Xbox controllers.

"Hmm," they said, "we don't have any Xboxes."

But they did have cordless phones. Not traditional cordless phones, but every employee (about 270 of them per floor, with each floor covering about 7,500 square feet of user space) has a wireless headset that connects to their desk phone for video conferencing.

And they did have this, from their spectrum analyzer:

The middle pane, which shows RF activity over time, shows broken vertical red lines, which is the sign of a frequency hopping device. 

The bottom pane, which records patterns of RF activity, shows narrow, 1 MHz wide spikes, which is the sign of a frequency hoping device.

A lot of headsets use Bluetooth (which uses fast [up to 1,600 hops per second] frequency hopping) or some other frequency hopping technology.

So...

"Let's see if it's these headsets," I said.

Now, I have to pause the story to give you all a little pop quiz. Which enterprise-grade, best in class, sophisticated Wi-Fi troubleshooting tool is the first one you should use when trying to gauge the interference impact of SPECIFIC DEVICE (in this case, the wireless headsets)?

I'll give you some time...

(And some space so that you have to scroll a little bit before seeing the answer...)

...

...

...

...

...

...

The Internet! (Who got it right? Give yourselves a pat on the back.)

Before launching any of my beloved Wi-Fi analysis tools, I always do some RESEARCH. In this case, I needed to do some research to find out what type of wireless technology is used by Plantronics W720 basestations w/ WH350 headsets.

I did my little "plantronics w720 bluetooth" Google search (guessing that Bluetooth was the culprit), I downloaded Plantronics' W720 data sheet, read the first few pages (with a product like this, the wireless technology is usually mentioned right away) and... 

Uh oh...

You see that little highlighted passage there? That is NOT what I wanted to see.

Actually, DECT 6.0 *is* what you want to see, but not when you're troubleshooting. You want to see DECT 6.0 when you're deploying Wi-Fi because (and I must emphasize this) DECT 6.0 does not interfere with Wi-Fi. DECT 6.0 uses the 1.9 GHz frequency band, which Wi-Fi uses 2.4 GHz and 5 GHz.

"Dang," I thought, "the wireless headsets AREN'T interfering with Wi-Fi, and now I have to start over."

Let's take another quick pause...

This is the part that is humbling about working in Wi-Fi and IT WILL NEVER CHANGE. There will always be times where you do the "right thing" and find out that it's just totally wrong. IT SUCKS. 

Now, what separates the wheat from the chaff is a love of bullshit: the chaff have it and the wheat don't. The bullshit-lover will protect the vendor/integrator/etc. at all costs in an effort to avoid an "embarrassing" situation where the touted solution doesn't work. For example, I could've said, "yep, the wireless headsets are the problem" and coerced the company into giving up on 2.4 GHz Wi-Fi entirely. 

Instead, I wanted to make sure that (WARNING: demagoguery ahead) users of iWatches, old smartphones and IoT devices (all of which support only 2.4 GHz Wi-Fi) remained supported. So, I had to eat some humble pie.

Back to the story...

Luckily for yours truly, I was working with a company that trusts me. When I told them that SOMETHING was causing frequency-hopping interference, but it wasn't those DECT 6.0 wireless headsets, they allowed me to keep working on it.

I couldn't shake the feeling that these Plantronics base station/wireless headset combos might be causing the problem, so I went back to my old research buddy, the Internet, and I kept looking. I looked at message boards and Amazon reviews and anywhere else that might give me some insight. Finally, I happened upon this little piece of marketing on Plantronics' website:

Check out that highlighted passage: "while seamlessly managing PC, mobile and desk phone calls".

Ah hah! If there's one thing I've learned working in Wi-Fi, it's to NEVER trust "seamlessly". "Seamlessly" always sucks. Controllers don't "seamlessly" facilitate roaming. APs don't "seamlessly" change channels to avoid interference. When I see "seamlessly", I know that problems will follow.

In this case, "seamlessly" meant that somehow this Plantronics basestation had to stay connected to a desk phone, a PC and a mobile phone all at the same time. For the desk phone and the PC, this was simple: a telephone jack and a USB port.

For the mobile phone, however, the connection to the Plantronics base station had to be wireless. And, sure enough, look at what the caption said for ol' number 11:

"Bluetooth". There it was. It all made sense now.

There was NO INTERFERENCE from the base station-to-wireless headset connection, just as the Plantronics data sheet said. It's a DECT 6.0 connection, and DECT 6.0 uses a different frequency than Wi-Fi.

There WAS INTERFERENCE emanating from the Plantronics base station. The base station had an active Bluetooth radio, which was used to allow mobile phones to connect for a "seamless" desk phone/PC/mobile phone experience. That Bluetooth radio was sending out messages looking for mobile phones, and those messages were messing up the 2.4 GHz Wi-Fi.

After this beautiful revelation, the rest of the work was simple. I looked up how to disable Bluetooth on Plantronics W720 base stations (turns out you just have to hold down the "mobile phone button" until the mobile phone LED [located near the "subscription button"] flashes red four times), asked the IT folks to disable Bluetooth as a test and, low and behold, their throughput tests shot up 400%.

This example of spectrum deception had a happy ending. And it can for you, too. It just takes a healthy dose of research with a dash of skepticism.

***

If you like my blog, you can support it by shopping through my Amazon link.  You can also donate Bitcoin to 1N8m1o9phSkFXpa9VUrMVHx4LJWfratseU or to my QR code:

Twitter: @Ben_SniffWiFi

ben at sniffwifi dot com