Once again I was asked to get involved when a customer had problems with an application. I won’t go into application or problem details here but the process eventually involved capturing traffic on a customer laptop to see what was actually happening.
The consultant doing the capture was not using Wireshark as I would have expected but he was using Microsoft Network Monitor. I got the resulted capture file and opened it in Wireshark, but I was puzzled about the output:
(Note: I needed to anonymize the data for showing here but because of the file format (“Microsoft NetMon 2.x” as Wireshark said) I couldn’t use TraceWrangler, so I exported the packet dissections as text in Wireshark and then replaced the client data in a text editor, from where the screenshots here are as well. Actually quite easy task for this short snippet.)
I expected to see a nice capture of the laptop’s IP communication, but instead I got only one-way communication of client (192.168.8.100) to server (10.192.10.55), and the rest was some weird 802.11 radio-related stuff (as I saw it), as the laptop was WiFi-connected. I even asked the consultant if the capture file surely included all the two-way communication and he said that there is everything included.
Ok then. When I looked at the frame list again I realized that apparently the 802.11 frames were not just some useless WiFi signaling frames (we weren’t troubleshooting WiFi here) but they were actually the incoming data frames from the WiFi access point to the laptop. And yes, it was confirmed that the WiFi was provided by a small Huawei mobile AP, and the laptop had the Intel MAC address. But still, the incoming frames did not have any meaningful IP information but a protocol stack like this, as shown in Wireshark packet details pane:
In the packet flow the frame #16 should be the TCP SYN-ACK response from the server, but as you see there was no IP (or TCP) header shown at all, there was just “IEEE 802.11 QoS Data” and 48 bytes of plain unidentified data.
I tried using Decode As function in Wireshark to get it to understand the protocol stack but I couldn’t get any results that way.
I continued with investigating the data structure. I checked the previous frame (#15) sent by the client:
When compared to frame #16 the four first layers were the same: the frame information summary, NetMon header, 802.11 radio information and IEEE 802.11 frame. But in #16 they were followed by just data, while #15 had LLC (Logical-Link Control).
In #15 the LLC header contained the first “magic bytes” AA AA 03 (see Subnetwork Access Protocol for example) and it was located right after the “IEEE 802.11 Data” field. In #16 however Wireshark showed that between “IEEE 802.11 QoS Data” and “Data (48 bytes)” there were four bytes “AA AA 03 00” in a kind of no man’s land. Those bytes clearly belonged to the LLC field but for some reason Wireshark just skipped them, and as the result the rest of the data was not in any kind of sync anymore.
Furthermore, I was able to find the relevant IP addresses in the hex data, so there it all was, but the decoding was not catching the data correctly.
I started searching for the 802.11 frame types, especially the “QoS Data”, and found the article CWAP – 802.11 Data Frame Types by fellow CCIE Rasika Nayanajith. He had a Wireshark screenshot about this 802.11 QoS Data frame. When I compared his output to my output, I saw his output had the Flags field as 0x02, while my frames had Flags set to 0x42:
The extra bit set in my case was Protected flag.
I went in searching mode again, and this time I found references that I may need to instruct Wireshark to ignore the Protection bit. And there it was in the IEEE 802.11 protocol preferences in Wireshark:
The Ignore the Protection bit setting is No by default. I set it to Yes – without IV, and this is what I got in the frame list immediately after that:
Now there was full L3 and L4 information shown also for the incoming frames! This is how frame #16 looked:
All the IP and TCP headers were there correctly.
So yes, the consultant was correct, the capture file really included all the data we wanted. With this corrected environment I could continue with the troubleshooting, and in the capture file I actually found an important lead for hopefully resolving the case. At the time of writing this I believe a major software vendor is still trying to find out how their product should actually be configured to work expectedly.