OpenWrt/LEDE Project

  • Status Researching
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Kernel
  • Assigned To No-one
  • Operating System All
  • Severity High
  • Priority High
  • Reported Version openwrt-19.07
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by vamanea - 16.12.2019
Last edited by Baptiste Jonglez - 13.01.2020

FS#2679 - Bad 5 GHz Wi-Fi performance with some wireless NICs (hostapd 2.9)

Supply the following if possible:

- Device problem occurs on
Zyxel NBG6617 and possibly others based on reports in https://forum.openwrt.org/t/openwrt-19-07-0-second-release-candidate/49593/24

- Software versions of OpenWrt/LEDE release, packages, etc.

19.07 release and snapshot

- Steps to reproduce
Flash 19.07rcX release image, connect to 5Ghz wifi and check throughput(preferably with a 802.11ac adapter) → the max throughput is very low (~10Mbps) and big spikes in lag which makes even a SSH connection unusable

I put my test details also in the forum post: https://forum.openwrt.org/t/zyxel-nbg6617-ipq4019-regression-in-wifi-5ghz/50544/3

I did some bisecting and found that the hostapd-2.9 upgrade was the reason. When I reverted the patch and rebuild the performance was good again.

Dmitry Tunin commented on 18.12.2019 17:54

I am afraid we need more information.

I have NBG6617 and EA6350v3 in use on 19.07 branch with hostapd-common 2019-08-08-ca8c2bd2-1.

No problems on 5 GHz so far.

systemcrash commented on 18.12.2019 17:58

What would be "more information" which would help?

systemcrash commented on 18.12.2019 17:59

Something antithetical would be interesting: does applying the same 'downgrade' he used to your AP give no worsening?

Dmitry Tunin commented on 18.12.2019 18:00

It is hard to tell for sure.

Did you install anything in addition to default packages regarding wireless, like wpad-full, etc?

Does this happen with all clients, or some specific ones?

Dmitry Tunin commented on 18.12.2019 18:02

I don't need to do a downgrade. I've been building and testing the 19.07 on these devices all the way. I didn't notice any kind of trouble with wireless performance. I ran iperf all the way.

vamanea commented on 18.12.2019 19:33

I guess you have also been testing only with the -ct variant of the driver and the firmware for ath10k?

Any idea how I could debug this problem some more? On my PC I'm using Linux 5.2 and the latest available firmware for the wireless card(whatever difference that makes).

I tried reducing to 40MHz channels, no improvement, I tried limiting the max VHT MCS rate that would be reached on both the AP and the computer, but no improvement.

ping times are all over the place, ssh unusable, TCP throughput very low < 20MBPS

vamanea commented on 18.12.2019 19:34

As far as clients go, the Mobile Phone with QCA WiFi seems OK, but the Marvel ac and Broadcom 43xx an are affected.

Dmitry Tunin commented on 18.12.2019 19:36

I am testing with default -ct firmware and default package set. All works well.
You need to find out what is the difference of your setup.

vamanea commented on 18.12.2019 19:42

One other related question, are your ping times stable, under 2-3ms all the time?
Mine looked like @systemcrash every 10 or so packets there is a spike up to 100s of ms.

Dmitry Tunin commented on 18.12.2019 19:53

My ping is stable. Not always 2-3, but 2-8.

vamanea commented on 19.12.2019 16:34

Thanks, I'll try to confirm first if downgrading solves problems for others and then I'll try to narrow the problem down. Just as a note, with the hostapd downgrade the connection is rock solid.

Warren Linton commented on 07.01.2020 08:32

See also FS#2563 and  FS#2682  - Similar issues on 5GHz on an Archer C7 v2.

Bill commented on 08.01.2020 14:07

On a similar note:

I have a pair of EA6350 v3. I started using them only recently as a simple wireless access point, wired to another openwrt router. I installed 19.07.0-rc2 and also tried NotTengoBattery v0.30, and very latest daily snapshot for EA6350v3 (8th Jan 2020)

Everything was working fine (apart from some minor ping latency issues) until I stumbled across transfer speed problems with Intel Centrino 6300 802.11n (3x3) wifi cards. Speeds were capped to 12-15 Mbps on different model Dell Windows 10 laptops on both 2.4 and 5 GHz bands. Same for Intel 6205 (2x2) card which uses same windows wifi driver. On 5 GHz, Windows reports wireless link speed at full 300 Mbps. On 2.4 GHz, 144 Mbps.

When testing laptop with a 6205 wifi card running Mint Linux 18.3, wifi transfer speeds seem to be better but still short of what is normally seen.

No error messages in openwrt system log. speedtests and iperf3 test results were just SLOW and capped to 12-15 Mbps under Windows 10.

There are no speed problems when using older Intel 6200 card (uses older driver) and later 7260 N and AC cards (newer driver), and Qualcomm AR5B22 (AR9462) N cards. All running under Windows 10.

I also use a number of wireless bridges (routers) connecting to EA6350 router. There are no speed problems. Belkin F7D4302 (Broadcom BCM4718A1) with Tomato, Home Hub 5A (QCA9880) with LEDE 17, CY-SWR1100 (RaLink)with Padavan.

I flashed older 19.07-r10293 snapshot (kernel 4.14.136, hostapd-common: 2018-12-02-c2c6c01b-6) and transfer speeds with Intel 6300 and 6205 wifi cards are back to normal in Windows 10. ie. not capped at 12-15 Mbps.

I then tried 19.07.0-rc1, and issue with wifi transfer speeds being capped to 12-15 Mbps returns. Kernel 4.14.151, hostapd-common: 2018-12-02-c2c6c01b-9

Update: I also tested very latest daily snapshot (8 January 2020) with same low speed transfer results with Intel 6300 wifi card.

I can confirm low speeds with Intel 6300 wifi card still exist when using 19.07.0 on EA6350. See test results two posts down.

Admin
Petr Štetiar commented on 08.01.2020 20:29

So on my nbg6617 with Intel(R) Dual Band Wireless AC 3165, REV=0x210 on channel 52 (5260 MHz), width: 80 MHz, center1: 5290 MHz I get following results on OpenWrt SNAPSHOT, r11931-41c19dd542:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   306 MBytes   257 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   303 MBytes   254 Mbits/sec                  receiver

100 packets transmitted, 100 received, 0% packet loss, time 99150ms
rtt min/avg/max/mdev = 1.407/4.197/7.718/1.294 ms

And following results on OpenWrt 19.07.0, r10860-a3ffeb413b:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   307 MBytes   258 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   304 MBytes   255 Mbits/sec                  receiver

100 packets transmitted, 100 received, 0% packet loss, time 99142ms
rtt min/avg/max/mdev = 2.081/4.280/6.018/0.723 ms
Bill commented on 09.01.2020 05:15

What OS are you using?

fwiw, the AC 3165 is same generation as AC 7260 & 7265 card. There is no speed issue with 7260 N and AC cards with Windows 10.

Results from 19.07.0 on EA6350 v3 (ipq4018) configured as simple WAP (ch.36, 80MHz), using Intel 6300 card in Windows 10 laptop:
Download speeds are still poor

iperf3 -c 192.168.1.11 -t 10 -R

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec sender
[ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec receiver

iperf3 -c 192.168.1.11 -t 10

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.01 sec 57.5 MBytes 48.2 Mbits/sec sender
[ 4] 0.00-10.01 sec 57.5 MBytes 48.2 Mbits/sec receiver

Results from 19.07.0 on EA6350 v3 using Intel AC7260 card in Windows 10 laptop:

iperf3 -c 192.168.1.155 -t 10 -P 5 -R

[SUM] 0.00-10.00 sec 419 MBytes 352 Mbits/sec sender
[SUM] 0.00-10.00 sec 419 MBytes 352 Mbits/sec receiver

iperf3 -c 192.168.1.155 -t 10 -P 5

[SUM] 0.00-10.00 sec 398 MBytes 334 Mbits/sec sender
[SUM] 0.00-10.00 sec 397 MBytes 333 Mbits/sec receiver

330-350 Mbits/sec is not as good as stock Linksys firmware, but still far better than 16 Mbps witnessed with Intel 6300 wifi card which should be capable of 130+ Mbps.

I hope cause of these problems can be found and fixed for future 19.07.1 for EA6350 v3. In the mean time, I will continue using old 19.07-r10293 snapshot which doesn't exhibit speed problems with Intel 6205 and 6300 wifi cards.

Admin
Petr Štetiar commented on 09.01.2020 08:06
What OS are you using?

Tests were performed on Ubuntu 18.04.3 LTS.

Admin
Petr Štetiar commented on 09.01.2020 10:48
I also tested very latest daily snapshot (8 January 2020) with same low speed transfer results with Intel 6300 wifi card.

Ok, so lets try the latest hostapd then, you've two options:

Please note, that this patch is only built tested.

Admin
Petr Štetiar commented on 09.01.2020 22:27
and Broadcom 43xx an are affected.

I'm not able to confirm this either with Firmware: BCM4345/6 wl0: Oct 23 2019 23:33:06 version 7.45.197 (r723044 CY) FWID 01-dfced692 on RPi4 client and OpenWrt SNAPSHOT, r11931-41c19dd542 on my nbg6617:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   131 MBytes   110 Mbits/sec    0             sender
[  5]   0.00-10.03  sec   129 MBytes   108 Mbits/sec                  receiver

100 packets transmitted, 100 packets received, 0% packet loss
round-trip min/avg/max = 4.710/6.714/9.489 ms
Admin
Petr Štetiar commented on 09.01.2020 23:07

Same as above, but the client is MacBook with BCM43224 running Ubuntu 19.10:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   110 MBytes  91.9 Mbits/sec   72             sender
[  4]   0.00-10.00  sec   107 MBytes  90.1 Mbits/sec                  receiver

100 packets transmitted, 99 received, 1% packet loss, time 99157ms
rtt min/avg/max/mdev = 1.164/4.018/35.677/3.589 ms

and macOS 10.15.1:

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  78.0 MBytes  65.4 Mbits/sec                  sender
[  4]   0.00-10.00  sec  77.8 MBytes  65.3 Mbits/sec                  receiver
Bill commented on 10.01.2020 05:13

I installed your prebuilt for EA6350v3 from link to your private server 3 posts up from this message.

Win10 laptop with intel 6300 802.11n 3x3 wifi card connecting to 5 GHz radio AC 80MHz

Sadly, there is no improvement to gigabit LAN to 5 GHz wifi transfer speeds when downloading. Still stuck at <15 Mbps.

OpenWrt SNAPSHOT, r0-0f945ff1

iperf3 -c 192.168.1.155 -t 5  -R

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-5.00   sec  7.51 MBytes  12.6 Mbits/sec                  sender
[  4]   0.00-5.00   sec  7.51 MBytes  12.6 Mbits/sec                  receiver


iperf3 -c 192.168.1.155 -t 5

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-5.00   sec  44.1 MBytes  74.0 Mbits/sec                  sender
[  4]   0.00-5.00   sec  44.1 MBytes  73.9 Mbits/sec                  receiver

(The upload speed of 74 Mbits/sec may be higher than previous test result posted 5 messages earlier, possibly because I used 20 MHz bandwidth in earlier test. I could be wrong though)

For comparison, I reconnected to my other EA6350 v3 5GHz AC 80 MHz radio

OpenWrt 19.07-SNAPSHOT, r10293-872cbcc628

iperf3 -c 192.168.1.155 -t 5  -R

[  4]   0.00-5.00   sec  75.0 MBytes   126 Mbits/sec                  sender
[  4]   0.00-5.00   sec  75.0 MBytes   126 Mbits/sec                  receiver

iperf3 -c 192.168.1.155 -t 5

[  4]   0.00-4.97   sec  67.3 MBytes   114 Mbits/sec                  sender

Admin
Petr Štetiar commented on 10.01.2020 14:27
OpenWrt SNAPSHOT, r0-0f945ff1 (hostap_2_9-542-g33c8a10498c1) doesn't work

So no luck, the problem still exists in the latest hostapd (which is as well a plus as we might be able to ask for help upstream).

Anyway, for the start I've tried to study the code changes in hostapd between version 2.8 and 2.9 related to 80MHz/11n and found 29d8bd1dec79 ("nl80211: Add driver multi iftype HE capability parsing") interesting.

So for the start, I've prepared next build for you which moves hostapd just before this 29d8bd1dec79 change. Lets see how it goes.

Vladi commented on 10.01.2020 18:08

Updating to the latest firmware from here fixed the issue for me:
https://github.com/kvalo/ath10k-firmware/tree/master/QCA9984/hw1.0/3.9.0.2

Bill commented on 10.01.2020 18:54

I install EA6350 v3 factory image from 2 posts before this message.

Sadly no improvement with download speeds. ie. < 15 Mbps

Win10 laptop with Intel 6300 3x3 wifi N card.
I only tested 5 GHz radio. I did not test 2.4 GHz.

OpenWrt SNAPSHOT, r0-eaaaf27e

192.168.1.155 -t 5 -R

[  4]   0.00-5.00   sec  7.38 MBytes  12.4 Mbits/sec                  sender
[  4]   0.00-5.00   sec  7.38 MBytes  12.4 Mbits/sec                  receiver


iperf3 -c 192.168.1.155 -t 5

[  4]   0.00-5.00   sec  48.7 MBytes  81.6 Mbits/sec                  sender
[  4]   0.00-5.00   sec  48.6 MBytes  81.5 Mbits/sec                  receiver


REM  this is the EA6350 v3 AP with above Openwrt snapshot
Pinging 192.168.1.248 with 32 bytes of data:
Reply from 192.168.1.248: bytes=32 time=11ms TTL=64
Reply from 192.168.1.248: bytes=32 time=5ms TTL=64
Reply from 192.168.1.248: bytes=32 time=6ms TTL=64
Reply from 192.168.1.248: bytes=32 time=5ms TTL=64
Reply from 192.168.1.248: bytes=32 time=5ms TTL=64
Reply from 192.168.1.248: bytes=32 time=6ms TTL=64
Reply from 192.168.1.248: bytes=32 time=6ms TTL=64
Reply from 192.168.1.248: bytes=32 time=7ms TTL=64
Reply from 192.168.1.248: bytes=32 time=7ms TTL=64
Reply from 192.168.1.248: bytes=32 time=7ms TTL=64

It is probably irrelevant, but I understand 19.07.0-rc1 does not use hostapd 2.8/2.9, and it too suffers from low wifi transfer speeds as reported by OP and myself. I thought I should mention it.

Admin
Petr Štetiar commented on 10.01.2020 19:49
It is probably irrelevant, but I understand 19.07.0-rc1 does not use hostapd 2.8/2.9, and it too suffers from low wifi transfer speeds as reported by OP and myself. I thought I shouldmention it.

Ok, thanks, so we're probably just wasting time here.

I've read following, also on that linked forum:

I did some bisecting and found that the hostapd-2.9 upgrade was the reason. When I reverted the patch and rebuild the performance was good again.

So yours issue is probably related to something different and you should create your own bug/issue for that.

Bill commented on 10.01.2020 20:08

fwiw, I don't quite understand why reverting hostapd 2.9 solves problem for the OP, and yet OP also reported 19.07.0-rc1 did not work too - which does not use hostapd 2.8/2.9.

It is possible my issue may be related to something different, but it seems too much of a coincidence the symptoms are similar but not as 'severe' as reported by OP.

Ideally if the OP can briefly test OpenWrt SNAPSHOT, r0-eaaaf27e on nbg6617, it would confirm whether or not issue is resolved for OP, if not for my Intel 6205/6300 wifi card issue.

Thank you for your help so far.

Update: following comment by FuN_KeY in next post below, I did not change country code when I tested Petr's snapshots.

vamanea commented on 18.01.2020 10:54

Sorry Petr for the delay in answering, I was traveling.

I built a ipq40xx image from your commit but I did not see any improvement on NBG6617. SSH is unusable(too big delay between typing and the echo), ping is all over the place:

64 bytes from xxx.lan (192.168.1.2): icmp_seq=1 ttl=64 time=2.39 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=2 ttl=64 time=1.62 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=3 ttl=64 time=4.04 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=4 ttl=64 time=3.61 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=5 ttl=64 time=4.09 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=6 ttl=64 time=28.3 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=7 ttl=64 time=65.8 ms
64 bytes from xxx.lan (192.168.1.2): icmp_seq=8 ttl=64 time=18.6 ms

and the throughput with the 802.11ac Marvel wifi adapter is around 15Mbits/s with iperf. I tried changing the country code, but I didn't see any big changes.

When I switch back to my 19.07 HEAD with the following patches reverted then everything returns to normal:

commit f9c41493920f45a05930a05bcaac11e12764c90a (HEAD -> 19.07-no2.9)
    Revert "hostapd: Update to version 2.9 (2019-08-08)"
    This reverts commit 5e8d1b52dafb04428898fe3ba9c2920a0fb6f653.

commit 2f675b6bd9ab3842390049fd4ffeb7223ea040b3
    Revert "hostapd: use config option CONFIG_NO_LINUX_PACKET_SOCKET_WAR"
    This reverts commit 90a0daf4fe2aca55169c5ff7862b867516837c30.

commit 6ce31905bfdf75026e3292f70bd8f1c1b0eb7bbe
    Revert "hostapd: Remove unneeded patch"
    This reverts commit 81908622a987b561a601bb144310bc35fd123bbf.

commit 6e52d96388973cf247661b137ae62d5f54a8f2fb
    Revert "hostapd: add IEEE 802.11k support"
    This reverts commit a6e7f68c7f7539d20d884acc06c07b53e9683d93.
vamanea commented on 18.01.2020 12:02

Just for reference here is the iperf3 with hostapd 2.8

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   447 MBytes   375 Mbits/sec  381             sender
[  4]   0.00-10.00  sec   444 MBytes   372 Mbits/sec                  receiver
Admin
Petr Štetiar commented on 18.01.2020 13:26
When I switch back to my 19.07 HEAD with the following patches reverted then everything returns to normal:

Ok, can you try to narrow it down to single commit? Do you really need all this 4 reverts, can you double check that? Just do one revert, flash, iperf test, repeat until you get 300Mbits/sec back, then report the state/commits of your tree.

What 802.11ac Marvel wifi adapter is that exactly, what operating system is running on the client, what firmware version are you using on that Marvell adapter?

vamanea commented on 20.01.2020 17:07

Yes, sorry should have mentioned this before, the offending commit is "hostapd: Update to version 2.9 (2019-08-08)" The rest are needed because the patches don't apply cleanly.

The Marvel adapter is Marvell Technology Group Ltd. 88W8897 [AVASTAR] 802.11ac Wireless running on a Ubuntu with kernel 5.2.21. The firmware running on the device is 15.68.7.p154

Warren Linton commented on 20.01.2020 19:33

I'll just add to this - I am using a MS Surface 6 with a Marvell Avastar 8897 NIC on windows 10 (release 1909 with the Avastar driver 15.68.17015.112) talking to a Archer C7 V2 on trunk (r11982-c6e972c877).

Same issues on hostapd 2.9 - see FS#2563. Reverting to 2.8 is ok, see https://forum.openwrt.org/t/zyxel-nbg6617-ipq4019-regression-in-wifi-5ghz/50544/26?u=muddyfeet

Using Openwrt 18.06.5 is fine.

Admin
Petr Štetiar commented on 23.01.2020 15:01

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing