Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#563 - cake qdisc high cpu usage on tp-link 841n #5477

Open
openwrt-bot opened this issue Feb 27, 2017 · 2 comments
Open

FS#563 - cake qdisc high cpu usage on tp-link 841n #5477

openwrt-bot opened this issue Feb 27, 2017 · 2 comments
Labels

Comments

@openwrt-bot
Copy link

glycoknob:

So, I had this crazy idea to just make cake the default qdisk after seeing the battlemesh presentation. Now we are seeing high spikes in CPU usage after some time - the peaks appear to be of periodic nature. I only have some nmeter plots (basically vmstat/sar with dots - it's a busybox applet) - the high load immediatly vanishes after switching the qdiscs back to fq_codel.

I've seen this on 2 841n routers one only using ap+ibss mode the other having wan+lan+ap+ibss+tap interfaces with cake running. Both also run a bridge for the AP.

Is the high system cpu usage expacted behavoir? cake qdisc is just using defaults:

This script does the activation:

# cat /etc/hotplug.d/iface/00-cake-qdisc
#!/bin/sh

case "$ACTION" in
ifup)
grep -q '^sch_cake' '/proc/modules' && {
command -V tc > /dev/null || return
tc qdisc replace dev "$DEVICE" root cake
}
;;
esac

vim: set filetype=sh ai noet ts=4 sw=4 sts=4 :

The results (1 line = 1 second):

root@weimarnetz-188:~ :) nmeter '%40c wlan: %[nbr-vap] ibss: %[nwlan0]'

cake

SSSSSSSSUii............................. wlan: 8856 756 ibss: 4026 9162
SSSSSSSSSSSSSSSSSSSSU................... wlan: 44k 1736 ibss: 8726 49k
SSSSSSSi................................ wlan: 58k 2130 ibss: 5974 62k
SSSUUUUi................................ wlan: 58k 1296 ibss: 15k 68k
SSSSSSSSi............................... wlan: 49k 2532 ibss: 5298 53k
SSSSSSSSSSiii........................... wlan: 45k 2082 ibss: 8282 49k
SSSSSSSSSSSSSSSSSSSSSSU................. wlan: 39k 1794 ibss: 14k 45k
SSSSSSSSSSSSSSUi........................ wlan: 34k 2112 ibss: 11k 29k
SSSSSSSSSSSSSSSSSSSSSUUi................ wlan: 47k 2172 ibss: 7614 60k
U....................................... wlan: 58k 2934 ibss: 7496 61k
SSUi.................................... wlan: 53k 1944 ibss: 9.8k 58k
SSSSSSSSSSSSSSSSSSU..................... wlan: 47k 1698 ibss: 6552 51k
SSSSSSSSSSSSSUi......................... wlan: 57k 2704 ibss: 9.9k 62k
UUUiii.................................. wlan: 57k 1512 ibss: 8956 63k
SSSSSSSSSSSSSSSSSSSSSSSSii.............. wlan: 77k 4010 ibss: 17k 85k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSUii wlan: 6400 318 ibss: 450 6784
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 47k 1296 ibss: 12k 55k
SUUii................................... wlan: 49k 2542 ibss: 5604 53k
SSSSSSSSUUi............................. wlan: 56k 2354 ibss: 10k 62k
SSSSSSSSSSSSSSSSSSSSSi.................. wlan: 58k 1926 ibss: 11k 64k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 59k 2464 ibss: 22k 67k
SSSSSSSSSSSSSSUi........................ wlan: 41k 972 ibss: 10k 50k
SUi..................................... wlan: 44k 2132 ibss: 6310 47k
SUUi.................................... wlan: 52k 1642 ibss: 11k 58k
SSSSSSSSSSSSSSSSSSSSSSUi................ wlan: 54k 3826 ibss: 11k 60k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSUi wlan: 44k 1482 ibss: 6580 48k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSi.......... wlan: 32k 2396 ibss: 9732 38k
SSSSSSSSSSSSSSSSUUi..................... wlan: 18k 1164 ibss: 6888 20k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSUi... wlan: 37k 864 ibss: 13k 46k
SSSSSSSSSSSi............................ wlan: 57k 2112 ibss: 4166 60k
SSi..................................... wlan: 56k 1746 ibss: 11k 62k
SSSSSSSSSSSSSSSSSSSSi................... wlan: 43k 28k ibss: 39k 48k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSUU....... wlan: 15k 18k ibss: 25k 21k
SSSSSSSSSSSSSSSSUiiii................... wlan: 42k 255k ibss: 261k 52k
SUiiii.................................. wlan: 39k 393k ibss: 398k 54k
SSSSSSSSSSSSSSSSSSSSSSSSiii............. wlan: 26k 244k ibss: 251k 33k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSii........ wlan: 44k 86k ibss: 93k 54k
SSSSSSSSSSSSSSSSSSSSSSSSSSSUUii......... wlan: 27k 33k ibss: 41k 33k
SSSSSUiii............................... wlan: 54k 36k ibss: 40k 60k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSii wlan: 30k 46k ibss: 52k 35k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 23k 13k ibss: 20k 26k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSUii.... wlan: 53k 31k ibss: 40k 66k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSU......... wlan: 31k 18k ibss: 29k 35k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 13k 1262 ibss: 8370 16k
SSSSSSSSSSSSUi.......................... wlan: 46k 35k ibss: 54k 63k
SU...................................... wlan: 53k 87k ibss: 93k 59k
SSSSSSSSSSUUi........................... wlan: 43k 23k ibss: 32k 50k
SSSUUUiii............................... wlan: 46k 164k ibss: 172k 54k
SSSSSSSSSUiii........................... wlan: 48k 245k ibss: 250k 59k
SSSSSSSSSSSSSSS......................... wlan: 38k 5959 ibss: 10k 44k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 38k 1838 ibss: 9944 41k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi wlan: 25k 1344 ibss: 2774 25k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS wlan: 43k 2442 ibss: 10k 44k
SSSSSSSSSSSSSSSSSSSSUUii................ wlan: 59k 1968 ibss: 11k 73k
Si...................................... wlan: 60k 2354 ibss: 10k 67k
SU...................................... wlan: 57k 1680 ibss: 11k 63k
Siii.................................... wlan: 51k 3560 ibss: 8042 54k
SSSSSSSSSSSi............................ wlan: 50k 1080 ibss: 14k 58k
SSSSSSSSSSSSSUU......................... wlan: 53k 2548 ibss: 13k 59k
SSSSSSSSSSSSi........................... wlan: 46k 2309 ibss: 9093 52k
SSSSSSSSSSSSSSSSSSSSi................... wlan: 83k 3816 ibss: 14k 91k
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSi........ wlan: 25k 3210 ibss: 5668 26k
SSSSSSSSSSSSSSSSSSSSSSSUii.............. wlan: 53k 2478 ibss: 11k 59k
SSSSSSSSSSSSSSSSSSSSSSSSSUi............. wlan: 15k 1308 ibss: 8142 20k
SSUUii.................................. wlan: 799 607 ibss: 11k 5339
SUi..................................... wlan: 1449 1436 ibss: 11k 9399
SSSSSSSSSSSSSSSSSSSSSSSSSSSSU........... wlan: 240 2738 ibss: 10k 3224
SSSSSSSSSU.............................. wlan: 132 66 ibss: 10k 4804
UUi..................................... wlan: 320 347 ibss: 11k 6020

-----> fq_codel

S....................................... wlan: 0 0 ibss: 8664 3682
SSSSSSSSSU.............................. wlan: 0 0 ibss: 9022 6566
SS...................................... wlan: 703 6076 ibss: 14k 3565
S....................................... wlan: 2971 1862 ibss: 8554 7219
SUi..................................... wlan: 0 0 ibss: 7810 3732
SSUUi................................... wlan: 0 0 ibss: 8440 2920
SSii.................................... wlan: 1449 1436 ibss: 10k 7771
Ui...................................... wlan: 0 0 ibss: 8068 3258
Ui...................................... wlan: 0 0 ibss: 7910 3614
SSUi.................................... wlan: 252 185 ibss: 12k 5022
SUi..................................... wlan: 4742 1368 ibss: 9.8k 10k
SU...................................... wlan: 0 0 ibss: 11k 4646
Ui...................................... wlan: 0 0 ibss: 15k 5802
SS...................................... wlan: 0 0 ibss: 6918 3742
SU...................................... wlan: 122 2576 ibss: 4978 404
SSUii................................... wlan: 0 0 ibss: 8856 5890
SU...................................... wlan: 1449 1436 ibss: 6598 4791
SU...................................... wlan: 41 54 ibss: 4234 2101
UUii.................................... wlan: 0 0 ibss: 12k 5438
SUi..................................... wlan: 280 239 ibss: 7599 1644
U....................................... wlan: 0 0 ibss: 5962 3606
U....................................... wlan: 0 0 ibss: 10k 5502
SU...................................... wlan: 0 0 ibss: 14k 7486
Ui...................................... wlan: 0 0 ibss: 14k 5030
SU...................................... wlan: 0 0 ibss: 7930 4702
Ui...................................... wlan: 799 606 ibss: 10k 5400
SSSSSSSSSSSSSU.......................... wlan: 1449 1436 ibss: 9936 5653
S....................................... wlan: 0 0 ibss: 5818 2964
SU...................................... wlan: 0 0 ibss: 11k 7547
SU...................................... wlan: 252 185 ibss: 6264 3875
Ui...................................... wlan: 4159 995 ibss: 8313 8409
U....................................... wlan: 0 0 ibss: 3912 1240
SU...................................... wlan: 0 0 ibss: 4600 3784
S....................................... wlan: 122 2604 ibss: 9706 3374
SUUUi................................... wlan: 0 0 ibss: 13k 6328
U....................................... wlan: 0 0 ibss: 10k 6940
ii...................................... wlan: 1504 1589 ibss: 8605 4736
SUUi.................................... wlan: 108 0 ibss: 13k 8516
SUi..................................... wlan: 5241 28k ibss: 32k 9431
SSUUi................................... wlan: 9.7k 24k ibss: 33k 14k
SUi..................................... wlan: 743 456 ibss: 14k 7591
U....................................... wlan: 0 0 ibss: 2212 2732
SSSSSSSSSSU............................. wlan: 0 0 ibss: 9750 6434
........................................ wlan: 4585 492 ibss: 4616 6971
SUi..................................... wlan: 326 1155 ibss: 11k 4874
Si...................................... wlan: 0 0 ibss: 2330 214
Si...................................... wlan: 4336 3047 ibss: 18k 11k
SUi..................................... wlan: 100 0 ibss: 9838 5264
S....................................... wlan: 0 0 ibss: 14k 6298
SSSU.................................... wlan: 644 511 ibss: 6719 4294
SUii.................................... wlan: 0 0 ibss: 13k 6274
S....................................... wlan: 0 0 ibss: 6822 1738
U....................................... wlan: 0 0 ibss: 4790 1730
Ui...................................... wlan: 124 110 ibss: 5518 3544
SU...................................... wlan: 229 42 ibss: 8218 4972
Si...................................... wlan: 1000 607 ibss: 4750 2432
........................................ wlan: 1650 1436 ibss: 8372 5635
UUi..................................... wlan: 201 0 ibss: 9418 5422
Ui...................................... wlan: 0 0 ibss: 14k 7438
SSUi.................................... wlan: 3268 1862 ibss: 14k 9690
SUU..................................... wlan: 0 108 ibss: 9030 4918
SSi..................................... wlan: 80 108 ibss: 9442 5162
Ui...................................... wlan: 41 54 ibss: 9768 4757
Sii..................................... wlan: 58 104 ibss: 5570 3562
UU...................................... wlan: 0 0 ibss: 7098 4134
SU...................................... wlan: 0 0 ibss: 8142 2818
S....................................... wlan: 2941 2271 ibss: 7081 6303
SUi..................................... wlan: 315 0 ibss: 8524 3738
Ui...................................... wlan: 1309 757 ibss: 10k 7143
SUU..................................... wlan: 1450 428 ibss: 7150 5470
SUi..................................... wlan: 3187 8952 ibss: 16k 7107
U....................................... wlan: 4559 2197 ibss: 7959 8811
Ui...................................... wlan: 5564 2171 ibss: 8323 9118
i....................................... wlan: 11k 41k ibss: 50k 18k

root@weimarnetz-188:~ 8( cat /tmp/sysinfo/model
TP-Link TL-WR841N/ND v11
root@weimarnetz-188:~ :) cat /etc/os-release
NAME="weimarnetz"
VERSION="SNAPSHOT, Reboot"
ID="weimarnetz"
ID_LIKE="lede openwrt"
PRETTY_NAME="weimarnetz Reboot SNAPSHOT"
VERSION_ID="snapshot"
HOME_URL="http://lede-project.org/"
BUG_URL="http://bugs.lede-project.org/"
SUPPORT_URL="http://forum.lede-project.org/"
BUILD_ID="v0.10.0-dirty+lede-r3598-eb09d79c16"
LEDE_BOARD="ar71xx/generic"
LEDE_ARCH="mips_24kc"
LEDE_TAINTS="no-all busybox"
LEDE_DEVICE_MANUFACTURER="LEDE"
LEDE_DEVICE_MANUFACTURER_URL="http://lede-project.org/"
LEDE_DEVICE_PRODUCT="weimarnetz"
LEDE_DEVICE_REVISION="v0"
LEDE_RELEASE="weimarnetz Reboot SNAPSHOT v0.10.0-dirty+lede-r3598-eb09d79c16"

(this is basically stock release just with a versioning patch and the symbols added:

CONFIG_KERNEL_KALLSYMS=y
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BUSYBOX_CONFIG_MKSWAP=n
CONFIG_BUSYBOX_CONFIG_SWAPONOFF=n
CONFIG_BUSYBOX_CONFIG_MKPASSWD=y
CONFIG_BUSYBOX_CONFIG_ARPING=y
CONFIG_BUSYBOX_CONFIG_NMETER=y
CONFIG_PACKAGE_MAC80211_MESH=n
CONFIG_TARGET_SQUASHFS_BLOCK_SIZE=256
CONFIG_STRIP_KERNEL_EXPORTS=y

@openwrt-bot
Copy link
Author

glycoknob:

Can't edit the ticket:

  • forget to add after a reboot it's normal - this seems to appear after some memory pressure
  • it's not related to traffic afaik - both routers had very little traffic

@openwrt-bot
Copy link
Author

weedy:

I have a tp-link 842nd using qos-scripts.

I think this is more of a ath9k/memory bug.
My box will grind to a halt after a good chunk of wifi traffic. And haing QoS running or darkstat only makes it happen faster.
Slab fragmentation and ath9k buffers?

I'll have the device back soon and I'll try to bisect it while hammering with iperf/flent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant