New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FS#4239 - flow_offloading_hw doesn't work with nftables (mt7621) #9241
Comments
szluki: I have exactly the same symptoms on the version |
amaumene: Hi, I found the root cause. It seems flowtable doesn't take bridges in the devices list, but takes actual devices: I guess firewall4 needs to be updated to reflect this change. |
amaumene: In terms of performances I did a quick and simple test. This has been done with a RB760iGS as router and one Raspberry Pi 4 on each side: Software offload would be slower? I've ran iperf with these options: "-R -Z -t 30 -O 3" on the client side (behind NAT). |
tkit1994: I can confirm that mir3g has the same issue. |
amaumene: I've redone my performance test and I can confirm that while the CPU usage is lower with soft offloading, speed is lower as well.
To enabled hw offload manually:
Documentation [[https://www.kernel.org/doc/html/latest/networking/nf_flowtable.html|here]] says: My understanding is we are missing a patch from upstream to support this, which means currently we need to give the physical devices and not the bridge. |
tkit1994: the default kernel is 5.10.92 on my device which is less than 5.13. |
tkit1994: How can I add wlan0 and wlan1 to flowtable devices?
/tmp/nftables.nft:4:12-13: Error: Could not process rule: Not supported
flowtable ft {
^^
/tmp/nftables.nft:22:29-40: Error: Could not process rule: No such file or directory
meta l4proto { tcp, udp } flow add @ft
Here is part of my nftables rule.
root@OpenWrt:~# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1504 qdisc fq_codel state UP group default qlen 1000
link/ether 40:31:3c:00:ad:28 brd ff:ff:ff:ff:ff:ff
inet6 fe80::4231:3cff:fe00:ad28/64 scope link
valid_lft forever preferred_lft forever
3: wan@eth0: mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 40:31:3c:00:ad:27 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.45/24 brd 192.168.1.255 scope global wan
valid_lft forever preferred_lft forever
inet6 2409:8a20:32ad:2cc0:4231:3cff:fe00:ad27/64 scope global dynamic noprefixroute
valid_lft 258974sec preferred_lft 172574sec
inet6 fe80::4231:3cff:fe00:ad27/64 scope link
valid_lft forever preferred_lft forever
4: lan2@eth0: mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN group default qlen 1000
link/ether 40:31:3c:00:ad:28 brd ff:ff:ff:ff:ff:ff
5: lan1@eth0: mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN group default qlen 1000
link/ether 40:31:3c:00:ad:28 brd ff:ff:ff:ff:ff:ff
8: br-lan: mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 40:31:3c:00:ad:28 brd ff:ff:ff:ff:ff:ff
inet 192.168.123.1/24 brd 192.168.123.255 scope global br-lan
valid_lft forever preferred_lft forever
inet6 fda4:f665:d60c::1/60 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::4231:3cff:fe00:ad28/64 scope link
valid_lft forever preferred_lft forever
9: wlan0: mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
link/ether 40:31:3c:00:ad:29 brd ff:ff:ff:ff:ff:ff
inet6 fe80::4231:3cff:fe00:ad29/64 scope link
valid_lft forever preferred_lft forever
10: wlan1: mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
link/ether 40:31:3c:00:ad:2a brd ff:ff:ff:ff:ff:ff
inet6 fe80::4231:3cff:fe00:ad2a/64 scope link
valid_lft forever preferred_lft forever
My device is mir3g with mt7621.
|
tiagogaspar8: The title should be changed since this isn't SoC specific, it happened on my AX3600, on my WRT3200ACM and on my Archer C7 v4. |
amaumene: Agreed, except I cannot edit it :( |
tiagogaspar8: Yeah, really looking forward to the day we move away from flyspray... |
jow-: Nftables/firewall4 issues fixed with
On my ER-X hw flow offloading is properly utilized:
root@er-x:~# grep HW_OFFLOAD /proc/net/nf_conntrack | sed -e 's#dst=[^[:space:]]*#dst=...#g'
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=48642 dport=993 packets=3 bytes=207 src=212.204.60.9 dst=... sport=993 dport=48642 packets=87 bytes=8526 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=48634 dport=993 packets=14 bytes=1191 src=212.204.60.9 dst=... sport=993 dport=48634 packets=97 bytes=9701 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.211 dst=... sport=56921 dport=80 packets=2 bytes=144 src=18.159.56.174 dst=... sport=80 dport=56921 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=45320 dport=443 packets=1 bytes=60 src=162.125.19.131 dst=... sport=443 dport=45320 packets=6 bytes=973 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.246 dst=... sport=57477 dport=5223 packets=1 bytes=64 src=17.57.146.173 dst=... sport=5223 dport=57477 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=49386 dport=443 packets=1 bytes=60 src=139.59.210.197 dst=... sport=443 dport=49386 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.153 dst=... sport=50663 dport=5223 packets=1 bytes=64 src=17.57.146.68 dst=... sport=5223 dport=50663 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=33622 dport=443 packets=1 bytes=60 src=130.211.10.112 dst=... sport=443 dport=33622 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=52920 dport=993 packets=15 bytes=1278 src=85.13.138.36 dst=... sport=993 dport=52920 packets=101 bytes=9814 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 udp 17 src=10.11.12.246 dst=... sport=123 dport=123 packets=1 bytes=76 src=17.253.54.253 dst=... sport=123 dport=123 packets=1 bytes=76 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=35796 dport=443 packets=1 bytes=60 src=104.16.181.15 dst=... sport=443 dport=35796 packets=16 bytes=2037 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=47338 dport=443 packets=131 bytes=19204 src=52.149.21.60 dst=... sport=443 dport=47338 packets=53 bytes=23989 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=48644 dport=993 packets=8 bytes=634 src=212.204.60.9 dst=... sport=993 dport=48644 packets=61 bytes=5935 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=48636 dport=993 packets=34 bytes=3006 src=212.204.60.9 dst=... sport=993 dport=48636 packets=96 bytes=9593 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=48638 dport=993 packets=5 bytes=408 src=212.204.60.9 dst=... sport=993 dport=48638 packets=96 bytes=9358 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=34664 dport=443 packets=1 bytes=60 src=140.82.113.25 dst=... sport=443 dport=34664 packets=5 bytes=598 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=47352 dport=443 packets=65 bytes=8140 src=52.149.21.60 dst=... sport=443 dport=47352 packets=48 bytes=19987 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=52922 dport=993 packets=1 bytes=60 src=85.13.138.36 dst=... sport=993 dport=52922 packets=9 bytes=844 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=35746 dport=443 packets=1 bytes=60 src=104.16.181.15 dst=... sport=443 dport=35746 packets=185 bytes=23581 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=43772 dport=443 packets=3 bytes=242 src=34.98.75.36 dst=... sport=443 dport=43772 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=41642 dport=443 packets=2 bytes=152 src=157.240.240.60 dst=... sport=443 dport=41642 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 tcp 6 src=10.11.12.7 dst=... sport=46492 dport=443 packets=4 bytes=651 src=162.125.19.130 dst=... sport=443 dport=46492 packets=1 bytes=60 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 udp 17 src=10.11.12.246 dst=... sport=123 dport=123 packets=1 bytes=76 src=17.253.54.125 dst=... sport=123 dport=123 packets=1 bytes=76 [HW_OFFLOAD] mark=0 zone=0 use=3
ipv4 2 udp 17 src=10.11.12.246 dst=... sport=123 dport=123 packets=1 bytes=76 src=17.253.52.125 dst=... sport=123 dport=123 packets=1 bytes=76 [HW_OFFLOAD] mark=0 zone=0 use=3
root@er-x:~#
|
tiagogaspar8: This issue isn't fixed. When I enable HW_OFFLOAD in WRT3200ACM, firewall4 doesn't start. |
jow-: @tiagogaspar8 - Please provide the exact output of: |
tiagogaspar8: Here it is (with hw offloading enabled):
|
jow-: Thanks, that is a different failure mode than which was addressed by the commits. What is reported by Please also include the output of |
tiagogaspar8: Here you go:
|
jow-: Another fix pushed with https://git.openwrt.org/?p=project/firewall4.git;a=commitdiff;h=a0518b6d0273ad3267e65953e52989a1589fefab;hp=ac99eba7d39c2ba8fa0c5335ea1e8943d8885c24 - it solves the problem on a WRT3200ACM for me. |
tiagogaspar8: I have just tested it, now the firewall starts and shows a warning that hardware offload is not available. Thanks for this! |
amaumene:
When enabling flow_offloading_hw on mt7621 device (Xiaomi Redmi Router AC2100 and Mikrotik RB760iGS) like this:
root@route2:~# grep flow_offload /etc/config/firewall
option flow_offloading '1'
option flow_offloading_hw '1'
firewall4 fails to start:
root@route2:~# /etc/init.d/firewall start
/proc/self/fd/0:9:12-13: Error: Could not process rule: Not supported
/proc/self/fd/0:51:29-44: Error: Could not process rule: No such file or directory
nftables rules generated looks good:
root@route2:
# fw4 print > /tmp/fw4# cat /tmp/fw4root@route2:
table inet fw4
flush table inet fw4
table inet fw4 {
#
# Flowtable
#
}
But for some reason nftables cannot process the rules:
root@route2:~# nft -f /tmp/fw4 -c
/tmp/fw4:9:12-13: Error: Could not process rule: Not supported
flowtable ft {
^^
/tmp/fw4:51:29-44: Error: Could not process rule: No such file or directory
meta l4proto { tcp, udp } flow offload @ft;
^^^^^^^^^^^^^^^^
This has been tested on latest trunk version:
root@route2:~# cat /etc/openwrt_version
r18638-ebc36ebb23
When only software flow offloading is enabled (option flow_offloading '1') everything works fine (verified in /proc/net/nf_conntrack)
The text was updated successfully, but these errors were encountered: