OpenWrt/LEDE Project

  • Status Closed
  • Percent Complete
    100%
  • Task Type Bug Report
  • Category Packages
  • Assigned To
    Yousong Zhou
  • Operating System All
  • Severity Low
  • Priority Very Low
  • Reported Version Trunk
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by Stanislav German-Evtushenko - 13.07.2020
Last edited by Yousong Zhou - 27.07.2020

FS#3231 - firewall3: mtu_fix should apply mss clamping for both outgoing and incoming syn packets

TL;DR When mtu_fix is enabled mss side is only altered for incoming traffic but not outgoing. This affects clients with pmtu disabled.

The issue is quite difficult to reproduce as it requires specific conditions however it happens in real setups quite regularly.

The conditions are bellow:
- OpenWrt router working in “router” mode with IPv4 traffic NATed towards Internet
- Upstream is connected by means of something that has reduced MTU (PPPoE, DS-LITE, etc)
- mtu_fix is applied to WAN interfaces

The issue:
- MSS from a client towards a server in the internet is not reduced to fit reduced MTU if the client has PMTU disabled (one example of such a client - LineageOS 16.x). As a result the packets of maximums MSS towards the server will be dropped.

Solution:
- Enable mss clamping form incoming SYN packets on the interface with mtu_fix enabled

iptables without the fix:

# iptables-save -t mangle | grep clamp
-A FORWARD -o ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu

iptables with the fix:

iptables-save -t mangle | grep clamp
-A FORWARD -o ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -i ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu

TCP session initiation without the fix (WAN mtu is 1452):

# tcpdump -ni wlan0 "(tcp[tcpflags] & (tcp-syn) != 0) and (host www.twitter.com)"
11:05:25.531925 IP 192.168.1.141.35040 > 104.244.42.129.80: Flags [S], seq 2878102236, win 64240, options [mss 1460,sackOK,TS val 2138517727 ecr 0,nop,wscale 7], length 0
11:05:25.536034 IP 104.244.42.129.80 > 192.168.1.141.35040: Flags [S.], seq 180174328, ack 2878102237, win 28960, options [mss 1420,sackOK,TS val 2685758451 ecr 2138517727,nop,wscale 8], length 0

Note *mss 1420* in the syn packet towards the client. This make the client consider 1420 being the maximum possible mss however of the client tries to send such a packet with DF flag set it will be silently dropped.

TCP session initiation the fix (WAN mtu is 1452):

# tcpdump -ni wlan0 "(tcp[tcpflags] & (tcp-syn) != 0) and (host www.twitter.com)"
11:02:42.062918 IP 192.168.1.141.34954 > 104.244.42.129.80: Flags [S], seq 4079972940, win 64240, options [mss 1460,sackOK,TS val 2138354260 ecr 0,nop,wscale 7], length 0
11:02:42.067116 IP 104.244.42.129.80 > 192.168.1.141.34954: Flags [S.], seq 416388380, ack 4079972941, win 28960, options [mss 1412,sackOK,TS val 920483133 ecr 2138354260,nop,wscale 8], length 0

Note *mss 1412* in the syn packet towards the client. This is a real maximum for mss.

Closed by  Yousong Zhou
27.07.2020 07:21
Reason for closing:  Fixed
Additional comments about closing:  

https:/ /git.openwrt.org/6c57fb7aa93df862c2db201 620089b2103d149aa

The fix is now present in master, openwrt-19.07, openwrt-18.06

Project Manager
Baptiste Jonglez commented on 24.07.2020 15:39

Good bug hunting.

For the record, it's not that obvious that the fix works: it depends on the behaviour of –clamp-mss-to-pmtu. If it only looks at the PMTU towards the destination IP of the packets, it won't work for incoming packets.

It turns out that –clamp-mss-to-pmtu looks at both source and destination IP and that's why it works:

--clamp-mss-to-pmtu
       Automatically clamp MSS value to (path_MTU - 40 for IPv4; -60 for IPv6).  This may not
       function as desired where asymmetric routes with differing path MTU exist — the kernel
       uses the path MTU which it would use to send packets from itself to the source and
       destination  IP  addresses.  Prior to Linux 2.6.25, only the path MTU to the destination
       IP address was considered by this option; subsequent kernels also consider the path MTU
       to the source IP address.
Project Manager
Yousong Zhou commented on 25.07.2020 00:40

Last time I saw this issue was when reviewing a wireguard setup. IP fragmentation
in the client→server direction was very frequent in the pppoe-wan interface. I think this is why the old way mostly works, fragmentation in one direction.

At that time I fixed it by setting a correct mtu value (16 multiples) on the local openwrt side, and having mss clamp in both directions on the remote side.

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing