OpenWrt/LEDE Project

  • Status Unconfirmed
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Kernel
  • Assigned To No-one
  • Operating System All
  • Severity Medium
  • Priority Very Low
  • Reported Version Trunk
  • Due in Version Undecided
  • Due Date Undecided
  • Votes 1
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by Vivek - 14.03.2019

FS#2186 - DSA driver performance issue


Device: Linksys EA9500
Trunk: Kernel v4.19.25

I’ve compiled my own kernel and used DSA driver instead of swconfig
one for this router. Note: Kernel v4.19 enables broadcom tags for
B53 family of switches in this DSA driver

Here are some of my observations:

1. Lan to Lan speed is indeed 1Gbps (file copy)
2. However, Lan to Wan port is capped at ~500Mbps

 i. My internet is limited to 200Mbps, please see by test 
    setup below
ii. One of the users who has 1GBps internet had initially
    reported this issue

I’ve tried all the lan ports but the result is same. Note that this
router has 8 lan ports and 1 wan port. Where, switch bcm53012
covers lan4, lan7, lan8 and wan ports. While bcm53125 covers lan1,
lan2, lan3, lan5 and lan6 ports. bcm53125’s cpu port(8) hangs of
bcm53012 at port(0). bcm53012 is connected to GMAC.

When I load factory firmware, I do get 1Gbps from lan to wan port

Test setup:


Router ⇔ pfsense box ⇔ Modem

             ||
            SSD

File copy is limited to ~500Mbps

Thanks,

Vivek


Project Manager
Florian Fainelli commented on 15.03.2019 23:28

Keep in mind that the stock firmware makes use of the NATP accelerator, whereas the upstream kernel or OpenWrt does not have that capability. This is somewhere on my TODO of things to play with, but I have not had the time yet. Maybe Rafal has something though I suspect he is in the same boat.

If you can get a "perf record" of your iperf session on the router while doing forwarding that would be helpful to see if there are simple hotspots that could be optimized.

Vivek commented on 16.03.2019 16:33

Sure, I'll try to get the perf record from my machine. As well as from the user that has 1Gbps internet.

btw, on the side note I modified the code to force enable tagging for bcm53125. Over all tagging is working great.

Thanks Florian!

LGA1150 commented on 17.03.2019 06:56

As Florian stated above, this is not related to b53 driver.
Nonetheless you can still enable 'software flow-offloading' in firewall settings to reach line speed.

Vivek commented on 20.03.2019 09:00

Florian,

Here you go:

root@OpenWrt:~# perf report -k /tmp/vmlinux
# To display the perf.data header info, please use --header/--header-only options.
#
Failed to open /lib/libc.so, continuing without symbols
Failed to open /usr/bin/iperf3, continuing without symbols
Failed to open /lib/libgcc_s.so.1, continuing without symbols
#
# Total Lost Samples: 0
#
# Samples: 39K of event 'cycles:ppp'
# Event count (approx.): 13631283929
#
# Overhead  Command  Shared Object           Symbol
# ........  .......  ......................  ......................................
#
    18.59%  iperf3   [kernel.kallsyms]       [k] csum_partial
    14.75%  iperf3   [kernel.kallsyms]       [k] __copy_to_user_std
     8.05%  iperf3   [kernel.kallsyms]       [k] v7_dma_inv_range
     7.07%  iperf3   [kernel.kallsyms]       [k] __irqentry_text_end
     5.42%  iperf3   [kernel.kallsyms]       [k] l2c210_inv_range
     2.43%  iperf3   [kernel.kallsyms]       [k] bcma_host_soc_read32
     1.83%  iperf3   [kernel.kallsyms]       [k] tcp_rcv_established
     1.72%  iperf3   [kernel.kallsyms]       [k] bgmac_poll
     1.38%  iperf3   [kernel.kallsyms]       [k] copy_page_to_iter
     1.38%  iperf3   [kernel.kallsyms]       [k] page_address
     1.31%  iperf3   [kernel.kallsyms]       [k] skb_try_coalesce
     1.21%  iperf3   [kernel.kallsyms]       [k] net_rx_action
     1.07%  iperf3   [kernel.kallsyms]       [k] netdev_alloc_frag
     1.05%  iperf3   [ip_tables]             [k] ipt_do_table
     0.91%  iperf3   [kernel.kallsyms]       [k] tcp_v4_do_rcv
     0.90%  iperf3   [kernel.kallsyms]       [k] enqueue_to_backlog
     0.80%  iperf3   [kernel.kallsyms]       [k] dev_gro_receive
     0.70%  iperf3   [kernel.kallsyms]       [k] build_skb
     0.68%  iperf3   [kernel.kallsyms]       [k] dma_cache_maint_page
     0.61%  iperf3   [kernel.kallsyms]       [k] __free_pages_ok
     0.59%  iperf3   [kernel.kallsyms]       [k] __skb_get_hash
     0.54%  iperf3   [kernel.kallsyms]       [k] ___bpf_prog_run
     0.51%  iperf3   [kernel.kallsyms]       [k] mmioset
     0.49%  iperf3   [kernel.kallsyms]       [k] __skb_flow_dissect
     0.48%  iperf3   [kernel.kallsyms]       [k] kmem_cache_alloc
     0.47%  iperf3   [kernel.kallsyms]       [k] __slab_free.constprop.2
     0.43%  iperf3   [kernel.kallsyms]       [k] ktime_get_with_offset
     0.40%  iperf3   [kernel.kallsyms]       [k] copyout
     0.40%  iperf3   [kernel.kallsyms]       [k] tcp_recvmsg
     0.39%  iperf3   [kernel.kallsyms]       [k] generic_exec_single
     0.38%  iperf3   [kernel.kallsyms]       [k] sched_clock
     0.38%  iperf3   [kernel.kallsyms]       [k] bcma_host_soc_write32
     0.38%  iperf3   [kernel.kallsyms]       [k] sch_direct_xmit
     0.36%  iperf3   [kernel.kallsyms]       [k] bgmac_dma_rx_skb_for_slot
     0.36%  iperf3   [kernel.kallsyms]       [k] skb_release_head_state
     0.36%  iperf3   [kernel.kallsyms]       [k] skb_copy_datagram_iter
     0.35%  iperf3   [kernel.kallsyms]       [k] kmem_cache_free
     0.35%  iperf3   [kernel.kallsyms]       [k] __netif_receive_skb_core
     0.34%  iperf3   [kernel.kallsyms]       [k] __tcp_ack_snd_check
     0.33%  iperf3   [kernel.kallsyms]       [k] netif_receive_skb_internal
     0.33%  iperf3   [kernel.kallsyms]       [k] arm_dma_unmap_page
     0.31%  iperf3   [kernel.kallsyms]       [k] skb_release_data
     0.31%  iperf3   [kernel.kallsyms]       [k] tcp_event_data_recv
     0.31%  iperf3   [kernel.kallsyms]       [k] skb_put
     0.30%  iperf3   [kernel.kallsyms]       [k] eth_type_trans
     0.30%  iperf3   [kernel.kallsyms]       [k] __release_sock
     0.29%  iperf3   [kernel.kallsyms]       [k] get_rps_cpu
     0.29%  iperf3   [kernel.kallsyms]       [k] tcp_v4_rcv
     0.28%  iperf3   [nf_conntrack]          [k] _429
     0.27%  iperf3   [kernel.kallsyms]       [k] __build_skb
     0.27%  iperf3   [kernel.kallsyms]       [k] tcp_try_coalesce.part.9
     0.27%  iperf3   [kernel.kallsyms]       [k] __dev_queue_xmit
     0.25%  iperf3   [kernel.kallsyms]       [k] __skb_checksum
     0.25%  iperf3   [xt_conntrack]          [k] _15
     0.24%  iperf3   [nf_conntrack]          [k] _254
     0.24%  iperf3   [kernel.kallsyms]       [k] __skb_checksum_complete
     0.23%  iperf3   [nf_conntrack]          [k] nf_conntrack_in
     0.23%  iperf3   [kernel.kallsyms]       [k] gt_sched_clock_read
     0.22%  iperf3   [kernel.kallsyms]       [k] napi_gro_receive
     0.22%  iperf3   [kernel.kallsyms]       [k] arm_dma_map_page
     0.22%  iperf3   [kernel.kallsyms]       [k] tcp_queue_rcv
     0.22%  iperf3   [kernel.kallsyms]       [k] tcp_send_delayed_ack
     0.22%  iperf3   [kernel.kallsyms]       [k] bcma_bgmac_read
     0.22%  iperf3   [kernel.kallsyms]       [k] nf_hook_slow
     0.22%  iperf3   [kernel.kallsyms]       [k] __tcp_select_window
     0.21%  iperf3   [kernel.kallsyms]       [k] __dma_page_dev_to_cpu
     0.21%  iperf3   [kernel.kallsyms]       [k] gic_raise_softirq
     0.21%  iperf3   [kernel.kallsyms]       [k] ipv4_dst_check
     0.20%  iperf3   [kernel.kallsyms]       [k] __local_bh_enable_ip
     0.20%  iperf3   [kernel.kallsyms]       [k] __dma_page_cpu_to_dev
     0.19%  iperf3   [kernel.kallsyms]       [k] smp_cross_call
     0.18%  iperf3   [kernel.kallsyms]       [k] bcma_bgmac_write
     0.18%  iperf3   [kernel.kallsyms]       [k] process_backlog
     0.18%  iperf3   [kernel.kallsyms]       [k] smp_call_function_single_async
     0.18%  iperf3   [kernel.kallsyms]       [k] __kunmap_atomic
     0.18%  iperf3   [kernel.kallsyms]       [k] _copy_to_iter
     0.16%  iperf3   [kernel.kallsyms]       [k] dsa_switch_rcv
     0.15%  iperf3   [kernel.kallsyms]       [k] arm_heavy_mb
     0.15%  iperf3   [kernel.kallsyms]       [k] tcp_rcv_space_adjust
     0.15%  iperf3   [kernel.kallsyms]       [k] tcp_poll
     0.14%  iperf3   [kernel.kallsyms]       [k] do_select
     0.13%  iperf3   [nf_nat]                [k] nf_nat_inet_fn
     0.13%  iperf3   [kernel.kallsyms]       [k] __tcp_transmit_skb
     0.13%  iperf3   [kernel.kallsyms]       [k] skb_gro_reset_offset
     0.13%  iperf3   [kernel.kallsyms]       [k] net_rps_send_ipi
     0.13%  iperf3   [kernel.kallsyms]       [k] skb_checksum
     0.12%  iperf3   [kernel.kallsyms]       [k] __kfree_skb_flush
     0.12%  iperf3   [kernel.kallsyms]       [k] kmap_atomic
     0.12%  iperf3   [kernel.kallsyms]       [k] napi_complete_done
     0.12%  iperf3   [kernel.kallsyms]       [k] sock_def_readable
     0.12%  iperf3   [kernel.kallsyms]       [k] csum_partial_ext
     0.12%  iperf3   [kernel.kallsyms]       [k] skb_pull
     0.12%  iperf3   [kernel.kallsyms]       [k] sock_poll
     0.12%  iperf3   [kernel.kallsyms]       [k] tcp_grow_window
     0.11%  iperf3   [kernel.kallsyms]       [k] arch_send_call_function_single_ipi
     0.11%  iperf3   [nf_conntrack]          [k] _422
     0.11%  iperf3   [kernel.kallsyms]       [k] mod_timer
     0.10%  iperf3   [kernel.kallsyms]       [k] rcu_bh_qs
     0.10%  iperf3   [kernel.kallsyms]       [k] _cond_resched
     0.10%  iperf3   [kernel.kallsyms]       [k] gt_clocksource_read
     0.10%  iperf3   [kernel.kallsyms]       [k] __vfs_read
     0.10%  iperf3   [kernel.kallsyms]       [k] cpumask_next
     0.10%  iperf3   [iptable_mangle]        [k] _10
     0.09%  iperf3   [kernel.kallsyms]       [k] rcu_all_qs
     0.09%  iperf3   [nf_nat_ipv4]           [k] _39
     0.09%  iperf3   [kernel.kallsyms]       [k] _raw_spin_unlock_irqrestore
     0.09%  iperf3   [nf_defrag_ipv4]        [k] _13
     0.09%  iperf3   [kernel.kallsyms]       [k] ktime_get_ts64
     0.09%  iperf3   [kernel.kallsyms]       [k] l2c210_sync
     0.09%  iperf3   [kernel.kallsyms]       [k] arm_dma_mapping_error
     0.09%  iperf3   [nf_conntrack]          [k] _389
     0.09%  iperf3   [kernel.kallsyms]       [k] bgmac_start_xmit
     0.09%  iperf3   [kernel.kallsyms]       [k] __kmalloc_track_caller
     0.09%  iperf3   [kernel.kallsyms]       [k] dev_hard_start_xmit
     0.09%  iperf3   [kernel.kallsyms]       [k] validate_xmit_skb.constprop.28
     0.08%  iperf3   [kernel.kallsyms]       [k] ip_rcv_core.constprop.2
     0.08%  iperf3   [nf_conntrack]          [k] _308
     0.08%  iperf3   [kernel.kallsyms]       [k] dst_release
     0.08%  iperf3   [kernel.kallsyms]       [k] ip_finish_output2
     0.08%  iperf3   [kernel.kallsyms]       [k] ip_rcv_finish_core.constprop.3
     0.08%  iperf3   [kernel.kallsyms]       [k] __inet_lookup_established
     0.08%  iperf3   [kernel.kallsyms]       [k] tcp_v4_early_demux
     0.08%  iperf3   [kernel.kallsyms]       [k] _raw_spin_lock_bh
     0.08%  iperf3   [kernel.kallsyms]       [k] core_sys_select
     0.08%  iperf3   [nf_conntrack]          [k] __nf_ct_refresh_acct
     0.08%  iperf3   [kernel.kallsyms]       [k] page_frag_free
     0.07%  iperf3   [nf_conntrack]          [k] _337
     0.07%  iperf3   [kernel.kallsyms]       [k] __ip_queue_xmit
     0.07%  iperf3   [kernel.kallsyms]       [k] should_failslab
     0.07%  iperf3   [xt_conntrack]          [k] _23
     0.07%  iperf3   [kernel.kallsyms]       [k] vector_swi
     0.07%  iperf3   [kernel.kallsyms]       [k] put_cpu_partial
     0.07%  iperf3   [kernel.kallsyms]       [k] _raw_spin_lock
     0.07%  iperf3   [nf_conntrack]          [k] _306
     0.07%  iperf3   [kernel.kallsyms]       [k] __qdisc_run
     0.07%  iperf3   [nf_conntrack]          [k] nf_ct_deliver_cached_events
     0.07%  iperf3   [kernel.kallsyms]       [k] __put_user_1
     0.07%  iperf3   [kernel.kallsyms]       [k] netif_skb_features
     0.07%  iperf3   [kernel.kallsyms]       [k] ip_local_deliver_finish
     0.07%  iperf3   [kernel.kallsyms]       [k] __do_div64
     0.06%  iperf3   [kernel.kallsyms]       [k] __usecs_to_jiffies
     0.06%  iperf3   [kernel.kallsyms]       [k] brcm_tag_rcv_ll.constprop.0
     0.06%  iperf3   [kernel.kallsyms]       [k] refcount_inc_not_zero_checked
     0.06%  iperf3   [kernel.kallsyms]       [k] vfs_read
     0.06%  iperf3   [kernel.kallsyms]       [k] ip_local_deliver
     0.06%  iperf3   [kernel.kallsyms]       [k] kfree_skb_partial
     0.06%  iperf3   [kernel.kallsyms]       [k] fq_codel_dequeue
     0.06%  iperf3   [kernel.kallsyms]       [k] _raw_spin_unlock_bh
     0.06%  iperf3   [kernel.kallsyms]       [k] memmove
     0.06%  iperf3   [kernel.kallsyms]       [k] poll_select_copy_remaining
     0.06%  iperf3   [kernel.kallsyms]       [k] __fget_light
     0.06%  iperf3   [kernel.kallsyms]       [k] fsnotify
     0.05%  iperf3   [kernel.kallsyms]       [k] refcount_sub_and_test_checked
     0.05%  iperf3   [nf_nat]                [k] nf_nat_packet
     0.05%  iperf3   [kernel.kallsyms]       [k] llist_add_batch
     0.05%  iperf3   [kernel.kallsyms]       [k] __bpf_prog_run32
     0.05%  iperf3   [kernel.kallsyms]       [k] do_softirq.part.2
     0.05%  iperf3   [kernel.kallsyms]       [k] div_s64_rem
     0.05%  iperf3   [kernel.kallsyms]       [k] ns_to_timespec
     0.05%  iperf3   [nf_nat_ipv4]           [k] _40
     0.05%  iperf3   [xt_comment]            [k] _9
     0.05%  iperf3   [kernel.kallsyms]       [k] _find_next_bit_le
     0.05%  iperf3   libc.so                 [.] 0x00056c70
     0.05%  iperf3   [kernel.kallsyms]       [k] mmiocpy
     0.05%  iperf3   [kernel.kallsyms]       [k] inet_ehashfn.constprop.4
     0.05%  iperf3   [kernel.kallsyms]       [k] tcp_v4_fill_cb
     0.05%  iperf3   [kernel.kallsyms]       [k] __netif_receive_skb
     0.05%  iperf3   [kernel.kallsyms]       [k] ip_finish_output
     0.05%  iperf3   [kernel.kallsyms]       [k] b15_dma_unmap_area
     0.05%  iperf3   [kernel.kallsyms]       [k] fput
     0.05%  iperf3   [kernel.kallsyms]       [k] tcp_cleanup_rbuf
     0.05%  iperf3   [kernel.kallsyms]       [k] skb_network_protocol
     0.05%  iperf3   [kernel.kallsyms]       [k] skb_checksum_help
     0.04%  iperf3   [kernel.kallsyms]       [k] brcm_tag_rcv
     0.04%  iperf3   [iptable_filter]        [k] _15
     0.04%  iperf3   [nf_conntrack]          [k] _341
     0.04%  iperf3   [kernel.kallsyms]       [k] _find_first_bit_le
     0.04%  iperf3   [kernel.kallsyms]       [k] kfree
     0.04%  iperf3   [kernel.kallsyms]       [k] sk_reset_timer
     0.04%  iperf3   [kernel.kallsyms]       [k] raw_local_deliver
     0.04%  iperf3   [kernel.kallsyms]       [k] tcp_add_backlog
     0.04%  iperf3   [kernel.kallsyms]       [k] ip_rcv
     0.04%  iperf3   [kernel.kallsyms]       [k] 0x004ef5c0
     0.04%  iperf3   [kernel.kallsyms]       [k] __fsnotify_parent
     0.04%  iperf3   [kernel.kallsyms]       [k] skb_push
     0.04%  iperf3   [kernel.kallsyms]       [k] __tcp_send_ack.part.6
     0.04%  iperf3   [kernel.kallsyms]       [k] skb_pull_rcsum
     0.04%  iperf3   [kernel.kallsyms]       [k] put_timespec64
     0.04%  iperf3   [kernel.kallsyms]       [k] sys_clock_gettime
     0.04%  iperf3   libc.so                 [.] 0x0005c194
     0.04%  iperf3   [kernel.kallsyms]       [k] ktime_get_real_ts64
     0.04%  iperf3   [kernel.kallsyms]       [k] inet_recvmsg
     0.04%  iperf3   [kernel.kallsyms]       [k] tcp_options_write
     0.04%  iperf3   libc.so                 [.] 0x00057850
     0.04%  iperf3   [kernel.kallsyms]       [k] tcp_filter
     0.04%  iperf3   [kernel.kallsyms]       [k] __alloc_skb
     0.04%  iperf3   [kernel.kallsyms]       [k] skb_free_head
     0.04%  iperf3   [kernel.kallsyms]       [k] arm_copy_from_user
     0.04%  iperf3   [iptable_raw]           [k] _14
     0.04%  iperf3   [kernel.kallsyms]       [k] ipv4_mtu
     0.04%  iperf3   [kernel.kallsyms]       [k] ksys_read
     0.03%  iperf3   [kernel.kallsyms]       [k] __netif_receive_skb_one_core
     0.03%  iperf3   [kernel.kallsyms]       [k] netif_receive_skb
     0.03%  iperf3   [kernel.kallsyms]       [k] rw_verify_area
     0.03%  iperf3   [kernel.kallsyms]       [k] release_sock
     0.03%  iperf3   [kernel.kallsyms]       [k] ksize
     0.03%  iperf3   [kernel.kallsyms]       [k] __pollwait
     0.03%  iperf3   [kernel.kallsyms]       [k] dql_completed
     0.03%  iperf3   iperf3                  [.] 0x00009c30
     0.03%  iperf3   [kernel.kallsyms]       [k] _test_and_set_bit
     0.03%  iperf3   [kernel.kallsyms]       [k] sock_rfree
     0.03%  iperf3   [kernel.kallsyms]       [k] posix_clock_realtime_get
     0.03%  iperf3   [nf_conntrack]          [k] _339
     0.03%  iperf3   [nf_conntrack]          [k] __nf_ct_l4proto_find
     0.03%  iperf3   [kernel.kallsyms]       [k] b15_dma_map_area
     0.03%  iperf3   [kernel.kallsyms]       [k] timespec64_add_safe
     0.03%  iperf3   [kernel.kallsyms]       [k] __sk_dst_check
     0.03%  iperf3   [kernel.kallsyms]       [k] sk_filter_trim_cap
     0.03%  iperf3   [kernel.kallsyms]       [k] ip_copy_addrs
     0.03%  iperf3   [kernel.kallsyms]       [k] dsa_slave_xmit
     0.03%  iperf3   libc.so                 [.] 0x0005e87c
     0.03%  iperf3   [kernel.kallsyms]       [k] __kfree_skb
     0.03%  iperf3   [kernel.kallsyms]       [k] kfree_skbmem
     0.03%  iperf3   [kernel.kallsyms]       [k] lock_sock_nested
     0.03%  iperf3   libc.so                 [.] 0x00052f68
     0.03%  iperf3   [kernel.kallsyms]       [k] sys_select
     0.02%  iperf3   [kernel.kallsyms]       [k] ns_to_timespec.part.0
     0.02%  iperf3   [kernel.kallsyms]       [k] refcount_dec_and_test_checked
     0.02%  iperf3   [nf_conntrack_rtcache]  [k] _21
     0.02%  iperf3   [kernel.kallsyms]       [k] ktime_get_seconds
     0.02%  iperf3   [kernel.kallsyms]       [k] skb_release_all
     0.02%  iperf3   libc.so                 [.] 0x0001d0c8
     0.02%  iperf3   [kernel.kallsyms]       [k] sock_read_iter
     0.02%  iperf3   [kernel.kallsyms]       [k] ip_rcv_finish
     0.02%  iperf3   [kernel.kallsyms]       [k] ptp_classify_raw
     0.02%  iperf3   [kernel.kallsyms]       [k] add_wait_queue
     0.02%  iperf3   [kernel.kallsyms]       [k] ip_local_out
     0.02%  iperf3   [kernel.kallsyms]       [k] iov_iter_init
     0.02%  iperf3   [kernel.kallsyms]       [k] ip_output
     0.02%  iperf3   [kernel.kallsyms]       [k] finish_task_switch
     0.02%  iperf3   [kernel.kallsyms]       [k] l2c210_clean_range
     0.02%  iperf3   [kernel.kallsyms]       [k] skb_condense
     0.02%  iperf3   [nf_nat_ipv4]           [k] _42
     0.02%  iperf3   [kernel.kallsyms]       [k] consume_skb
     0.02%  iperf3   libgcc_s.so.1           [.] 0x00006164
     0.02%  iperf3   [kernel.kallsyms]       [k] skb_csum_hwoffload_help
     0.02%  iperf3   iperf3                  [.] 0x0000de9c
     0.02%  iperf3   libc.so                 [.] 0x0005bef4
     0.02%  iperf3   [kernel.kallsyms]       [k] __aeabi_idiv
     0.02%  iperf3   [kernel.kallsyms]       [k] brcm_tag_xmit_ll
     0.02%  iperf3   libgcc_s.so.1           [.] 0x00008658
     0.02%  iperf3   [kernel.kallsyms]       [k] v7_dma_clean_range
     0.02%  iperf3   [kernel.kallsyms]       [k] 0x004f0294
     0.02%  iperf3   libc.so                 [.] 0x0005beb8
     0.02%  iperf3   [kernel.kallsyms]       [k] _raw_spin_lock_irqsave
     0.02%  iperf3   [kernel.kallsyms]       [k] netdev_pick_tx
     0.02%  iperf3   iperf3                  [.] 0x0000e04c
     0.02%  iperf3   iperf3                  [.] 0x00009bb8
     0.02%  iperf3   [kernel.kallsyms]       [k] select_estimate_accuracy
     0.02%  iperf3   [kernel.kallsyms]       [k] tcp_v4_send_check
     0.02%  iperf3   [kernel.kallsyms]       [k] kern_select
     0.02%  iperf3   iperf3                  [.] 0x00003f8c
     0.01%  iperf3   [kernel.kallsyms]       [k] set_normalized_timespec64
     0.01%  iperf3   [kernel.kallsyms]       [k] __entry_text_start
     0.01%  iperf3   [kernel.kallsyms]       [k] refcount_add_checked
     0.01%  iperf3   [nf_conntrack]          [k] _338
     0.01%  iperf3   [kernel.kallsyms]       [k] run_timer_softirq
     0.01%  iperf3   [kernel.kallsyms]       [k] _test_and_clear_bit
     0.01%  iperf3   libc.so                 [.] 0x0005bf04
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004ef5dc
     0.01%  iperf3   libc.so                 [.] 0x00056c48
     0.01%  iperf3   [kernel.kallsyms]       [k] brcm_tag_xmit
     0.01%  iperf3   [kernel.kallsyms]       [k] __page_cache_release
     0.01%  iperf3   libc.so                 [.] 0x00052ffc
     0.01%  iperf3   iperf3                  [.] 0x0000ad1c
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004ef5e0
     0.01%  iperf3   libc.so                 [.] 0x00052ff8
     0.01%  iperf3   [kernel.kallsyms]       [k] __kmalloc_reserve.constprop.22
     0.01%  iperf3   libc.so                 [.] 0x00057824
     0.01%  iperf3   [kernel.kallsyms]       [k] __ip_local_out
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004ef5e8
     0.01%  iperf3   libc.so                 [.] 0x00056864
     0.01%  iperf3   [kernel.kallsyms]       [k] ip_queue_xmit
     0.01%  iperf3   iperf3                  [.] 0x00001aec
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004f02a0
     0.01%  iperf3   [kernel.kallsyms]       [k] ip_send_check
     0.01%  iperf3   libc.so                 [.] 0x0005e8a4
     0.01%  iperf3   [kernel.kallsyms]       [k] __fdget_pos
     0.01%  iperf3   iperf3                  [.] 0x00001d50
     0.01%  iperf3   [kernel.kallsyms]       [k] poll_select_set_timeout
     0.01%  iperf3   [kernel.kallsyms]       [k] __fdget
     0.01%  iperf3   [kernel.kallsyms]       [k] tcp_release_cb
     0.01%  iperf3   libc.so                 [.] 0x00049980
     0.01%  iperf3   [kernel.kallsyms]       [k] tcp_sendmsg_locked
     0.01%  iperf3   [kernel.kallsyms]       [k] dev_queue_xmit
     0.01%  iperf3   [kernel.kallsyms]       [k] kmalloc_slab
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004ef5f0
     0.01%  iperf3   iperf3                  [.] 0x0000ad40
     0.01%  iperf3   [kernel.kallsyms]       [k] refcount_add_not_zero_checked
     0.01%  iperf3   iperf3                  [.] 0x00009bf4
     0.01%  iperf3   libgcc_s.so.1           [.] 0x000043c4
     0.01%  iperf3   iperf3                  [.] 0x0000df48
     0.01%  iperf3   iperf3                  [.] 0x0000e070
     0.01%  iperf3   iperf3                  [.] 0x00003fd4
     0.01%  iperf3   iperf3                  [.] 0x0000ad60
     0.01%  iperf3   libgcc_s.so.1           [.] 0x00008660
     0.01%  iperf3   [kernel.kallsyms]       [k] sys_read
     0.01%  iperf3   [kernel.kallsyms]       [k] poll_freewait
     0.01%  iperf3   iperf3                  [.] 0x00009b58
     0.01%  iperf3   iperf3                  [.] 0x00003f54
     0.01%  iperf3   [kernel.kallsyms]       [k] tcp_stream_memory_free
     0.01%  iperf3   [kernel.kallsyms]       [k] find_busiest_group
     0.01%  iperf3   [kernel.kallsyms]       [k] validate_xmit_skb_list
     0.01%  iperf3   libc.so                 [.] 0x0005bec8
     0.01%  iperf3   libc.so                 [.] 0x00053004
     0.01%  iperf3   [kernel.kallsyms]       [k] free_slab
     0.01%  iperf3   [kernel.kallsyms]       [k] free_unref_page
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004f02b0
     0.01%  iperf3   libc.so                 [.] 0x000577e4
     0.01%  iperf3   libc.so                 [.] 0x00053000
     0.01%  iperf3   iperf3                  [.] 0x00009acc
     0.01%  iperf3   libc.so                 [.] 0x00057800
     0.01%  iperf3   [kernel.kallsyms]       [k] free_compound_page
     0.01%  iperf3   [kernel.kallsyms]       [k] 0x004ef5f8
     0.01%  iperf3   libgcc_s.so.1           [.] 0x0000876c
     0.01%  iperf3   libc.so                 [.] 0x00053008
     0.01%  iperf3   iperf3                  [.] 0x00009a20
     0.01%  iperf3   iperf3                  [.] 0x0000ad58
     0.01%  iperf3   iperf3                  [.] 0x0000d9b0
     0.01%  iperf3   iperf3                  [.] 0x0000ad20
     0.00%  iperf3   [kernel.kallsyms]       [k] __put_page
     0.00%  iperf3   iperf3                  [.] 0x00009a10
     0.00%  iperf3   libc.so                 [.] 0x0005782c
     0.00%  iperf3   iperf3                  [.] 0x00009a58
     0.00%  iperf3   iperf3                  [.] 0x00009bc4
     0.00%  iperf3   [kernel.kallsyms]       [k] tcp_send_ack
     0.00%  iperf3   libc.so                 [.] 0x0003c69c
     0.00%  iperf3   [nf_nat_ipv4]           [k] _46
     0.00%  iperf3   libc.so                 [.] 0x0003bcec
     0.00%  iperf3   libc.so                 [.] 0x00052fb0
     0.00%  iperf3   libc.so                 [.] 0x0005bec0
     0.00%  iperf3   iperf3                  [.] 0x0000d99c
     0.00%  iperf3   iperf3                  [.] 0x00009bd8
     0.00%  iperf3   iperf3                  [.] 0x00003f60
     0.00%  iperf3   iperf3                  [.] 0x00003f3c
     0.00%  iperf3   [kernel.kallsyms]       [k] load_balance
     0.00%  iperf3   [kernel.kallsyms]       [k] mod_node_page_state
     0.00%  iperf3   iperf3                  [.] 0x0000acfc
     0.00%  iperf3   iperf3                  [.] 0x0000e068
     0.00%  iperf3   iperf3                  [.] 0x00009ab0
     0.00%  iperf3   iperf3                  [.] 0x0000df80
     0.00%  iperf3   iperf3                  [.] 0x0000dea0
     0.00%  iperf3   libc.so                 [.] 0x00052f80
     0.00%  iperf3   [kernel.kallsyms]       [k] lock_timer_base
     0.00%  iperf3   iperf3                  [.] 0x0000d9f4
     0.00%  iperf3   iperf3                  [.] 0x00009c38
     0.00%  iperf3   [kernel.kallsyms]       [k] __free_slab
     0.00%  iperf3   libc.so                 [.] 0x00057808
     0.00%  iperf3   iperf3                  [.] 0x00001cb4
     0.00%  iperf3   libc.so                 [.] 0x0005bf00
     0.00%  iperf3   libc.so                 [.] 0x00052fe0
     0.00%  iperf3   [kernel.kallsyms]       [k] free_pcp_prepare
     0.00%  iperf3   [kernel.kallsyms]       [k] tcp_ack
     0.00%  iperf3   libc.so                 [.] 0x000499a8
     0.00%  iperf3   [kernel.kallsyms]       [k] tcp_try_rmem_schedule
     0.00%  iperf3   [kernel.kallsyms]       [k] preempt_schedule_common
     0.00%  iperf3   libc.so                 [.] 0x00056c64
     0.00%  iperf3   [kernel.kallsyms]       [k] cpu_v7_bpiall_set_pte_ext
     0.00%  iperf3   iperf3                  [.] 0x00009974
     0.00%  iperf3   iperf3                  [.] 0x0000e080
     0.00%  iperf3   [xt_tcpudp]             [k] _13
     0.00%  iperf3   iperf3                  [.] 0x0000ad50
     0.00%  iperf3   iperf3                  [.] 0x0000ebc4
     0.00%  iperf3   libc.so                 [.] 0x00021094
     0.00%  iperf3   iperf3                  [.] 0x00009be0
     0.00%  iperf3   iperf3                  [.] 0x00009af4
     0.00%  iperf3   [kernel.kallsyms]       [k] do_tcp_getsockopt.constprop.9
     0.00%  iperf3   libc.so                 [.] 0x0005c1ac
     0.00%  iperf3   iperf3                  [.] 0x00001b78
     0.00%  iperf3   [kernel.kallsyms]       [k] __sock_wfree
     0.00%  iperf3   iperf3                  [.] 0x00001b58
     0.00%  iperf3   iperf3                  [.] 0x00003f90
     0.00%  iperf3   [kernel.kallsyms]       [k] percpu_counter_add_batch
     0.00%  perf     [kernel.kallsyms]       [k] perf_event_exec
Project Manager
Florian Fainelli commented on 20.03.2019 22:00

Yes so this comes down to bgmac not having receive checksum offload enabled, which means you spend a lot of time computing the packet's checksum in SW. I believe that FA/CTF might be able to help there since it should have checksum capabilities, how to support it properly is still a bit of mystery for me (it is a separate intermediate network device? an add-on to the bgmac driver?)

Vivek commented on 21.03.2019 16:56

Florian,

Thanks for the explanation. Will this issue persist even when we have a NATP accelerator implementation in place? Or - are the FA/CTF/NATP accelerator all one and the same?

@LGA1150 I'll try the software offload option.

Thanks,

Vivek

Project Manager
Florian Fainelli commented on 21.03.2019 22:48

AFAIR, the FA/NAPT should solve the RX/TX checksum problem and would speed up NAT in HW anyway.

Vivek commented on 22.03.2019 12:45

Thank you for clarifying.

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing