New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FS#2186 - DSA driver performance issue #7023
Comments
ffainelli: Keep in mind that the stock firmware makes use of the NATP accelerator, whereas the upstream kernel or OpenWrt does not have that capability. This is somewhere on my TODO of things to play with, but I have not had the time yet. Maybe Rafal has something though I suspect he is in the same boat. If you can get a "perf record" of your iperf session on the router while doing forwarding that would be helpful to see if there are simple hotspots that could be optimized. |
npcomplete: Sure, I'll try to get the perf record from my machine. As well as from the user that has 1Gbps internet. btw, on the side note I modified the code to force enable tagging for bcm53125. Over all tagging is working great. Thanks Florian! |
LGA1150: As Florian stated above, this is not related to b53 driver. |
npcomplete: Florian, Here you go:
root@OpenWrt:~# perf report -k /tmp/vmlinux
# To display the perf.data header info, please use --header/--header-only options.
#
Failed to open /lib/libc.so, continuing without symbols
Failed to open /usr/bin/iperf3, continuing without symbols
Failed to open /lib/libgcc_s.so.1, continuing without symbols
#
# Total Lost Samples: 0
#
# Samples: 39K of event 'cycles:ppp'
# Event count (approx.): 13631283929
#
# Overhead Command Shared Object Symbol
# ........ ....... ...................... ......................................
#
18.59% iperf3 [kernel.kallsyms] [k] csum_partial
14.75% iperf3 [kernel.kallsyms] [k] __copy_to_user_std
8.05% iperf3 [kernel.kallsyms] [k] v7_dma_inv_range
7.07% iperf3 [kernel.kallsyms] [k] __irqentry_text_end
5.42% iperf3 [kernel.kallsyms] [k] l2c210_inv_range
2.43% iperf3 [kernel.kallsyms] [k] bcma_host_soc_read32
1.83% iperf3 [kernel.kallsyms] [k] tcp_rcv_established
1.72% iperf3 [kernel.kallsyms] [k] bgmac_poll
1.38% iperf3 [kernel.kallsyms] [k] copy_page_to_iter
1.38% iperf3 [kernel.kallsyms] [k] page_address
1.31% iperf3 [kernel.kallsyms] [k] skb_try_coalesce
1.21% iperf3 [kernel.kallsyms] [k] net_rx_action
1.07% iperf3 [kernel.kallsyms] [k] netdev_alloc_frag
1.05% iperf3 [ip_tables] [k] ipt_do_table
0.91% iperf3 [kernel.kallsyms] [k] tcp_v4_do_rcv
0.90% iperf3 [kernel.kallsyms] [k] enqueue_to_backlog
0.80% iperf3 [kernel.kallsyms] [k] dev_gro_receive
0.70% iperf3 [kernel.kallsyms] [k] build_skb
0.68% iperf3 [kernel.kallsyms] [k] dma_cache_maint_page
0.61% iperf3 [kernel.kallsyms] [k] __free_pages_ok
0.59% iperf3 [kernel.kallsyms] [k] __skb_get_hash
0.54% iperf3 [kernel.kallsyms] [k] ___bpf_prog_run
0.51% iperf3 [kernel.kallsyms] [k] mmioset
0.49% iperf3 [kernel.kallsyms] [k] __skb_flow_dissect
0.48% iperf3 [kernel.kallsyms] [k] kmem_cache_alloc
0.47% iperf3 [kernel.kallsyms] [k] __slab_free.constprop.2
0.43% iperf3 [kernel.kallsyms] [k] ktime_get_with_offset
0.40% iperf3 [kernel.kallsyms] [k] copyout
0.40% iperf3 [kernel.kallsyms] [k] tcp_recvmsg
0.39% iperf3 [kernel.kallsyms] [k] generic_exec_single
0.38% iperf3 [kernel.kallsyms] [k] sched_clock
0.38% iperf3 [kernel.kallsyms] [k] bcma_host_soc_write32
0.38% iperf3 [kernel.kallsyms] [k] sch_direct_xmit
0.36% iperf3 [kernel.kallsyms] [k] bgmac_dma_rx_skb_for_slot
0.36% iperf3 [kernel.kallsyms] [k] skb_release_head_state
0.36% iperf3 [kernel.kallsyms] [k] skb_copy_datagram_iter
0.35% iperf3 [kernel.kallsyms] [k] kmem_cache_free
0.35% iperf3 [kernel.kallsyms] [k] __netif_receive_skb_core
0.34% iperf3 [kernel.kallsyms] [k] __tcp_ack_snd_check
0.33% iperf3 [kernel.kallsyms] [k] netif_receive_skb_internal
0.33% iperf3 [kernel.kallsyms] [k] arm_dma_unmap_page
0.31% iperf3 [kernel.kallsyms] [k] skb_release_data
0.31% iperf3 [kernel.kallsyms] [k] tcp_event_data_recv
0.31% iperf3 [kernel.kallsyms] [k] skb_put
0.30% iperf3 [kernel.kallsyms] [k] eth_type_trans
0.30% iperf3 [kernel.kallsyms] [k] __release_sock
0.29% iperf3 [kernel.kallsyms] [k] get_rps_cpu
0.29% iperf3 [kernel.kallsyms] [k] tcp_v4_rcv
0.28% iperf3 [nf_conntrack] [k] _429
0.27% iperf3 [kernel.kallsyms] [k] __build_skb
0.27% iperf3 [kernel.kallsyms] [k] tcp_try_coalesce.part.9
0.27% iperf3 [kernel.kallsyms] [k] __dev_queue_xmit
0.25% iperf3 [kernel.kallsyms] [k] __skb_checksum
0.25% iperf3 [xt_conntrack] [k] _15
0.24% iperf3 [nf_conntrack] [k] _254
0.24% iperf3 [kernel.kallsyms] [k] __skb_checksum_complete
0.23% iperf3 [nf_conntrack] [k] nf_conntrack_in
0.23% iperf3 [kernel.kallsyms] [k] gt_sched_clock_read
0.22% iperf3 [kernel.kallsyms] [k] napi_gro_receive
0.22% iperf3 [kernel.kallsyms] [k] arm_dma_map_page
0.22% iperf3 [kernel.kallsyms] [k] tcp_queue_rcv
0.22% iperf3 [kernel.kallsyms] [k] tcp_send_delayed_ack
0.22% iperf3 [kernel.kallsyms] [k] bcma_bgmac_read
0.22% iperf3 [kernel.kallsyms] [k] nf_hook_slow
0.22% iperf3 [kernel.kallsyms] [k] __tcp_select_window
0.21% iperf3 [kernel.kallsyms] [k] __dma_page_dev_to_cpu
0.21% iperf3 [kernel.kallsyms] [k] gic_raise_softirq
0.21% iperf3 [kernel.kallsyms] [k] ipv4_dst_check
0.20% iperf3 [kernel.kallsyms] [k] __local_bh_enable_ip
0.20% iperf3 [kernel.kallsyms] [k] __dma_page_cpu_to_dev
0.19% iperf3 [kernel.kallsyms] [k] smp_cross_call
0.18% iperf3 [kernel.kallsyms] [k] bcma_bgmac_write
0.18% iperf3 [kernel.kallsyms] [k] process_backlog
0.18% iperf3 [kernel.kallsyms] [k] smp_call_function_single_async
0.18% iperf3 [kernel.kallsyms] [k] __kunmap_atomic
0.18% iperf3 [kernel.kallsyms] [k] _copy_to_iter
0.16% iperf3 [kernel.kallsyms] [k] dsa_switch_rcv
0.15% iperf3 [kernel.kallsyms] [k] arm_heavy_mb
0.15% iperf3 [kernel.kallsyms] [k] tcp_rcv_space_adjust
0.15% iperf3 [kernel.kallsyms] [k] tcp_poll
0.14% iperf3 [kernel.kallsyms] [k] do_select
0.13% iperf3 [nf_nat] [k] nf_nat_inet_fn
0.13% iperf3 [kernel.kallsyms] [k] __tcp_transmit_skb
0.13% iperf3 [kernel.kallsyms] [k] skb_gro_reset_offset
0.13% iperf3 [kernel.kallsyms] [k] net_rps_send_ipi
0.13% iperf3 [kernel.kallsyms] [k] skb_checksum
0.12% iperf3 [kernel.kallsyms] [k] __kfree_skb_flush
0.12% iperf3 [kernel.kallsyms] [k] kmap_atomic
0.12% iperf3 [kernel.kallsyms] [k] napi_complete_done
0.12% iperf3 [kernel.kallsyms] [k] sock_def_readable
0.12% iperf3 [kernel.kallsyms] [k] csum_partial_ext
0.12% iperf3 [kernel.kallsyms] [k] skb_pull
0.12% iperf3 [kernel.kallsyms] [k] sock_poll
0.12% iperf3 [kernel.kallsyms] [k] tcp_grow_window
0.11% iperf3 [kernel.kallsyms] [k] arch_send_call_function_single_ipi
0.11% iperf3 [nf_conntrack] [k] _422
0.11% iperf3 [kernel.kallsyms] [k] mod_timer
0.10% iperf3 [kernel.kallsyms] [k] rcu_bh_qs
0.10% iperf3 [kernel.kallsyms] [k] _cond_resched
0.10% iperf3 [kernel.kallsyms] [k] gt_clocksource_read
0.10% iperf3 [kernel.kallsyms] [k] __vfs_read
0.10% iperf3 [kernel.kallsyms] [k] cpumask_next
0.10% iperf3 [iptable_mangle] [k] _10
0.09% iperf3 [kernel.kallsyms] [k] rcu_all_qs
0.09% iperf3 [nf_nat_ipv4] [k] _39
0.09% iperf3 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
0.09% iperf3 [nf_defrag_ipv4] [k] _13
0.09% iperf3 [kernel.kallsyms] [k] ktime_get_ts64
0.09% iperf3 [kernel.kallsyms] [k] l2c210_sync
0.09% iperf3 [kernel.kallsyms] [k] arm_dma_mapping_error
0.09% iperf3 [nf_conntrack] [k] _389
0.09% iperf3 [kernel.kallsyms] [k] bgmac_start_xmit
0.09% iperf3 [kernel.kallsyms] [k] __kmalloc_track_caller
0.09% iperf3 [kernel.kallsyms] [k] dev_hard_start_xmit
0.09% iperf3 [kernel.kallsyms] [k] validate_xmit_skb.constprop.28
0.08% iperf3 [kernel.kallsyms] [k] ip_rcv_core.constprop.2
0.08% iperf3 [nf_conntrack] [k] _308
0.08% iperf3 [kernel.kallsyms] [k] dst_release
0.08% iperf3 [kernel.kallsyms] [k] ip_finish_output2
0.08% iperf3 [kernel.kallsyms] [k] ip_rcv_finish_core.constprop.3
0.08% iperf3 [kernel.kallsyms] [k] __inet_lookup_established
0.08% iperf3 [kernel.kallsyms] [k] tcp_v4_early_demux
0.08% iperf3 [kernel.kallsyms] [k] _raw_spin_lock_bh
0.08% iperf3 [kernel.kallsyms] [k] core_sys_select
0.08% iperf3 [nf_conntrack] [k] __nf_ct_refresh_acct
0.08% iperf3 [kernel.kallsyms] [k] page_frag_free
0.07% iperf3 [nf_conntrack] [k] _337
0.07% iperf3 [kernel.kallsyms] [k] __ip_queue_xmit
0.07% iperf3 [kernel.kallsyms] [k] should_failslab
0.07% iperf3 [xt_conntrack] [k] _23
0.07% iperf3 [kernel.kallsyms] [k] vector_swi
0.07% iperf3 [kernel.kallsyms] [k] put_cpu_partial
0.07% iperf3 [kernel.kallsyms] [k] _raw_spin_lock
0.07% iperf3 [nf_conntrack] [k] _306
0.07% iperf3 [kernel.kallsyms] [k] __qdisc_run
0.07% iperf3 [nf_conntrack] [k] nf_ct_deliver_cached_events
0.07% iperf3 [kernel.kallsyms] [k] __put_user_1
0.07% iperf3 [kernel.kallsyms] [k] netif_skb_features
0.07% iperf3 [kernel.kallsyms] [k] ip_local_deliver_finish
0.07% iperf3 [kernel.kallsyms] [k] __do_div64
0.06% iperf3 [kernel.kallsyms] [k] __usecs_to_jiffies
0.06% iperf3 [kernel.kallsyms] [k] brcm_tag_rcv_ll.constprop.0
0.06% iperf3 [kernel.kallsyms] [k] refcount_inc_not_zero_checked
0.06% iperf3 [kernel.kallsyms] [k] vfs_read
0.06% iperf3 [kernel.kallsyms] [k] ip_local_deliver
0.06% iperf3 [kernel.kallsyms] [k] kfree_skb_partial
0.06% iperf3 [kernel.kallsyms] [k] fq_codel_dequeue
0.06% iperf3 [kernel.kallsyms] [k] _raw_spin_unlock_bh
0.06% iperf3 [kernel.kallsyms] [k] memmove
0.06% iperf3 [kernel.kallsyms] [k] poll_select_copy_remaining
0.06% iperf3 [kernel.kallsyms] [k] __fget_light
0.06% iperf3 [kernel.kallsyms] [k] fsnotify
0.05% iperf3 [kernel.kallsyms] [k] refcount_sub_and_test_checked
0.05% iperf3 [nf_nat] [k] nf_nat_packet
0.05% iperf3 [kernel.kallsyms] [k] llist_add_batch
0.05% iperf3 [kernel.kallsyms] [k] __bpf_prog_run32
0.05% iperf3 [kernel.kallsyms] [k] do_softirq.part.2
0.05% iperf3 [kernel.kallsyms] [k] div_s64_rem
0.05% iperf3 [kernel.kallsyms] [k] ns_to_timespec
0.05% iperf3 [nf_nat_ipv4] [k] _40
0.05% iperf3 [xt_comment] [k] _9
0.05% iperf3 [kernel.kallsyms] [k] _find_next_bit_le
0.05% iperf3 libc.so [.] 0x00056c70
0.05% iperf3 [kernel.kallsyms] [k] mmiocpy
0.05% iperf3 [kernel.kallsyms] [k] inet_ehashfn.constprop.4
0.05% iperf3 [kernel.kallsyms] [k] tcp_v4_fill_cb
0.05% iperf3 [kernel.kallsyms] [k] __netif_receive_skb
0.05% iperf3 [kernel.kallsyms] [k] ip_finish_output
0.05% iperf3 [kernel.kallsyms] [k] b15_dma_unmap_area
0.05% iperf3 [kernel.kallsyms] [k] fput
0.05% iperf3 [kernel.kallsyms] [k] tcp_cleanup_rbuf
0.05% iperf3 [kernel.kallsyms] [k] skb_network_protocol
0.05% iperf3 [kernel.kallsyms] [k] skb_checksum_help
0.04% iperf3 [kernel.kallsyms] [k] brcm_tag_rcv
0.04% iperf3 [iptable_filter] [k] _15
0.04% iperf3 [nf_conntrack] [k] _341
0.04% iperf3 [kernel.kallsyms] [k] _find_first_bit_le
0.04% iperf3 [kernel.kallsyms] [k] kfree
0.04% iperf3 [kernel.kallsyms] [k] sk_reset_timer
0.04% iperf3 [kernel.kallsyms] [k] raw_local_deliver
0.04% iperf3 [kernel.kallsyms] [k] tcp_add_backlog
0.04% iperf3 [kernel.kallsyms] [k] ip_rcv
0.04% iperf3 [kernel.kallsyms] [k] 0x004ef5c0
0.04% iperf3 [kernel.kallsyms] [k] __fsnotify_parent
0.04% iperf3 [kernel.kallsyms] [k] skb_push
0.04% iperf3 [kernel.kallsyms] [k] __tcp_send_ack.part.6
0.04% iperf3 [kernel.kallsyms] [k] skb_pull_rcsum
0.04% iperf3 [kernel.kallsyms] [k] put_timespec64
0.04% iperf3 [kernel.kallsyms] [k] sys_clock_gettime
0.04% iperf3 libc.so [.] 0x0005c194
0.04% iperf3 [kernel.kallsyms] [k] ktime_get_real_ts64
0.04% iperf3 [kernel.kallsyms] [k] inet_recvmsg
0.04% iperf3 [kernel.kallsyms] [k] tcp_options_write
0.04% iperf3 libc.so [.] 0x00057850
0.04% iperf3 [kernel.kallsyms] [k] tcp_filter
0.04% iperf3 [kernel.kallsyms] [k] __alloc_skb
0.04% iperf3 [kernel.kallsyms] [k] skb_free_head
0.04% iperf3 [kernel.kallsyms] [k] arm_copy_from_user
0.04% iperf3 [iptable_raw] [k] _14
0.04% iperf3 [kernel.kallsyms] [k] ipv4_mtu
0.04% iperf3 [kernel.kallsyms] [k] ksys_read
0.03% iperf3 [kernel.kallsyms] [k] __netif_receive_skb_one_core
0.03% iperf3 [kernel.kallsyms] [k] netif_receive_skb
0.03% iperf3 [kernel.kallsyms] [k] rw_verify_area
0.03% iperf3 [kernel.kallsyms] [k] release_sock
0.03% iperf3 [kernel.kallsyms] [k] ksize
0.03% iperf3 [kernel.kallsyms] [k] __pollwait
0.03% iperf3 [kernel.kallsyms] [k] dql_completed
0.03% iperf3 iperf3 [.] 0x00009c30
0.03% iperf3 [kernel.kallsyms] [k] _test_and_set_bit
0.03% iperf3 [kernel.kallsyms] [k] sock_rfree
0.03% iperf3 [kernel.kallsyms] [k] posix_clock_realtime_get
0.03% iperf3 [nf_conntrack] [k] _339
0.03% iperf3 [nf_conntrack] [k] __nf_ct_l4proto_find
0.03% iperf3 [kernel.kallsyms] [k] b15_dma_map_area
0.03% iperf3 [kernel.kallsyms] [k] timespec64_add_safe
0.03% iperf3 [kernel.kallsyms] [k] __sk_dst_check
0.03% iperf3 [kernel.kallsyms] [k] sk_filter_trim_cap
0.03% iperf3 [kernel.kallsyms] [k] ip_copy_addrs
0.03% iperf3 [kernel.kallsyms] [k] dsa_slave_xmit
0.03% iperf3 libc.so [.] 0x0005e87c
0.03% iperf3 [kernel.kallsyms] [k] __kfree_skb
0.03% iperf3 [kernel.kallsyms] [k] kfree_skbmem
0.03% iperf3 [kernel.kallsyms] [k] lock_sock_nested
0.03% iperf3 libc.so [.] 0x00052f68
0.03% iperf3 [kernel.kallsyms] [k] sys_select
0.02% iperf3 [kernel.kallsyms] [k] ns_to_timespec.part.0
0.02% iperf3 [kernel.kallsyms] [k] refcount_dec_and_test_checked
0.02% iperf3 [nf_conntrack_rtcache] [k] _21
0.02% iperf3 [kernel.kallsyms] [k] ktime_get_seconds
0.02% iperf3 [kernel.kallsyms] [k] skb_release_all
0.02% iperf3 libc.so [.] 0x0001d0c8
0.02% iperf3 [kernel.kallsyms] [k] sock_read_iter
0.02% iperf3 [kernel.kallsyms] [k] ip_rcv_finish
0.02% iperf3 [kernel.kallsyms] [k] ptp_classify_raw
0.02% iperf3 [kernel.kallsyms] [k] add_wait_queue
0.02% iperf3 [kernel.kallsyms] [k] ip_local_out
0.02% iperf3 [kernel.kallsyms] [k] iov_iter_init
0.02% iperf3 [kernel.kallsyms] [k] ip_output
0.02% iperf3 [kernel.kallsyms] [k] finish_task_switch
0.02% iperf3 [kernel.kallsyms] [k] l2c210_clean_range
0.02% iperf3 [kernel.kallsyms] [k] skb_condense
0.02% iperf3 [nf_nat_ipv4] [k] _42
0.02% iperf3 [kernel.kallsyms] [k] consume_skb
0.02% iperf3 libgcc_s.so.1 [.] 0x00006164
0.02% iperf3 [kernel.kallsyms] [k] skb_csum_hwoffload_help
0.02% iperf3 iperf3 [.] 0x0000de9c
0.02% iperf3 libc.so [.] 0x0005bef4
0.02% iperf3 [kernel.kallsyms] [k] __aeabi_idiv
0.02% iperf3 [kernel.kallsyms] [k] brcm_tag_xmit_ll
0.02% iperf3 libgcc_s.so.1 [.] 0x00008658
0.02% iperf3 [kernel.kallsyms] [k] v7_dma_clean_range
0.02% iperf3 [kernel.kallsyms] [k] 0x004f0294
0.02% iperf3 libc.so [.] 0x0005beb8
0.02% iperf3 [kernel.kallsyms] [k] _raw_spin_lock_irqsave
0.02% iperf3 [kernel.kallsyms] [k] netdev_pick_tx
0.02% iperf3 iperf3 [.] 0x0000e04c
0.02% iperf3 iperf3 [.] 0x00009bb8
0.02% iperf3 [kernel.kallsyms] [k] select_estimate_accuracy
0.02% iperf3 [kernel.kallsyms] [k] tcp_v4_send_check
0.02% iperf3 [kernel.kallsyms] [k] kern_select
0.02% iperf3 iperf3 [.] 0x00003f8c
0.01% iperf3 [kernel.kallsyms] [k] set_normalized_timespec64
0.01% iperf3 [kernel.kallsyms] [k] __entry_text_start
0.01% iperf3 [kernel.kallsyms] [k] refcount_add_checked
0.01% iperf3 [nf_conntrack] [k] _338
0.01% iperf3 [kernel.kallsyms] [k] run_timer_softirq
0.01% iperf3 [kernel.kallsyms] [k] _test_and_clear_bit
0.01% iperf3 libc.so [.] 0x0005bf04
0.01% iperf3 [kernel.kallsyms] [k] 0x004ef5dc
0.01% iperf3 libc.so [.] 0x00056c48
0.01% iperf3 [kernel.kallsyms] [k] brcm_tag_xmit
0.01% iperf3 [kernel.kallsyms] [k] __page_cache_release
0.01% iperf3 libc.so [.] 0x00052ffc
0.01% iperf3 iperf3 [.] 0x0000ad1c
0.01% iperf3 [kernel.kallsyms] [k] 0x004ef5e0
0.01% iperf3 libc.so [.] 0x00052ff8
0.01% iperf3 [kernel.kallsyms] [k] __kmalloc_reserve.constprop.22
0.01% iperf3 libc.so [.] 0x00057824
0.01% iperf3 [kernel.kallsyms] [k] __ip_local_out
0.01% iperf3 [kernel.kallsyms] [k] 0x004ef5e8
0.01% iperf3 libc.so [.] 0x00056864
0.01% iperf3 [kernel.kallsyms] [k] ip_queue_xmit
0.01% iperf3 iperf3 [.] 0x00001aec
0.01% iperf3 [kernel.kallsyms] [k] 0x004f02a0
0.01% iperf3 [kernel.kallsyms] [k] ip_send_check
0.01% iperf3 libc.so [.] 0x0005e8a4
0.01% iperf3 [kernel.kallsyms] [k] __fdget_pos
0.01% iperf3 iperf3 [.] 0x00001d50
0.01% iperf3 [kernel.kallsyms] [k] poll_select_set_timeout
0.01% iperf3 [kernel.kallsyms] [k] __fdget
0.01% iperf3 [kernel.kallsyms] [k] tcp_release_cb
0.01% iperf3 libc.so [.] 0x00049980
0.01% iperf3 [kernel.kallsyms] [k] tcp_sendmsg_locked
0.01% iperf3 [kernel.kallsyms] [k] dev_queue_xmit
0.01% iperf3 [kernel.kallsyms] [k] kmalloc_slab
0.01% iperf3 [kernel.kallsyms] [k] 0x004ef5f0
0.01% iperf3 iperf3 [.] 0x0000ad40
0.01% iperf3 [kernel.kallsyms] [k] refcount_add_not_zero_checked
0.01% iperf3 iperf3 [.] 0x00009bf4
0.01% iperf3 libgcc_s.so.1 [.] 0x000043c4
0.01% iperf3 iperf3 [.] 0x0000df48
0.01% iperf3 iperf3 [.] 0x0000e070
0.01% iperf3 iperf3 [.] 0x00003fd4
0.01% iperf3 iperf3 [.] 0x0000ad60
0.01% iperf3 libgcc_s.so.1 [.] 0x00008660
0.01% iperf3 [kernel.kallsyms] [k] sys_read
0.01% iperf3 [kernel.kallsyms] [k] poll_freewait
0.01% iperf3 iperf3 [.] 0x00009b58
0.01% iperf3 iperf3 [.] 0x00003f54
0.01% iperf3 [kernel.kallsyms] [k] tcp_stream_memory_free
0.01% iperf3 [kernel.kallsyms] [k] find_busiest_group
0.01% iperf3 [kernel.kallsyms] [k] validate_xmit_skb_list
0.01% iperf3 libc.so [.] 0x0005bec8
0.01% iperf3 libc.so [.] 0x00053004
0.01% iperf3 [kernel.kallsyms] [k] free_slab
0.01% iperf3 [kernel.kallsyms] [k] free_unref_page
0.01% iperf3 [kernel.kallsyms] [k] 0x004f02b0
0.01% iperf3 libc.so [.] 0x000577e4
0.01% iperf3 libc.so [.] 0x00053000
0.01% iperf3 iperf3 [.] 0x00009acc
0.01% iperf3 libc.so [.] 0x00057800
0.01% iperf3 [kernel.kallsyms] [k] free_compound_page
0.01% iperf3 [kernel.kallsyms] [k] 0x004ef5f8
0.01% iperf3 libgcc_s.so.1 [.] 0x0000876c
0.01% iperf3 libc.so [.] 0x00053008
0.01% iperf3 iperf3 [.] 0x00009a20
0.01% iperf3 iperf3 [.] 0x0000ad58
0.01% iperf3 iperf3 [.] 0x0000d9b0
0.01% iperf3 iperf3 [.] 0x0000ad20
0.00% iperf3 [kernel.kallsyms] [k] __put_page
0.00% iperf3 iperf3 [.] 0x00009a10
0.00% iperf3 libc.so [.] 0x0005782c
0.00% iperf3 iperf3 [.] 0x00009a58
0.00% iperf3 iperf3 [.] 0x00009bc4
0.00% iperf3 [kernel.kallsyms] [k] tcp_send_ack
0.00% iperf3 libc.so [.] 0x0003c69c
0.00% iperf3 [nf_nat_ipv4] [k] _46
0.00% iperf3 libc.so [.] 0x0003bcec
0.00% iperf3 libc.so [.] 0x00052fb0
0.00% iperf3 libc.so [.] 0x0005bec0
0.00% iperf3 iperf3 [.] 0x0000d99c
0.00% iperf3 iperf3 [.] 0x00009bd8
0.00% iperf3 iperf3 [.] 0x00003f60
0.00% iperf3 iperf3 [.] 0x00003f3c
0.00% iperf3 [kernel.kallsyms] [k] load_balance
0.00% iperf3 [kernel.kallsyms] [k] mod_node_page_state
0.00% iperf3 iperf3 [.] 0x0000acfc
0.00% iperf3 iperf3 [.] 0x0000e068
0.00% iperf3 iperf3 [.] 0x00009ab0
0.00% iperf3 iperf3 [.] 0x0000df80
0.00% iperf3 iperf3 [.] 0x0000dea0
0.00% iperf3 libc.so [.] 0x00052f80
0.00% iperf3 [kernel.kallsyms] [k] lock_timer_base
0.00% iperf3 iperf3 [.] 0x0000d9f4
0.00% iperf3 iperf3 [.] 0x00009c38
0.00% iperf3 [kernel.kallsyms] [k] __free_slab
0.00% iperf3 libc.so [.] 0x00057808
0.00% iperf3 iperf3 [.] 0x00001cb4
0.00% iperf3 libc.so [.] 0x0005bf00
0.00% iperf3 libc.so [.] 0x00052fe0
0.00% iperf3 [kernel.kallsyms] [k] free_pcp_prepare
0.00% iperf3 [kernel.kallsyms] [k] tcp_ack
0.00% iperf3 libc.so [.] 0x000499a8
0.00% iperf3 [kernel.kallsyms] [k] tcp_try_rmem_schedule
0.00% iperf3 [kernel.kallsyms] [k] preempt_schedule_common
0.00% iperf3 libc.so [.] 0x00056c64
0.00% iperf3 [kernel.kallsyms] [k] cpu_v7_bpiall_set_pte_ext
0.00% iperf3 iperf3 [.] 0x00009974
0.00% iperf3 iperf3 [.] 0x0000e080
0.00% iperf3 [xt_tcpudp] [k] _13
0.00% iperf3 iperf3 [.] 0x0000ad50
0.00% iperf3 iperf3 [.] 0x0000ebc4
0.00% iperf3 libc.so [.] 0x00021094
0.00% iperf3 iperf3 [.] 0x00009be0
0.00% iperf3 iperf3 [.] 0x00009af4
0.00% iperf3 [kernel.kallsyms] [k] do_tcp_getsockopt.constprop.9
0.00% iperf3 libc.so [.] 0x0005c1ac
0.00% iperf3 iperf3 [.] 0x00001b78
0.00% iperf3 [kernel.kallsyms] [k] __sock_wfree
0.00% iperf3 iperf3 [.] 0x00001b58
0.00% iperf3 iperf3 [.] 0x00003f90
0.00% iperf3 [kernel.kallsyms] [k] percpu_counter_add_batch
0.00% perf [kernel.kallsyms] [k] perf_event_exec
|
ffainelli: Yes so this comes down to bgmac not having receive checksum offload enabled, which means you spend a lot of time computing the packet's checksum in SW. I believe that FA/CTF might be able to help there since it should have checksum capabilities, how to support it properly is still a bit of mystery for me (it is a separate intermediate network device? an add-on to the bgmac driver?) |
npcomplete: Florian, Thanks for the explanation. Will this issue persist even when we have a NATP accelerator implementation in place? Or - are the FA/CTF/NATP accelerator all one and the same? @LGA1150 I'll try the software offload option. Thanks, Vivek |
ffainelli: AFAIR, the FA/NAPT should solve the RX/TX checksum problem and would speed up NAT in HW anyway. |
npcomplete: Thank you for clarifying. |
npcomplete: Florian, After reading through the GPL source, it seems that FA is not enabled for this router (i.e it maybe used in bypass mode) when GMAC3 is enabled. I found this declaration:
The decompressed factory firmware only contains ctf.ko but no FA related modules. Also, the ctf_fa_mode nvram parameter is missing for this router. Thanks, |
a_guy: Hello team. Any updates on this? This bug affects all Broadcom-based routers from bcm53xx target (at least) and makes OpenWrt unusable for 500Mbps+ connections. Example: https://forum.openwrt.org/t/nighthawk-r8000-throughput-speed-issues/66740/3 I'm ready to perform necessary tests if needed. |
npcomplete:
Device: Linksys EA9500
Trunk: Kernel v4.19.25
I've compiled my own kernel and used DSA driver instead of swconfig
one for this router. Note: Kernel v4.19 enables broadcom tags for
B53 family of switches in this DSA driver
Here are some of my observations:
i. My internet is limited to 200Mbps, please see by test
setup below
ii. One of the users who has 1GBps internet had initially
reported this issue
I've tried all the lan ports but the result is same. Note that this
router has 8 lan ports and 1 wan port. Where, switch bcm53012
covers lan4, lan7, lan8 and wan ports. While bcm53125 covers lan1,
lan2, lan3, lan5 and lan6 ports. bcm53125's cpu port(8) hangs of
bcm53012 at port(0). bcm53012 is connected to GMAC.
When I load factory firmware, I do get 1Gbps from lan to wan port
Test setup:
Router <=> pfsense box <=> Modem
||
SSD
File copy is limited to ~500Mbps
Thanks,
Vivek
The text was updated successfully, but these errors were encountered: