Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#4118 - logread crashes sometimes #9100

Open
openwrt-bot opened this issue Nov 2, 2021 · 5 comments
Open

FS#4118 - logread crashes sometimes #9100

openwrt-bot opened this issue Nov 2, 2021 · 5 comments
Labels

Comments

@openwrt-bot
Copy link

bittorf:

logread is dying several times a day here.
This is with r17950 and at least ath79 and mpc85xx and ramips.

Tue Nov 2 17:12:05 2021 kern.info kernel: [ 3745.330244] do_page_fault(): sending SIGSEGV to logread for invalid write access to 00000000 Tue Nov 2 17:12:05 2021 kern.info kernel: [ 3745.338789] epc = 77de87ac in libc.so[77d60000+a5000] Tue Nov 2 17:12:05 2021 kern.info kernel: [ 3745.343833] ra = 004017b3 in logread[400000+3000]

for checking if the log-service is healthy, we do regularly:

ubus call network.device status ubus call log read logread >/dev/null

The last call has (when failing) always RC = 139 with above message in log.

@openwrt-bot
Copy link
Author

bittorf:

the same with r18106 (after GCC11 fix).

relevant config settings are:

system.@system[0].cronloglevel='9'
system.@system[0].log_size='16'

and no memory pressure:

root@ArcherC6U-stube:~ :) free
total used free shared buff/cache available
Mem: 121680 30692 67616 3684 23372 74188
Swap: 4092 0 4092

@bittorf
Copy link
Contributor

bittorf commented Jul 31, 2022

This still happens with r20187
Here a way to reproduce that:

# ssh-session1
root@box:~ while true; do logread -l1 >/dev/null || { echo $? && break; }; done

# ssh-session2
# it works reliable while sending strings with changing length to log-system:
root@box:~ while :;do I=100;while [ $I -gt 0 ];do logger "$(printf %${I}s|tr ' ' =)";I=$((I-1));done;done

Output in ssh-session1:

Segmentation fault
139

and dmesg shows e.g.:

[13036.324969] do_page_fault(): sending SIGSEGV to logread for invalid write access to 00000000
[13036.333505] epc = 77e3b090 in libc.so[77db0000+a9000]
[13036.338585] ra  = 004017b3 in logread[400000+3000]

@bittorf
Copy link
Contributor

bittorf commented Aug 16, 2022

This is reproducible on x86_64 OpenWRT, with a snapshot
from https://downloads.openwrt.org/snapshots/targets/x86/64/

bringup the system, e.g. "OpenWrt SNAPSHOT, r20337-a66f843971"

qemu-system-x86_64 \
    -net nic -enable-kvm \
    -hda openwrt-x86-64-generic-squashfs-rootfs.img \
    -kernel openwrt-x86-64-generic-kernel.bin \
    -append "root=/dev/sda console=ttyS0"

Session1, "Segfault" will happen here:

root@OpenWrt:~ while true; do logread -l1 >/dev/null || { echo $? && break; }; done

Session2, it works reliable while sending strings with changing length to log-system:

root@OpenWrt:~ while :;do I=100;while [ $I -gt 0 ];do logger "$(printf %${I}s|tr ' ' =)";I=$((I-1));done;done

leads to:

[  350.300519] traps: logread[5222] general protection fault ip:7f9d6094a520 sp:7ffc1fcc1580 error:0 in libc.so[7f9d60907000+4b000]
Segmentation fault
139

@cagnulein
Copy link

same here unfortunately :(

@bittorf
Copy link
Contributor

bittorf commented Jan 9, 2024

still the same issue with r23293 on ath79

root@WDR4300:~ dmesg
[16921.203721] do_page_fault(): sending SIGSEGV to logread for invalid write access to 00000000
[16921.212551] epc = 77e12354 in libc.so[77d86000+ab000]
[16921.217865] ra  = 0040165b in logread[400000+3000]
[35641.028068] do_page_fault(): sending SIGSEGV to logread for invalid write access to 00000000
[35641.036709] epc = 77e22354 in libc.so[77d96000+ab000]
[35641.041973] ra  = 0040165b in logread[400000+3000]
[37501.123330] do_page_fault(): sending SIGSEGV to logread for invalid write access to 00000000
[37501.132222] epc = 77e22354 in libc.so[77d96000+ab000]
[37501.137471] ra  = 0040165b in logread[400000+3000]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants