OpenWrt/LEDE Project

  • Status Closed
  • Percent Complete
    100%
  • Task Type Bug Report
  • Category Base system
  • Assigned To
    Petr Štetiar
  • Operating System All
  • Severity Medium
  • Priority Very Low
  • Reported Version openwrt-19.07
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by Peter Geis - 09.12.2019
Last edited by Petr Štetiar - 24.01.2020

FS#2666 - procd: reboot inside lxc container causes shutdown

- Device problem occurs on
armvirt64
- Software versions of OpenWrt/LEDE release, packages, etc.
openwrt-19.07-rc2
- Steps to reproduce

Create a lxc package from the openwrt-19.07-rc2 rootfs package.
Start a lxc container using the openwrt lxc package.
Initiate a `reboot` command from inside the container.

Expected result: Container reboots.
Actual result: Container halts.

Current Version openwrt-19.07-rc2.
While running in a lxc container, initiating a reboot from openwrt results in a shutdown of the container instead of rebooting the container.

This appears to have been caused by commit <832369078d818d19ab64051fdc8da9e06c90ad88> state: fix shutdown when running in a container ( FS#2425 ).
Instead of triggering reboot(reboot_event);, it detects that it is inside a container and exits pid 1, resulting in the container halting.

There is a patch at [0] to resolve this, but would likely break the original intention of the fix above.

Reference links:
https://github.com/mikma/lxd-openwrt/blob/master/patches/procd-master/0001-state-support-reboot-of-c
Closed by  Petr Štetiar
24.01.2020 07:45
Reason for closing:  Fixed
Additional comments about closing:  

Fixed in https:/ /git.openwrt.org/8038846b62edff5fbe20f6c 4cc8050b9fd401f80

Peter Geis commented on 09.12.2019 20:49

Add a link button has a maximum character limit which truncated the reference link.

Reference Link:
[0] https://github.com/mikma/lxd-openwrt/blob/master/patches/procd-master/0001-state-support-reboot-of-containers.patch

Admin
Petr Štetiar commented on 17.01.2020 19:56

Hi,

I've just tried following: 

docker run --rm -it openwrtorg/rootfs:x86-64-19.07.0 reboot

and it works as expected, it doesnt hang, so probably lxd/lxc does it differently. I don't have any prior experience with lxc/lxd, can you provide exact commands one has to run in order to reproduce it?

Admin
Petr Štetiar commented on 20.01.2020 07:31

In other words, following commands: 

docker run --env DBGLVL=5 --rm -it openwrtorg/rootfs:x86-64-19.07.0
root@4c37be61eef4:/# logread -f &

provides following debugging log output: 

Mon Jan 20 07:28:50 2020 daemon.notice procd: Triggering reboot
Mon Jan 20 07:28:50 2020 daemon.notice procd: Shutting down system with event 1234567
Mon Jan 20 07:28:50 2020 daemon.info procd: - shutdown -
Mon Jan 20 07:28:50 2020 daemon.notice procd: running /etc/rc.d/K* shutdown
Mon Jan 20 07:28:50 2020 daemon.notice procd: start /etc/rc.d/K10gpio_switch shutdown
Mon Jan 20 07:28:50 2020 daemon.notice procd: stop /etc/rc.d/K10gpio_switch shutdown - took 0.030491852s
Mon Jan 20 07:28:50 2020 daemon.notice procd: start /etc/rc.d/K50dropbear shutdown
Mon Jan 20 07:28:50 2020 daemon.notice procd: stop /etc/rc.d/K50dropbear shutdown - took 0.021862346s
Mon Jan 20 07:28:50 2020 authpriv.info dropbear[389]: Early exit: Terminated by signal
Mon Jan 20 07:28:50 2020 daemon.notice procd: start /etc/rc.d/K85odhcpd shutdown
Mon Jan 20 07:28:50 2020 daemon.notice procd: Instance dropbear::instance1 exit with error code 256 after 10 seconds
Mon Jan 20 07:28:50 2020 daemon.notice procd: Stop instance odhcpd::instance1
Mon Jan 20 07:28:50 2020 daemon.notice procd: Instance odhcpd::instance1 exit with error code 0 after 10 seconds
Mon Jan 20 07:28:50 2020 daemon.notice procd: stop /etc/rc.d/K85odhcpd shutdown - took 0.030896977s
Mon Jan 20 07:28:50 2020 daemon.notice procd: start /etc/rc.d/K89log shutdown
Mon Jan 20 07:28:50 2020 daemon.notice procd: Stop instance log::instance1

I would like to get the same logs from LXC container running under LXD.

Project Manager
Baptiste Jonglez commented on 20.01.2020 09:15

Are you running an OpenWrt LXC container on an OpenWrt host? Or an OpenWrt container on a host running another distribution? (Debian, Fedora...)

I can test OpenWrt LXC container on a x86_64 host running Debian, but I want to make sure it's really the same setup as you :)

Also, how do you prepare your image? Usually I just dump openwrt-x86-64-rootfs-ext4.img to a LVM volume.

Project Manager
Baptiste Jonglez commented on 20.01.2020 09:48

To create a container: 

cd /tmp/
wget http://downloads.openwrt.org/releases/19.07.0/targets/x86/64/openwrt-19.07.0-x86-64-generic-rootfs.tar.gz

mkdir -p /var/lib/lxc/openwrt/rootfs
touch /var/lib/lxc/openwrt/config
cd /var/lib/lxc/openwrt/rootfs
tar xvf /tmp/openwrt-19.07.0-x86-64-generic-rootfs.tar.gz

Edit `/var/lib/lxc/openwrt/config`: 

# LXC 2: use "lxc.rootfs" instead
lxc.rootfs.path = /var/lib/lxc/openwrt/rootfs

# Common configuration
lxc.include = /usr/share/lxc/config/common.conf

# LXC 2: use "lxc.utsname" instead
lxc.uts.name = openwrt
lxc.arch = amd64

Start container: 

lxc-start -n openwrt

To debug if it doesn't work, add -F.

To connect to the container: 

lxc-attach -n openwrt --clear-env /bin/ash

To look at the current container state: 

lxc-ls -f

When testing here, on a x86_64 host running Arch Linux and LXC 3.2.1, indeed a reboot causes the container to shut down.

Project Manager
Baptiste Jonglez commented on 20.01.2020 09:53

Hmm, couldn't get any useful log: 

# lxc-attach -n openwrt --clear-env --set-var DBGLVL=5 /bin/ash


BusyBox v1.30.1 () built-in shell (ash)

~ # logread  -f &
~ # reboot
~ # Mon Jan 20 09:51:32 2020 daemon.info procd: - shutdown -
Mon Jan 20 09:51:32 2020 authpriv.info dropbear[287]: Early exit: Terminated by signal
Failed to find log object: Not found
Failed to find log object: Not found
Failed to find log object: Connection failed
Failed to find log object: Invalid argument
Project Manager
Baptiste Jonglez commented on 20.01.2020 09:57

Ok, the DBGLVL=5 should go in /var/lib/lxc/openwrt/config: 

...
lxc.environment = DBGLVL=5

Then I get some debug: 

# lxc-attach -n openwrt --clear-env /bin/ash
...
~ # logread  -f &
~ # reboot
~ # Mon Jan 20 09:56:00 2020 daemon.notice procd: Triggering reboot
Mon Jan 20 09:56:00 2020 daemon.notice procd: Shutting down system with event 1234567
Mon Jan 20 09:56:00 2020 daemon.info procd: - shutdown -
Mon Jan 20 09:56:00 2020 daemon.notice procd: running /etc/rc.d/K* shutdown
Mon Jan 20 09:56:00 2020 daemon.notice procd: start /etc/rc.d/K10gpio_switch shutdown
Mon Jan 20 09:56:00 2020 daemon.notice procd: stop /etc/rc.d/K10gpio_switch shutdown - took 0.024061621s
Mon Jan 20 09:56:00 2020 daemon.notice procd: start /etc/rc.d/K50dropbear shutdown
Mon Jan 20 09:56:00 2020 authpriv.info dropbear[287]: Early exit: Terminated by signal
Mon Jan 20 09:56:00 2020 daemon.notice procd: stop /etc/rc.d/K50dropbear shutdown - took 0.015703745s
Mon Jan 20 09:56:00 2020 daemon.notice procd: Instance dropbear::instance1 exit with error code 256 after 10 seconds
Mon Jan 20 09:56:00 2020 daemon.notice procd: start /etc/rc.d/K85odhcpd shutdown
Mon Jan 20 09:56:00 2020 daemon.notice procd: Stop instance odhcpd::instance1
Mon Jan 20 09:56:00 2020 daemon.notice procd: Instance odhcpd::instance1 exit with error code 0 after 10 seconds
Mon Jan 20 09:56:00 2020 daemon.notice procd: stop /etc/rc.d/K85odhcpd shutdown - took 0.022317393s
Mon Jan 20 09:56:00 2020 daemon.notice procd: start /etc/rc.d/K89log shutdown
Mon Jan 20 09:56:00 2020 daemon.notice procd: Stop instance log::instance1
Failed to find log object: Not found
Failed to find log object: Not found
Failed to find log object: Not found
Failed to find log object: Invalid argument
Project Manager
Baptiste Jonglez commented on 20.01.2020 21:25

Petr, your patch https://gitlab.com/ynezz/openwrt-procd/commit/aa8689ccdff0124f3d477f86433496aeb2c49d24 fixes the issue, you can add my Tested-By.

I've tested with both LXC 2.0.7 on Debian and LXC 3.2.1 on Arch Linux.

Peter Geis commented on 21.01.2020 19:21

Confirmed your patch 

https://gitlab.com/ynezz/openwrt-procd/commit/aa8689ccdff0124f3d477f86433496aeb2c49d24

fixes the issue on Ubuntu 19.10 LXD 3.18.

Tested-by: Peter Geis pgwipeout@gmail.com

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing