OpenWrt/LEDE Project

  • Status Unconfirmed
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Base system
  • Assigned To No-one
  • Operating System All
  • Severity Low
  • Priority Very Low
  • Reported Version openwrt-19.07
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by higuita - 16.03.2020

FS#2904 - usb-storage fails to load in pre-init

As per forum post:

https://forum.openwrt.org/t/usb-storage-fail-to-load-in-preinit/54903

Basically, installed openwrt, added external root, all looks correctly configured, but after reboot external root fails to mount.
searching in the logs we can see that usb-storage is only loaded after pre-init, but it is listed in /etc/modprobe-boot.d/ ... checking /rom/etc/modules-boot.d/ it is not there.

Either something is missing in the https://openwrt.org/docs/guide-user/additional-software/extroot_configuration to add the usb-storage to the pre-init, or something is broken that makes pre-init fail to load usb-storage when it should.

The same setup worked fine with a older openwrt version (15.01 IIRC)

Supply the following if possible:
- Device problem occurs on
asus wl-500w

- Software versions of OpenWrt/LEDE release, packages, etc.
OpenWrt 19.07.1, r10911-c155900f66

- Steps to reproduce
Install openwrt
add external root
manually mount to confirm setup
reboot
external root not mounted nor switched to, logs show that usb-storage is only loaded after normal root is mounted
manual mount still works

 


Paul Oranje commented on 16.03.2020 09:52

So /overlay is still on internal storage.

Did you recreate the extroot from the rootfs_data ?
(assuming the external storage is a suitable filesystem on /dev/sda1)

mount /dev/sda1 /mnt
cd /mnt
tar cf - -C /overlay . |tar x

And, assuming the extroot has been added into /etc/config/fstab, did you remove the old marker from the external storage before trying to boot with extroot ?

rm /mnt/etc/.extroot-uuid
higuita commented on 16.03.2020 17:02

yes, done all those steps, i just redone then and same result
notice the dmesg:

[ 41.505538] block: extroot: cannot find device sda1
(...)
[ 42.888687] procd: - init -
(...)
[ 47.273106] usb-storage 1-2:1.0: USB Mass Storage device detected

extroot fail to work due the missing usb_storage, that is being loaded only on the init step, not in the pre-init

Paul Oranje commented on 16.03.2020 21:45

Okay, but if your analysis is correct then extroot would not run anywhere, but it does here (ipq806x, OpenWrt 19.07.2). Or d I miss something.

higuita commented on 17.03.2020 00:20

I agree with you, this is weird.
Either this is simply a documentation bug and i'm missing some step, or something that may be broken on some systems only. I'm using a asus wl-500w, so something related to the image openwrt-19.07.1-brcm47xx-legacy-asus-wl-500w ... maybe "legacy" related?

You are also using a usb-storage device, correct?
what is your output for this:

find  /rom/etc/modules-boot.d/ /rom/etc/modules.d/ /etc/modules-boot.d/ /etc/modules.d/ -name usb\*

for me, only the /etc/modules*.d/ i have the usb-storage, the /rom do not have anything for usb-storage

and how about the /etc/config/fstab? mine is this one:

config global
	option anon_swap '0'
	option anon_mount '0'
	option auto_swap '1'
	option auto_mount '1'
	option delay_root '15'
	option check_fs '0'

config mount
	option target '/mnt/sda1'
	option uuid '04f05f45-f506-4465-9ecc-8d836d0588b6'
	option enabled '0'

config mount 'rwm'
	option device '/dev/mtdblock5'
	option target '/rwm'

config mount 'overlay'
	option uuid '04f05f45-f506-4465-9ecc-8d836d0588b6'
	option target '/overlay'
Paul Oranje commented on 21.03.2020 13:26

Your has two entries for the external filesystem (same UUID). Removing the first one that's disabled may help.

Also some options seem missing. Assuming your extroot has ext4 fs, these are:

  option fstype 'ext4'
  option enabled_fsck '1'

The initially installed modules in /rom are determined by the DEVICE_PACKAGES in the makefile for the device, in case of the wl-500w those are:

  kmod-b43 kmod-usb-uhci kmod-usb2-pci

Assuming ext4 for the extroot fs, extra packages needed would be:

  kmod-usb-storage kmod-usb2 kmod-fs-ext4 block-mount

Using imagebuilder is recommended as it allows to include all needed packages in /rom which saves space in the initial overlay fs (non-extroot) as the fs for /rom (squashfs) is denser than the fs used for the internal overlay (jffs).

Good luck.

higuita commented on 03.04.2020 04:08

i have removed the first /dev/sda1 config, added the ext4 and fsck to the last entry, confirmed that i have all those packages installed... rebooted and still same result, no extroot mounted, usb_storage is only loaded in init step, after giving up of mounting extroot.

i do not know if there is some bug in pre-init scripts that fail to check the /etc/modules-boot.d/usb-storage or some magic inside kmodloader (i strace and it never even looks to the /etc/modules-boot.d, but i do not know if during boot it does different steps). Can you please check if in your system, usb_storage is loaded in pre-init?

i will probably try the imagebuilder, some years ago i failed to build a working image, but lets try again.

thanks for the help

Paul Oranje commented on 03.04.2020 11:51

To be sure:
- the contents of the internal storage mounted on /overlay has been copied to the external fs and
- a pre-existing etc/.extroot-uuid has been removed from the external fs

Once the external fs is mounted on /overlay, etc/.extroot-uuid is created on that fs and the file contains the UUID of the fs mounted on /rom. If the UUID in etc/.extroot-uuid on the external fs does not match that of the fs mounted on /rom, then the internal overlay storage is mounted.

When still having problems, please show actual contents of /etc/config/fstab, and possibly etc/.extroot-uuid on the external fs and the output of the "block info" command.

higuita commented on 04.04.2020 00:00

Yes, content was been copied in 2 different attempts, with tar and with cp (the old2 was the first attempt and inside also have another subfolder with my previous openwrt extroot, so i can copy some configs to the new setup, when it works)

no .extroot-uuid is found anywhere

root@submarino:~# df -hT
Filesystem           Type            Size      Used Available Use% Mounted on
/dev/root            squashfs        2.6M      2.6M         0 100% /rom
tmpfs                tmpfs          13.8M     60.0K     13.7M   0% /tmp
/dev/mtdblock5       jffs2           3.8M   1000.0K      2.8M  26% /overlay
overlayfs:/overlay   overlay         3.8M   1000.0K      2.8M  26% /
tmpfs                tmpfs         512.0K         0    512.0K   0% /dev
/dev/sda1            ext4          233.2M     22.7M    194.3M  10% /mnt
root@submarino:~# ls -la /mnt/ /overlay/
/mnt/:
drwxr-xr-x    5 root     root          1024 Feb  9 02:03 .
drwxr-xr-x    1 root     root             0 Mar 16  2020 ..
lrwxrwxrwx    1 root     root             1 Feb  9 02:03 .fs_state -> 2
drwxr-xr-x   10 root     root          1024 Feb  9 02:03 old2
drwxr-xr-x    7 root     root          1024 Feb  9 02:03 upper
drwxr-xr-x    3 root     root          1024 Feb  9 02:03 work

/overlay/:
drwxr-xr-x    5 root     root             0 Jan  1  1970 .
drwxr-xr-x    1 root     root             0 Mar 16  2020 ..
lrwxrwxrwx    1 root     root             1 Feb  2 23:35 .fs_state -> 2
drwxr-xr-x    7 root     root             0 Mar 16  2020 upper
drwxr-xr-x    3 root     root             0 Jan  1  1970 work
root@submarino:~# find / -name .extroot-uuid 
root@submarino:~# 
root@submarino:~# cat  /etc/config/fstab

config global
	option anon_swap '0'
	option anon_mount '0'
	option auto_swap '1'
	option auto_mount '1'
	option delay_root '15'
	option check_fs '0'

config mount 'rwm'
	option device '/dev/mtdblock5'
	option target '/rwm'

config mount 'overlay'
	option uuid '04f05f45-f506-4465-9ecc-8d836d0588b6'
	option target '/overlay'
        option fstype 'ext4'
        option enabled_fsck '1'

root@submarino:~# block info
/dev/mtdblock4: UUID="7eeba3f8-70b2ccf1-634a5d1c-24be005d" VERSION="4.0" MOUNT="/rom" TYPE="squashfs"
/dev/mtdblock5: MOUNT="/overlay" TYPE="jffs2"
/dev/sda1: UUID="04f05f45-f506-4465-9ecc-8d836d0588b6" VERSION="1.0" MOUNT="/mnt" TYPE="ext4"


dmesg: 
[    4.263493] hub 3-0:1.0: USB hub found
[    4.269334] hub 3-0:1.0: 2 ports detected
[    4.286376] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    4.305484] init: - preinit -
[    4.584140] usb 1-2: new high-speed USB device number 2 using ehci-pci
[    4.914701] random: procd: uninitialized urandom read (4 bytes read)
[    6.148297] random: jshn: uninitialized urandom read (4 bytes read)
[    6.332918] random: jshn: uninitialized urandom read (4 bytes read)
[    6.565122] random: jshn: uninitialized urandom read (4 bytes read)
[    7.195246] b44 ssb0:0 eth0: Link is up at 100 Mbps, half duplex
[    7.201361] b44 ssb0:0 eth0: Flow control is off for TX and off for RX
[    7.211354] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    7.220610] IPv6: ADDRCONF(NETDEV_UP): eth0.1: link is not ready
[    7.227698] IPv6: ADDRCONF(NETDEV_CHANGE): eth0.1: link becomes ready
[    8.243993] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[    8.251716] b44 ssb0:0 eth0: Link is Up - 100Mbps/Full - flow control off
[   10.930924] jffs2: notice: (407) jffs2_build_xattr_subsystem: complete building xattr subsystem, 49 of xdatum (3 unchecked, 46 orphan) and 107 of xref (47 dead, 0 orphan) found.
[   10.952986] mount_root: loading kmods from internal overlay
[   10.982820] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/b43.ko
[   10.991633] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/cfg80211.ko
[   11.000853] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/compat.ko
[   11.090934] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/mac80211.ko
[   11.105972] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/ppp_async.ko
[   11.115301] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/ppp_generic.ko
[   11.124812] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/pppoe.ko
[   11.133953] kmodloader: failed to open /tmp/overlay/upper/lib/modules/4.14.167/pppox.ko
[   11.235164] mount_root: failed to launch kmodloader from internal overlay
[   11.391388] block: attempting to load /tmp/jffs_cfg/upper/etc/config/fstab
[   11.403586] block: extroot: device not present, retrying in 15 seconds
[   26.480534] block: extroot: cannot find device with UUID 04f05f45-f506-4465-9ecc-8d836d0588b6
[   26.491973] mount_root: switching to jffs2 overlay
[   26.554260] overlayfs: upper fs does not support tmpfile.
[   26.572590] urandom-seed: Seeding with /etc/urandom.seed
[   26.719184] b44 ssb0:0 eth0: powering down PHY
[   26.778444] procd: - early -
[   26.781797] procd: - watchdog -
[   26.964069] b44 ssb0:0 eth0: Link is Down
[   27.644848] procd: - watchdog -
[   27.649449] procd: - ubus -
[   27.808520] random: ubusd: uninitialized urandom read (4 bytes read)
[   27.828997] random: ubusd: uninitialized urandom read (4 bytes read)
[   27.844500] random: ubusd: uninitialized urandom read (4 bytes read)
[   27.872200] procd: - init -
[   30.324110] kmodloader: loading kernel modules from /etc/modules.d/*
[   30.581077] urngd: v1.0.2 started.
[   31.283602] random: crng init done
[   31.287233] random: 3 urandom warning(s) missed due to ratelimiting
[   31.661511] SCSI subsystem initialized
[   31.838084] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   31.908246] ip_tables: (C) 2000-2006 Netfilter Core Team
[   31.947042] nf_conntrack version 0.5.0 (1024 buckets, 4096 max)
[   32.255561] usb-storage 1-2:1.0: USB Mass Storage device detected
[   32.285212] scsi host0: usb-storage 1-2:1.0
[   32.291388] usbcore: registered new interface driver usb-storage
[   32.393673] xt_time: kernel timezone is -0000
[   32.444459] kmodloader: done loading kernel modules from /etc/modules.d/*
[   33.366178] scsi 0:0:0:0: Direct-Access     VBTM     Store 'n' Go     1.04 PQ: 0 ANSI: 0 CCS
[   33.384774] sd 0:0:0:0: [sda] 501760 512-byte logical blocks: (257 MB/245 MiB)
[   33.405928] sd 0:0:0:0: [sda] Write Protect is off
[   33.410951] sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
[   33.424002] sd 0:0:0:0: [sda] No Caching mode page found
[   33.429533] sd 0:0:0:0: [sda] Assuming drive cache: write through
[   33.453193]  sda: sda1
[   33.479422] sd 0:0:0:0: [sda] Attached SCSI removable disk
[   44.944060] b44 ssb0:0 eth0: Link is up at 100 Mbps, full duplex
[   44.950178] b44 ssb0:0 eth0: Flow control is off for TX and off for RX
[   44.958684] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   44.964880] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   46.004324] b44 ssb0:0 eth0: Link is Up - 100Mbps/Full - flow control off
[   49.647074] b44 ssb0:0 eth0: powering down PHY
[   49.752742] b44 ssb0:0 eth0: Link is up at 100 Mbps, full duplex
[   49.759018] b44 ssb0:0 eth0: Flow control is off for TX and off for RX

The missing b43 and ppp modules are ok, i removed the wifi card of this router and i do not use adsl

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing