russell:

Supply the following if possible:

• Device problem occurs on

Meraki MR24

• Software versions of OpenWrt/LEDE release, packages, etc.

reboot-10835-g0441edfb7f

• Steps to reproduce

The problem has been identified as resulting from an unpadded root.squashfs being close enough to the end of a UBI volume LEB, that a squashfs_read_data() call fails during boot, resulting in not having a rootfs, and thus a kernel panic and boot loop. Meraki MR24 uses logical eraseblock (LEB) sizes of 15.5 KiB (15872 bytes). To reproduce, add incompressible files of appropriate size to the $TOPDIR/files overlay during build such that the resulting binary's root component (discoverable by: tar tvf bin/targets/apm821xx/nand/openwrt-apm821xx-nand-meraki_mr24-squashfs-sysupgrade.bin) is close to, but less than an integer multiple of 15872 bytes).

An example failing bootlog follows:

Booting using the fdt blob at 0xc00400
Uncompressing Kernel Image ... OK
[ 0.000000] Linux version 4.19.66 (openwrt@hawg) (gcc version 7.4.0 (OpenWrt GCC 7.4.0 r10828-b8b7d4cbca)) #0 Sun Aug 18 23:02:43 2019
[ 0.000000] Using PowerPC 44x Platform machine description
[ 0.000000] -----------------------------------------------------
[ 0.000000] phys_mem_size = 0x8000000
[ 0.000000] dcache_bsize = 0x20
[ 0.000000] icache_bsize = 0x20
[ 0.000000] cpu_features = 0x0000000000000120
[ 0.000000] possible = 0x0000000040000120
[ 0.000000] always = 0x0000000000000020
[ 0.000000] cpu_user_features = 0x8c008000 0x00000000
[ 0.000000] mmu_features = 0x00000008
[ 0.000000] -----------------------------------------------------
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000000000-0x0000000007ffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000007ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000007ffffff]
[ 0.000000] MMU: Allocated 1088 bytes of context maps for 255 contexts
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32480
[ 0.000000] Kernel command line: console=ttyS0,115200 rootfstype=squashfs mtdoops.mtddev=oops
[ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.000000] Memory: 122976K/131072K available (4940K kernel code, 220K rwdata, 1140K rodata, 172K init, 232K bss, 8096K reserved, 0K cma-reserved)
[ 0.000000] Kernel virtual memory layout:
[ 0.000000] * 0xfffdf000..0xfffff000 : fixmap
[ 0.000000] * 0xfde00000..0xfe000000 : consistent mem
[ 0.000000] * 0xfde00000..0xfde00000 : early ioremap
[ 0.000000] * 0xd1000000..0xfde00000 : vmalloc & ioremap
[ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16
[ 0.000000] UIC0 (32 IRQ sources) at DCR 0xc0
[ 0.000000] UIC1 (32 IRQ sources) at DCR 0xd0
[ 0.000000] UIC2 (32 IRQ sources) at DCR 0xe0
[ 0.000000] UIC3 (32 IRQ sources) at DCR 0xf0
[ 0.000020] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0xb881274fa3, max_idle_ns: 440795210636 ns
[ 0.000035] clocksource: timebase mult[1400000] shift[24] registered
[ 0.000097] pid_max: default: 32768 minimum: 301
[ 0.000316] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.000336] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.004270] random: get_random_u32 called from bucket_table_alloc.isra.19+0x14c/0x1b0 with crng_init=0
[ 0.004894] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.004925] futex hash table entries: 256 (order: -1, 3072 bytes)
[ 0.005519] NET: Registered protocol family 16
[ 0.006758] PPC4XX OCM1: 32768 Bytes (enabled)
[ 0.006783] PPC4XX OCM1: 32768 Bytes (non-cached)
[ 0.006790] PPC4XX OCM1: 0 Bytes (cached)
[ 0.006860] debugfs ppc4xx ocm: failed to create file
[ 0.007032] 256k L2-cache enabled
[ 0.007132] PCIE0: Checking link...
[ 0.007138] PCIE0: Device detected, waiting for link...
[ 0.007144] PCIE0: link is up !
[ 0.114213] PCI host bridge /plb/pciex@d00000000 (primary) ranges:
[ 0.114251] MEM 0x0000000e00000000..0x0000000e7fffffff -> 0x0000000080000000
[ 0.114264] MEM 0x0000000f00000000..0x0000000f000fffff -> 0x0000000000000000
[ 0.114275] IO 0x0000000f80000000..0x0000000f8000ffff -> 0x0000000000000000
[ 0.114299] 4xx PCI DMA offset set to 0x00000000
[ 0.114305] 4xx PCI DMA window base to 0x0000000000000000
[ 0.114311] DMA window size 0x0000000080000000
[ 0.114337] PCIE0: successfully set as root-complex
[ 0.115045] PCI: Probing PCI hardware
[ 0.115178] PCI host bridge to bus 0000:40
[ 0.115199] pci_bus 0000:40: root bus resource [io 0x0000-0xffff]
[ 0.115213] pci_bus 0000:40: root bus resource [mem 0xe00000000-0xe7fffffff] (bus address [0x80000000-0xffffffff])
[ 0.115226] pci_bus 0000:40: root bus resource [mem 0xf00000000-0xf000fffff] (bus address [0x00000000-0x000fffff])
[ 0.115238] pci_bus 0000:40: root bus resource [bus 40-ff]
[ 0.115362] PCI: Hiding 4xx host bridge resources 0000:40:00.0
[ 0.116508] pci 0000:41:00.0: Max Payload Size set to 256 (was 128, max 256)
[ 0.116522] pci 0000:41:00.0: enabling Extended Tags
[ 0.117505] pci 0000:40:00.0: PCI bridge to [bus 41-7f]
[ 0.117552] pci 0000:41:00.0: bridge configuration invalid ([bus 02-04]), reconfiguring
[ 0.117846] pci 0000:42:02.0: Max Payload Size set to 256 (was 128, max 256)
[ 0.117859] pci 0000:42:02.0: enabling Extended Tags
[ 0.118240] pci 0000:42:03.0: Max Payload Size set to 256 (was 128, max 256)
[ 0.118253] pci 0000:42:03.0: enabling Extended Tags
[ 0.119196] pci 0000:41:00.0: PCI bridge to [bus 42-7f]
[ 0.119245] pci 0000:42:02.0: bridge configuration invalid ([bus 03-03]), reconfiguring
[ 0.119263] pci 0000:42:03.0: bridge configuration invalid ([bus 04-04]), reconfiguring
[ 0.119539] pci 0000:43:00.0: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[ 0.120564] pci 0000:42:02.0: PCI bridge to [bus 43-7f]
[ 0.120989] pci 0000:44:00.0: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[ 0.122010] pci 0000:42:03.0: PCI bridge to [bus 44-7f]
[ 0.122245] pci 0000:40:00.0: disabling bridge window [io 0x0000-0xffffffffffffffff] to [bus 41-7f] (unused)
[ 0.122273] pci 0000:40:00.0: BAR 8: assigned [mem 0xe00000000-0xe001fffff]
[ 0.122289] pci 0000:41:00.0: BAR 8: assigned [mem 0xe00000000-0xe001fffff]
[ 0.122303] pci 0000:42:02.0: BAR 8: assigned [mem 0xe00000000-0xe000fffff]
[ 0.122315] pci 0000:42:03.0: BAR 8: assigned [mem 0xe00100000-0xe001fffff]
[ 0.122330] pci 0000:43:00.0: BAR 0: assigned [mem 0xe00000000-0xe0001ffff 64bit]
[ 0.122353] pci 0000:43:00.0: BAR 6: assigned [mem 0xe00020000-0xe0002ffff pref]
[ 0.122364] pci 0000:42:02.0: PCI bridge to [bus 43]
[ 0.122378] pci 0000:42:02.0: bridge window [mem 0xe00000000-0xe000fffff]
[ 0.122396] pci 0000:44:00.0: BAR 0: assigned [mem 0xe00100000-0xe0011ffff 64bit]
[ 0.122418] pci 0000:44:00.0: BAR 6: assigned [mem 0xe00120000-0xe0012ffff pref]
[ 0.122428] pci 0000:42:03.0: PCI bridge to [bus 44]
[ 0.122439] pci 0000:42:03.0: bridge window [mem 0xe00100000-0xe001fffff]
[ 0.122453] pci 0000:41:00.0: PCI bridge to [bus 42-44]
[ 0.122465] pci 0000:41:00.0: bridge window [mem 0xe00000000-0xe001fffff]
[ 0.122479] pci 0000:40:00.0: PCI bridge to [bus 41-7f]
[ 0.122490] pci 0000:40:00.0: bridge window [mem 0xe00000000-0xe001fffff]
[ 0.133430] SCSI subsystem initialized
[ 0.135734] clocksource: Switched to clocksource timebase
[ 0.136668] NET: Registered protocol family 2
[ 0.137157] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes)
[ 0.137197] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.137215] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.137232] TCP: Hash tables configured (established 1024 bind 1024)
[ 0.137318] UDP hash table entries: 256 (order: 0, 4096 bytes)
[ 0.137340] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[ 0.137506] NET: Registered protocol family 1
[ 0.140998] dw_dmac 4bffd0800.dma: DesignWare DMA Controller, 2 channels
[ 0.146982] Crashlog allocated RAM at address 0x3f00000
[ 0.147946] workingset: timestamp_bits=14 max_order=15 bucket_order=1
[ 0.152997] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.165703] io scheduler noop registered
[ 0.165739] io scheduler deadline registered (default)
[ 0.167294] aer: probe of 0000:40:00.0:pcie002 failed with error -22
[ 0.167361] pcie_pme: probe of 0000:40:00.0:pcie001 failed with error -22
[ 0.167494] Serial: 8250/16550 driver, 2 ports, IRQ sharing enabled
[ 0.168222] console [ttyS0] disabled
[ 0.168278] 4ef600400.serial: ttyS0 at MMIO 0x4ef600400 (irq = 36, base_baud = 462962) is a TI16750
[ 0.899159] console [ttyS0] enabled
[ 0.903607] nand: device found, Manufacturer ID: 0xad, Chip ID: 0x75
[ 0.909972] nand: Hynix NAND 32MiB 3,3V 8-bit
[ 0.914318] nand: 32 MiB, SLC, erase size: 16 KiB, page size: 512, OOB size: 16
[ 0.921619] Scanning device for bad blocks
[ 1.027370] 5 fixed-partitions partitions found on MTD device 4e4000000.ndfc.nand
[ 1.034856] Creating 5 MTD partitions on "4e4000000.ndfc.nand":
[ 1.040764] 0x000000000000-0x000000150000 : "u-boot"
[ 1.046412] 0x000000150000-0x000000160000 : "u-boot-env"
[ 1.052236] 0x000000160000-0x000000170000 : "u-boot-env-redundant"
[ 1.058978] 0x000000170000-0x000000180000 : "oops"
[ 1.064281] 0x000000180000-0x000002000000 : "ubi"
[ 1.071568] libphy: Fixed MDIO Bus: probed
[ 1.075690] PPC 4xx OCP EMAC driver, version 3.54
[ 1.080688] MAL v2 /plb/mcmal, 1 TX channels, 1 RX channels
[ 1.086440] RGMII /plb/opb/emac-rgmii@ef601500 initialized with MDIO support
[ 1.093598] TAH /plb/opb/emac-tah@ef601350 initialized
[ 1.099038] /plb/opb/emac-rgmii@ef601500: input 0 in rgmii-id mode
[ 1.105339] libphy: emac_mdio: probed
[ 1.223408] eth0: EMAC-0 /plb/opb/ethernet@ef600c00, MAC 00:01:73:01:23:41
[ 1.230294] eth0: found Atheros 8035 ethernet PHY (0x01)
[ 1.235837] i2c /dev entries driver
[ 1.239651] ibm-iic 4ef600700.i2c: using standard (100 kHz) mode
[ 1.248886] booke_wdt: powerpc book-e watchdog driver loaded
[ 1.257652] NET: Registered protocol family 10
[ 1.263779] Segment Routing with IPv6
[ 1.267523] NET: Registered protocol family 17
[ 1.272016] 8021q: 802.1Q VLAN Support v1.8
[ 1.276268] drmem: No dynamic reconfiguration memory found
[ 1.283826] UBI: auto-attach mtd4
[ 1.287174] ubi0: attaching mtd4
[ 1.456177] ubi0: scanning is finished
[ 1.468674] ubi0: attached mtd4 (name "ubi", size 30 MiB)
[ 1.474102] ubi0: PEB size: 16384 bytes (16 KiB), LEB size: 15872 bytes
[ 1.480696] ubi0: min./max. I/O unit sizes: 512/512, sub-page size 256
[ 1.487202] ubi0: VID header offset: 256 (aligned 256), data offset: 512
[ 1.493880] ubi0: good PEBs: 1952, bad PEBs: 0, corrupted PEBs: 0
[ 1.499954] ubi0: user volume: 5, internal volumes: 1, max. volumes count: 92
[ 1.507068] ubi0: max/mean erase counter: 947/659, WL threshold: 4096, image sequence number: 1992864020
[ 1.516510] ubi0: available PEBs: 0, total reserved PEBs: 1952, PEBs reserved for bad PEB handling: 40
[ 1.525843] ubi0: background thread "ubi_bgt0d" started, PID 420
[ 1.532942] block ubiblock0_3: created from ubi0:3(rootfs)
[ 1.538437] ubiblock: device ubiblock0_3 (rootfs) set to be root filesystem
[ 1.545381] hctosys: unable to open rtc device (rtc0)
[ 1.552825] SQUASHFS error: squashfs_read_data failed to read block 0x62141e
[ 1.559877] squashfs: SQUASHFS error: unable to read id index table
[ 1.566454] VFS: Cannot open root device "(null)" or unknown-block(254,0): error -5
[ 1.574106] Please append a correct "root=" boot option; here are the available partitions:
[ 1.582433] 1f00 1344 mtdblock0
[ 1.582437] (driver?)
[ 1.588974] 1f01 64 mtdblock1
[ 1.588976] (driver?)
[ 1.595496] 1f02 64 mtdblock2
[ 1.595498] (driver?)
[ 1.602020] 1f03 64 mtdblock3
[ 1.602023] (driver?)
[ 1.608542] 1f04 31232 mtdblock4
[ 1.608545] (driver?)
[ 1.615065] fe00 6277 ubiblock0_3
[ 1.615067] (driver?)
[ 1.621761] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(254,0)
[ 1.631492] Rebooting in 1 seconds..


There exist patches which provide a solution to this problem by ensuring that, for Meraki MR24, the root.squashfs component has been padded to 4 kiB boundary to ensure sufficient LEBs are allocated for the ubi volume:

• [[https://patchwork.ozlabs.org/patch/1151491/|build: add squashfs padding infrastructure]]
• [[https://patchwork.ozlabs.org/patch/1151494/|amp821xx: use newly added pad-squashfs for Meraki MR24]]