OpenWrt/LEDE Project

  • Status New
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Base system
  • Assigned To No-one
  • Operating System All
  • Severity Medium
  • Priority Medium
  • Reported Version All
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by Baptiste Jonglez - 21.12.2019

FS#2690 - Imagebuilder: "opkg_install_pkg: Package size mismatch" error

The Imagebuilder fails in the following situation:

  • download and unpack the imagebuilder
  • build an image: it works
  • wait a few days/weeks
  • try to build an image again with the same imagebuilder: it fails

It fails quite late during building with lots of errors like this:

   opkg_install_pkg: Package size mismatch: libubox is 19365 bytes, expecting 19361 bytes
   opkg_install_cmd: Cannot install package luci.
   opkg_install_pkg: Package size mismatch: libuci is 15454 bytes, expecting 15461 bytes
   opkg_install_cmd: Cannot install package base-files.
   opkg_install_pkg: Package size mismatch: busybox is 212951 bytes, expecting 212973 bytes
   opkg_install_cmd: Cannot install package busybox.

This happens with stable releases (18.06.5), RC releases (19.07.0-rc2), and with several targets.

It looks like package sizes are somehow kept in a local cache during the first build, and are not updated when building again a few days later. This is strange because the imagebuilder seems to fetch new versions of the Packages.gz files.

Below is the full log.

  $ make image PROFILE=rb-nor-flash-16M"" BIN_DIR="${IMAGEBUILDER_DIR}" EXTRA_IMAGE_NAME="rzn" FILES="files/" PACKAGES="luci luci-mod-admin-full luci-i18n-base-en luci-i18n-base-fr luci-i18n-firewall-fr -ppp -ppp-mod-pppoe -kmod-ppp -kmod-pppoe -kmod-pppox"
  
  Building images for ar71xx - MikroTik RouterBoard (16 MB SPI NOR)
  Packages: luci luci-mod-admin-full luci-i18n-base-en luci-i18n-base-fr luci-i18n-firewall-fr base-files busybox dnsmasq dropbear firewall fstools ip6tables iptables kernel kmod-ath9k kmod-gpio-button-hotplug kmod-ledtrig-gpio libc libgcc logd mtd netifd odhcp6c odhcpd-ipv6only opkg rbcfg rssileds swconfig uboot-envtools uci uclient-fetch wpad-mini
  
  Downloading http://downloads.openwrt.org/releases/18.06.5/targets/ar71xx/mikrotik/packages/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_core
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/base/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_base
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_luci
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/packages/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_packages
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/routing/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_routing
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/telephony/Packages.gz
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/openwrt_telephony
  Downloading file:packages/Packages
  Updated list of available packages in /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/build_dir/target-mips_24kc_musl/root-ar71xx/../../../../../../../../../home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/imagebuilder
  
  Installing packages...
  Installing libc (1.1.19-2) to root...
  Installing libgcc (7.3.0-2) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libgcc_7.3.0-2_mips_24kc.ipk.
  Configuring libgcc.
  Configuring libc.
  Installing kernel (4.9.198-1-3ba2e8c5db2fc4389b537731cf37120c) to root...
  Configuring kernel.
  Installing luci (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci_git-19.353.67991-868abc1-1_all.ipk
  Installing libubox (2018-07-25-c83a84af-2) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libubox_2018-07-25-c83a84af-2_mips_24kc.ipk.
  Installing luci-mod-admin-full (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-mod-admin-full_git-19.353.67991-868abc1-1_mips_24kc.ipk
  Installing luci-i18n-base-en (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-i18n-base-en_git-19.353.67991-868abc1-1_all.ipk
  Installing luci-base (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-base_git-19.353.67991-868abc1-1_mips_24kc.ipk
  Installing luci-i18n-base-fr (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-i18n-base-fr_git-19.353.67991-868abc1-1_all.ipk
  Installing luci-i18n-firewall-fr (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-i18n-firewall-fr_git-19.353.67991-868abc1-1_all.ipk
  Installing luci-app-firewall (git-19.353.67991-868abc1-1) to root...
  Downloading http://downloads.openwrt.org/releases/18.06.5/packages/mips_24kc/luci/luci-app-firewall_git-19.353.67991-868abc1-1_all.ipk
  Installing base-files (194.2-r7897-9d401013fc) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/base-files_194.2-r7897-9d401013fc_mips_24kc.ipk.
  Installing libuci (2019-05-17-f199b961-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libuci_2019-05-17-f199b961-1_mips_24kc.ipk.
  Installing busybox (1.28.4-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/busybox_1.28.4-3_mips_24kc.ipk.
  Installing dnsmasq (2.80-1.4) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/dnsmasq_2.80-1.4_mips_24kc.ipk.
  Installing dropbear (2017.75-7.1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/dropbear_2017.75-7.1_mips_24kc.ipk.
  Installing firewall (2018-08-13-1c4d5bcd-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/firewall_2018-08-13-1c4d5bcd-1_mips_24kc.ipk.
  Installing fstools (2019-03-28-ff1ded63-5) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/fstools_2019-03-28-ff1ded63-5_mips_24kc.ipk.
  Installing ip6tables (1.6.2-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/ip6tables_1.6.2-3_mips_24kc.ipk.
  Installing kmod-nf-reject6 (4.9.198-1) to root...
  Downloading file:packages/kmod-nf-reject6_4.9.198-1_mips_24kc.ipk
  Installing kmod-nf-ipt (4.9.198-1) to root...
  Downloading file:packages/kmod-nf-ipt_4.9.198-1_mips_24kc.ipk
  Installing kmod-nf-ipt6 (4.9.198-1) to root...
  Downloading file:packages/kmod-nf-ipt6_4.9.198-1_mips_24kc.ipk
  Installing kmod-ipt-core (4.9.198-1) to root...
  Downloading file:packages/kmod-ipt-core_4.9.198-1_mips_24kc.ipk
  Installing kmod-ip6tables (4.9.198-1) to root...
  Downloading file:packages/kmod-ip6tables_4.9.198-1_mips_24kc.ipk
  Installing libip4tc (1.6.2-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libip4tc_1.6.2-3_mips_24kc.ipk.
  Installing libip6tc (1.6.2-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libip6tc_1.6.2-3_mips_24kc.ipk.
  Installing libxtables (1.6.2-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libxtables_1.6.2-3_mips_24kc.ipk.
  Installing iptables (1.6.2-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/iptables_1.6.2-3_mips_24kc.ipk.
  Package iptables (1.6.2-3) installed in root is up to date.
  Package kernel (4.9.198-1-3ba2e8c5db2fc4389b537731cf37120c) installed in root is up to date.
  Installing kmod-ath9k (4.9.198+2017-11-01-10) to root...
  Downloading file:packages/kmod-ath9k_4.9.198%2b2017-11-01-10_mips_24kc.ipk
  Installing libnl-tiny (0.1-5) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/libnl-tiny_0.1-5_mips_24kc.ipk.
  Installing kmod-gpio-button-hotplug (4.9.198-2) to root...
  Downloading file:packages/kmod-gpio-button-hotplug_4.9.198-2_mips_24kc.ipk
  Installing kmod-ledtrig-gpio (4.9.198-1) to root...
  Downloading file:packages/kmod-ledtrig-gpio_4.9.198-1_mips_24kc.ipk
  Package libc (1.1.19-2) installed in root is up to date.
  Package libgcc (7.3.0-2) installed in root is up to date.
  Installing logd (2018-02-14-128bc35f-2) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/logd_2018-02-14-128bc35f-2_mips_24kc.ipk.
  Installing mtd (23) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/mtd_23_mips_24kc.ipk.
  Installing libubox (2018-07-25-c83a84af-2) to root...
  Installing netifd (2019-01-31-a2aba5c7-2.1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/netifd_2019-01-31-a2aba5c7-2.1_mips_24kc.ipk.
  Installing odhcp6c (2018-07-14-67ae6a71-15) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/odhcp6c_2018-07-14-67ae6a71-15_mips_24kc.ipk.
  Installing odhcpd-ipv6only (1.15-3) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/odhcpd-ipv6only_1.15-3_mips_24kc.ipk.
  Installing opkg (2019-06-14-dcbc142e-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/opkg_2019-06-14-dcbc142e-1_mips_24kc.ipk.
  Installing rbcfg (2) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/rbcfg_2_mips_24kc.ipk.
  Installing rssileds (0.2-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/rssileds_0.2-1_mips_24kc.ipk.
  Installing swconfig (11) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/swconfig_11_mips_24kc.ipk.
  Installing uboot-envtools (2018.03-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/uboot-envtools_2018.03-1_mips_24kc.ipk.
  Installing uci (2019-05-17-f199b961-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/uci_2019-05-17-f199b961-1_mips_24kc.ipk.
  Installing uclient-fetch (2018-11-24-3ba74ebc-1) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/uclient-fetch_2018-11-24-3ba74ebc-1_mips_24kc.ipk.
  Installing wpad-mini (2018-05-21-62566bc2-8) to root...
  Copying /home/foo/doc/openwrt_imagebuilder/openwrt-imagebuilder-18.06.5-ar71xx-mikrotik.Linux-x86_64/dl/wpad-mini_2018-05-21-62566bc2-8_mips_24kc.ipk.
  Configuring kmod-nf-reject6.
  Configuring uboot-envtools.
  Configuring kmod-ledtrig-gpio.
  Configuring rbcfg.
  Configuring luci-app-firewall.
  Configuring kmod-nf-ipt.
  Configuring libxtables.
  Configuring libip4tc.
  Configuring kmod-nf-ipt6.
  Configuring kmod-ipt-core.
  Configuring kmod-ip6tables.
  Configuring fstools.
  Configuring libip6tc.
  Configuring luci-base.
  Configuring luci-mod-admin-full.
  Configuring kmod-gpio-button-hotplug.
  Configuring luci-i18n-firewall-fr.
  Configuring iptables.
  Configuring ip6tables.
  Configuring luci-i18n-base-en.
  Configuring luci-i18n-base-fr.
  Collected errors:
   * opkg_install_pkg: Package size mismatch: libubox is 19365 bytes, expecting 19361 bytes
   * opkg_install_cmd: Cannot install package luci.
   * opkg_install_pkg: Package size mismatch: libuci is 15454 bytes, expecting 15461 bytes
   * opkg_install_cmd: Cannot install package base-files.
   * opkg_install_pkg: Package size mismatch: busybox is 212951 bytes, expecting 212973 bytes
   * opkg_install_cmd: Cannot install package busybox.
   * opkg_install_pkg: Package size mismatch: dnsmasq is 99783 bytes, expecting 99775 bytes
   * opkg_install_cmd: Cannot install package dnsmasq.
   * opkg_install_pkg: Package size mismatch: dropbear is 86613 bytes, expecting 86610 bytes
   * opkg_install_cmd: Cannot install package dropbear.
   * opkg_install_pkg: Package size mismatch: firewall is 40776 bytes, expecting 40777 bytes
   * opkg_install_cmd: Cannot install package firewall.
   * opkg_install_pkg: Package size mismatch: libnl-tiny is 12953 bytes, expecting 12948 bytes
   * opkg_install_cmd: Cannot install package kmod-ath9k.
   * opkg_install_pkg: Package size mismatch: logd is 10114 bytes, expecting 10111 bytes
   * opkg_install_cmd: Cannot install package logd.
   * opkg_install_pkg: Package size mismatch: libubox is 19365 bytes, expecting 19361 bytes
   * opkg_install_cmd: Cannot install package mtd.
   * opkg_install_pkg: Package size mismatch: netifd is 62005 bytes, expecting 62004 bytes
   * opkg_install_cmd: Cannot install package netifd.
   * opkg_install_pkg: Package size mismatch: odhcp6c is 23650 bytes, expecting 23646 bytes
   * opkg_install_cmd: Cannot install package odhcp6c.
   * opkg_install_pkg: Package size mismatch: odhcpd-ipv6only is 28830 bytes, expecting 28845 bytes
   * opkg_install_cmd: Cannot install package odhcpd-ipv6only.
   * opkg_install_pkg: Package size mismatch: opkg is 58890 bytes, expecting 58895 bytes
   * opkg_install_cmd: Cannot install package opkg.
   * opkg_install_pkg: Package size mismatch: rssileds is 4056 bytes, expecting 4059 bytes
   * opkg_install_cmd: Cannot install package rssileds.
   * opkg_install_pkg: Package size mismatch: swconfig is 9478 bytes, expecting 9484 bytes
   * opkg_install_cmd: Cannot install package swconfig.
   * opkg_install_pkg: Package size mismatch: uci is 7267 bytes, expecting 7262 bytes
   * opkg_install_cmd: Cannot install package uci.
   * opkg_install_pkg: Package size mismatch: uclient-fetch is 7489 bytes, expecting 7485 bytes
   * opkg_install_cmd: Cannot install package uclient-fetch.
   * opkg_install_pkg: Package size mismatch: wpad-mini is 258733 bytes, expecting 258725 bytes
   * opkg_install_cmd: Cannot install package wpad-mini.
  make[2]: *** [Makefile:153: package_install] Error 255
  make[1]: *** [Makefile:114: _call_image] Error 2
  make: *** [Makefile:193: image] Error 2
Project Manager
Baptiste Jonglez commented on 01.01.2020 17:10
Project Manager
Baptiste Jonglez commented on 01.01.2020 17:18

Debugging this issue a bit, looking at a specific package:

  opkg_install_pkg: Package size mismatch: libjson-c2 is 14704 bytes, expecting 14699 bytes

My imagebuilder is from December and I try to build again today, here is the age and size of the files:

  $ ls -l openwrt-imagebuilder-19.07.0-rc2-ath79-generic.Linux-x86_64/dl/
  -rw-r--r-- 1 zorun zorun  14704 Dec 11 09:39 libjson-c2_0.12.1-3_mips_24kc.ipk
  -rw-r--r-- 1 zorun zorun  52200 Jan  1 05:59 openwrt_base
  $ less openwrt-imagebuilder-19.07.0-rc2-ath79-generic.Linux-x86_64/dl/openwrt_base
  Package: libjson-c2
  Version: 0.12.1-3
  Depends: libc
  Provides: libjson-c
  License: MIT
  Section: libs
  Architecture: mips_24kc
  Installed-Size: 13978
  Filename: libjson-c2_0.12.1-3_mips_24kc.ipk
  Size: 14699
  SHA256sum: bde7f9f579c561b9af3f30108d2bc0537d690b6d07b94e2a1a18f4d5b9a66081
  Description:  This package contains a library for javascript object notation backends.
  $ sha256sum openwrt-imagebuilder-19.07.0-rc2-ath79-generic.Linux-x86_64/dl/libjson-c2_0.12.1-3_mips_24kc.ipk
  4d8487c4b7d67ff0ffe4e7be43c49813a36afdb8584a73a078e0d53cf757d8ce  dl/libjson-c2_0.12.1-3_mips_24kc.ipk

To sum it up:

  • I still have old versions of packages in my local "dl/" folder
  • the new package index is correctly downloaded
  • the new package index refers to updated versions of packages, which have different sizes and checksum
  • instead of just deleting the outdated packages and downloading them again, opkg fails

This looks like a regression: before introducing the size check, I suspect the same issue happened (checksum mismatch), but opkg handled it fine.

Project Manager
Baptiste Jonglez commented on 01.01.2020 17:19

Same issue being reported here: https://github.com/openwrt/packages/issues/10847

Brian J. Murrell commented on 22.01.2020 04:54

I am seeing the same issue on 19.07.0 final.

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing