OpenWrt/LEDE Project

  • Status Unconfirmed
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Base system
  • Assigned To No-one
  • Operating System All
  • Severity Low
  • Priority Very Low
  • Reported Version Trunk
  • Due in Version Undecided
  • Due Date Undecided
  • Private
Attached to Project: OpenWrt/LEDE Project
Opened by Markus Eckhardt - 04.11.2017

FS#1144 - Opkg fails with Out of memory on a TP-Link RE450

Hi!

I a problem on my TPLINK RE450 on upgrading of packages. I’m restarting the system and the upgrade is still not possible, console output:

BusyBox v1.25.1 () built-in shell (ash)

     _________
    /        /\      _    ___ ___  ___
   /  LE    /  \    | |  | __|   \| __|
  /    DE  /    \   | |__| _|| |) | _|
 /________/  LE  \  |____|___|___/|___|                      lede-project.org
 \        \   DE /
  \    LE  \    /  -----------------------------------------------------------
   \  DE    \  /    Reboot (17.01.4, r3560-79f57e422d)
    \________\/    -----------------------------------------------------------

root@LEDE:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 2560      2560         0 100% /rom
tmpfs                    30332        80     30252   0% /tmp
/dev/mtdblock3            2176       236      1940  11% /overlay
overlayfs:/overlay        2176       236      1940  11% /
tmpfs                      512         0       512   0% /dev
root@LEDE:~# free
             total       used       free     shared    buffers     cached
Mem:         60664      43600      17064         80       2320       6048
-/+ buffers/cache:      35232      25432
Swap:            0          0          0
root@LEDE:~# opkg update
Downloading http://downloads.lede-project.org/releases/17.01.4/targets/ar71xx/generic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_core
Downloading http://downloads.lede-project.org/releases/17.01.4/targets/ar71xx/generic/packages/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/base/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_base
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/base/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_luci
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_packages
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/packages/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_routing
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/routing/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_telephony
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/telephony/Packages.sig
Signature check passed.
root@LEDE:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 2560      2560         0 100% /rom
tmpfs                    30332       568     29764   2% /tmp
/dev/mtdblock3            2176       236      1940  11% /overlay
overlayfs:/overlay        2176       236      1940  11% /
tmpfs                      512         0       512   0% /dev
root@LEDE:~# free
             total       used       free     shared    buffers     cached
Mem:         60664      44804      15860        568       2320       6756
-/+ buffers/cache:      35728      24936
Swap:            0          0          0
root@LEDE:~# opkg list upgradable
root@LEDE:~# opkg list-upgradable
luci-lib-ip - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-theme-bootstrap - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
dnsmasq - 2.78-1 - 2.78-2
luci-app-firewall - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-proto-ppp - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-mod-admin-full - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-base - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-proto-ipv6 - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-lib-nixio - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-lib-jsonc - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
root@LEDE:~# opkg upgrade luci-base
Upgrading luci-base on root from git-17.290.79498-d3f0685-1 to git-17.298.45605-07e7ff0-1...
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/luci-base_git-17.298.45605-07e7ff0-1_mips_24kc.ipk
Collected errors:
 * xsystem: wget: vfork: Out of memory.
 * opkg_download: Failed to download http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/luci-base_git-17.298.45605-07e7ff0-1_mips_24kc.ipk, wget returned -1.
 * opkg_install_pkg: Failed to download luci-base. Perhaps you need to run 'opkg update'?
root@LEDE:~#

It looks like there is enough memory availible, where is the problem? Can someone explain it? With a sysctl -w vm.min_free_kbytes=0 the upgrade is possible.


Markus Eckhardt commented on 04.11.2017 09:39

https://forum.lede-project.org/t/tp-link-re450-64m-128m-two-version/3448/15

Some devices seems to have 60 and some 125 MB RAM... my has 60.

ibex-are-goats commented on 06.12.2017 09:58

Edit: Replied to wrong bug.

TDFKAOlli commented on 23.01.2018 19:39

Same problem on TP-Link TL-WR1043N/ND v1:

BusyBox v1.25.1 () built-in shell (ash)

   _________
  /        /\      _    ___ ___  ___
 /  LE    /  \    | |  | __|   \| __|
/    DE  /    \   | |__| _|| |) | _|

// LE \ ||_|_/|_| lede-project.org
\ \ DE /
\ LE \ / ———————————————————– \ DE \ / Reboot (17.01.4, r3560-79f57e422d)
\
__\/ ———————————————————–

root@LEDE:~# opkg list
Collected errors:
* pkg_hash_add_from_file: Failed to open /var/opkg-lists/reboot_routing: Out of memory.

Update of packages still works. When I stop one of the tasks in the Startup list, then listing the packages works.

Seems that okpg has a memory requirement which makes it difficult to run on a low mem router. If would be great if it could be tuned for low memory consumption. Shutting down processes to install packages is a workaround but dangerous and not handy.

BTW: sysctl -w vm.min_free_kbytes=0 doesn't help in my case.

Admin
Jo-Philipp Wich commented on 25.01.2018 09:37

Opkg in LEDE already is heavily tuned for low memory consumption. Compared to the original version it requires 80% less RAM for processing package lists but even that is not enough for certain low memory devices.

Your only choice on such devices is to not use opkg at all or to reduce the number of package feed entries.

BUG-reporter commented on 05.08.2018 17:04
BusyBox v1.28.3 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 18.06.0, r7188-b0b5c64c22
 -----------------------------------------------------

TP-Link TL-WR1043N/ND v1

The same issue I'm expiriencing all the time since 17.01 or even earlier

Yuriy Fedorinov commented on 30.01.2019 19:59

TP-Link wr842nd v2.
After install OpenVPN server.


OpenWrt 18.06.1, r7258-5eb055306f


root@OpenWrt:~# opkg update
Collected errors:
* pkg_hash_add_from_file: Failed to open /var/opkg-lists/openwrt_routing: Out of memory.
root@OpenWrt:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 2560 2560 0 100% /rom
tmpfs 13916 1164 12752 8% /tmp
/dev/sda1 14080464 47392 13298096 0% /overlay
overlayfs:/overlay 14080464 47392 13298096 0% /
tmpfs 512 0 512 0% /dev
root@OpenWrt:~#

two minutes later...
opkg install kmod-nls-cp1251 kmod-nls-koi8r kmod-nls-utf8 kmod-n
ls-cp866 kmod-nls-cp437

Installing kmod-nls-cp1251 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp1251_4.9.120-1_mips_24kc.ipk Installing kmod-nls-koi8r (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-koi8r_4.9.120-1_mips_24kc.ipk Installing kmod-nls-utf8 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-utf8_4.9.120-1_mips_24kc.ipk Installing kmod-nls-cp866 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp866_4.9.120-1_mips_24kc.ipk Installing kmod-nls-cp437 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp437_4.9.120-1_mips_24kc.ipk Configuring kmod-nls-cp1251.
Configuring kmod-nls-utf8.
Configuring kmod-nls-cp437.
Configuring kmod-nls-cp866.
Configuring kmod-nls-koi8r.
root@OpenWrt:~#
root@OpenWrt:~# opkg update
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/Packages.gz Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/base/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/base/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/luci/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/luci/Packages.gz Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/packages/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/packages/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/routing/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/routing/Packages.gz Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/telephony/Packages.gz * Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/telephony/Packages.gz

Collected errors:
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/Packages.gz, wget returned -1.
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/base/Packages.gz, wget returned -1.
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/luci/Packages.gz, wget returned -1.
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/packages/Packages.gz, wget returned -1.
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/routing/Packages.gz, wget returned -1.
* xsystem: wget: vfork: Out of memory.
* opkg_download: Failed to download http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/telephony/Packages.gz, wget returned -1.
root@OpenWrt:~#

then after next try...

root@OpenWrt:~#opkg update
Collected errors:
* glob_errfunc: glob failed for /: I/O error
root@OpenWrt:~#

P.S. Seems that problem was with my swap file. partition /dev/sda2 was assigned as swap, but not formatted. After format and reboot opkg update works fine.

Admin
Petr Štetiar commented on 31.01.2019 12:25

Please can you try new 18.06.2 release and provide output of `free` command before and after opkg run?

Christian Kujau commented on 09.03.2020 03:31

This happens here on a External LinkTP-Link Archer C7 v2 (16 MB Flash, 128 MB RAM) with 19.07.1 (ath79/generic) installed:

# opkg update
Downloading http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz
[...]
Collected errors:
 * xsystem: wget: vfork: Out of memory.
 * opkg_download: Failed to download http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz, wget returned -1.
[...]


# cat /proc/meminfo 
MemTotal:         124528 kB
MemFree:           33704 kB
MemAvailable:      17584 kB
Buffers:            1784 kB
Cached:            44044 kB
SwapCached:            0 kB
Active:            11728 kB
Inactive:          40108 kB
Active(anon):       6896 kB
Inactive(anon):    36156 kB
Active(file):       4832 kB
Inactive(file):     3952 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          6016 kB
Mapped:             3532 kB
Shmem:             37044 kB
Slab:              15320 kB
SReclaimable:       8128 kB
SUnreclaim:         7192 kB
KernelStack:         392 kB
PageTables:          396 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:       62264 kB
Committed_AS:      48376 kB
VmallocTotal:    1048372 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB

# free
              total        used        free      shared  buff/cache   available
Mem:         124528       44748       33952       37044       45828       17832
Swap:             0           0           0

Setting "vm.min_free_kbytes=0" helps indeed. The default appears to be 16384 (16 MB). However, according to "free" it has 33 MB of "free" memory, that should be enough to complete the task. But then it also prints 37 MB "shared" and 45 MB cached, which should all count as "available" memory, so this 17 MB "available" memory looks kinda confusing. On a "normal" desktop system free+shared+buff/cache usually equals "available", but not on an embedded system?

In any case, this did not happen with 18.06 and earlier releases for my device (then with "ar71xx/generic" installed).

   dmesg.txt (16.9 KiB)

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing