Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#1144 - Opkg fails with Out of memory on a TP-Link RE450 #5631

Open
openwrt-bot opened this issue Nov 4, 2017 · 8 comments
Open

FS#1144 - Opkg fails with Out of memory on a TP-Link RE450 #5631

openwrt-bot opened this issue Nov 4, 2017 · 8 comments
Labels

Comments

@openwrt-bot
Copy link

Markinus:

Hi!

I a problem on my TPLINK RE450 on upgrading of packages. I'm restarting the system and the upgrade is still not possible, console output:

BusyBox v1.25.1 () built-in shell (ash)

 _________
/        /\      _    ___ ___  ___

/ LE / \ | | | | | |
/ DE / \ | || _|| |) | _|
/
_/ LE \ |||/|| lede-project.org
\ \ DE /
\ LE \ / -----------------------------------------------------------
\ DE \ / Reboot (17.01.4, r3560-79f57e422d)
________/ -----------------------------------------------------------

root@LEDE:# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 2560 2560 0 100% /rom
tmpfs 30332 80 30252 0% /tmp
/dev/mtdblock3 2176 236 1940 11% /overlay
overlayfs:/overlay 2176 236 1940 11% /
tmpfs 512 0 512 0% /dev
root@LEDE:
# free
total used free shared buffers cached
Mem: 60664 43600 17064 80 2320 6048
-/+ buffers/cache: 35232 25432
Swap: 0 0 0
root@LEDE:# opkg update
Downloading http://downloads.lede-project.org/releases/17.01.4/targets/ar71xx/generic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_core
Downloading http://downloads.lede-project.org/releases/17.01.4/targets/ar71xx/generic/packages/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/base/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_base
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/base/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_luci
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_packages
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/packages/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_routing
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/routing/Packages.sig
Signature check passed.
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_telephony
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/telephony/Packages.sig
Signature check passed.
root@LEDE:
# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 2560 2560 0 100% /rom
tmpfs 30332 568 29764 2% /tmp
/dev/mtdblock3 2176 236 1940 11% /overlay
overlayfs:/overlay 2176 236 1940 11% /
tmpfs 512 0 512 0% /dev
root@LEDE:# free
total used free shared buffers cached
Mem: 60664 44804 15860 568 2320 6756
-/+ buffers/cache: 35728 24936
Swap: 0 0 0
root@LEDE:
# opkg list upgradable
root@LEDE:# opkg list-upgradable
luci-lib-ip - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-theme-bootstrap - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
dnsmasq - 2.78-1 - 2.78-2
luci-app-firewall - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-proto-ppp - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-mod-admin-full - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-base - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-proto-ipv6 - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-lib-nixio - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci-lib-jsonc - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
luci - git-17.290.79498-d3f0685-1 - git-17.298.45605-07e7ff0-1
root@LEDE:
# opkg upgrade luci-base
Upgrading luci-base on root from git-17.290.79498-d3f0685-1 to git-17.298.45605-07e7ff0-1...
Downloading http://downloads.lede-project.org/releases/17.01.4/packages/mips_24kc/luci/luci-base_git-17.298.45605-07e7ff0-1_mips_24kc.ipk
Collected errors:

It looks like there is enough memory availible, where is the problem? Can someone explain it? With a sysctl -w vm.min_free_kbytes=0 the upgrade is possible.

@openwrt-bot
Copy link
Author

Markinus:

https://forum.lede-project.org/t/tp-link-re450-64m-128m-two-version/3448/15

Some devices seems to have 60 and some 125 MB RAM... my has 60.

@openwrt-bot
Copy link
Author

ibex-are-goats:

Edit: Replied to wrong bug.

@openwrt-bot
Copy link
Author

TDFKAOlli:

Same problem on TP-Link TL-WR1043N/ND v1:

BusyBox v1.25.1 () built-in shell (ash)

 _________
/        /\      _    ___ ___  ___

/ LE / \ | | | | | |
/ DE / \ | || _|| |) | _|
/
_/ LE \ |||/|| lede-project.org
\ \ DE /
\ LE \ / -----------------------------------------------------------
\ DE \ / Reboot (17.01.4, r3560-79f57e422d)
________/ -----------------------------------------------------------

root@LEDE:~# opkg list
Collected errors:

  • pkg_hash_add_from_file: Failed to open /var/opkg-lists/reboot_routing: Out of memory.

Update of packages still works. When I stop one of the tasks in the Startup list, then listing the packages works.

Seems that okpg has a memory requirement which makes it difficult to run on a low mem router. If would be great if it could be tuned for low memory consumption. Shutting down processes to install packages is a workaround but dangerous and not handy.

BTW: sysctl -w vm.min_free_kbytes=0 doesn't help in my case.

@openwrt-bot
Copy link
Author

jow-:

Opkg in LEDE already is heavily tuned for low memory consumption. Compared to the original version it requires 80% less RAM for processing package lists but even that is not enough for certain low memory devices.

Your only choice on such devices is to not use opkg at all or to reduce the number of package feed entries.

@openwrt-bot
Copy link
Author

BUG-reporter:

BusyBox v1.28.3 () built-in shell (ash)


| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -| || | | || || |
|
_____|| |
||||___||| |____|
|
| W I R E L E S S F R E E D O M

OpenWrt 18.06.0, r7188-b0b5c64c22
-----------------------------------------------------

TP-Link TL-WR1043N/ND v1

The same issue I'm expiriencing all the time since 17.01 or even earlier

@openwrt-bot
Copy link
Author

fedorinoff:

TP-Link wr842nd v2.
After install OpenVPN server.

OpenWrt 18.06.1, r7258-5eb055306f

root@OpenWrt:~# opkg update
Collected errors:

  • pkg_hash_add_from_file: Failed to open /var/opkg-lists/openwrt_routing: Out of memory.
    root@OpenWrt:# df
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/root 2560 2560 0 100% /rom
    tmpfs 13916 1164 12752 8% /tmp
    /dev/sda1 14080464 47392 13298096 0% /overlay
    overlayfs:/overlay 14080464 47392 13298096 0% /
    tmpfs 512 0 512 0% /dev
    root@OpenWrt:
    #

two minutes later...
opkg install kmod-nls-cp1251 kmod-nls-koi8r kmod-nls-utf8 kmod-n
ls-cp866 kmod-nls-cp437

Installing kmod-nls-cp1251 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp1251_4.9.120-1_mips_24kc.ipk
Installing kmod-nls-koi8r (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-koi8r_4.9.120-1_mips_24kc.ipk
Installing kmod-nls-utf8 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-utf8_4.9.120-1_mips_24kc.ipk
Installing kmod-nls-cp866 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp866_4.9.120-1_mips_24kc.ipk
Installing kmod-nls-cp437 (4.9.120-1) to root...
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/kmod-nls-cp437_4.9.120-1_mips_24kc.ipk
Configuring kmod-nls-cp1251.
Configuring kmod-nls-utf8.
Configuring kmod-nls-cp437.
Configuring kmod-nls-cp866.
Configuring kmod-nls-koi8r.
root@OpenWrt:#
root@OpenWrt:
# opkg update
Downloading http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/targets/ar71xx/generic/packages/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/base/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/base/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/luci/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/luci/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/packages/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/packages/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/routing/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/routing/Packages.gz

Downloading http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/telephony/Packages.gz
*** Failed to download the package list from http://downloads.openwrt.org/releases/18.06.1/packages/mips_24kc/telephony/Packages.gz

Collected errors:

then after next try...

root@OpenWrt:~#opkg update
Collected errors:

  • glob_errfunc: glob failed for /: I/O error
    root@OpenWrt:~#

P.S. Seems that problem was with my swap file. partition /dev/sda2 was assigned as swap, but not formatted. After format and reboot opkg update works fine.

@openwrt-bot
Copy link
Author

ynezz:

Please can you try new 18.06.2 release and provide output of free command before and after opkg run?

@openwrt-bot
Copy link
Author

ckujau:

This happens here on a [[https://openwrt.org/toh/tp-link/archer-c5-c7-wdr7500|External LinkTP-Link Archer C7 v2]] (16 MB Flash, 128 MB RAM) with 19.07.1 (ath79/generic) installed:

# opkg update Downloading http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz *** Failed to download the package list from http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz [...] Collected errors: * xsystem: wget: vfork: Out of memory. * opkg_download: Failed to download http://downloads.openwrt.org/releases/19.07.1/targets/ath79/generic/packages/Packages.gz, wget returned -1. [...]

cat /proc/meminfo

MemTotal: 124528 kB
MemFree: 33704 kB
MemAvailable: 17584 kB
Buffers: 1784 kB
Cached: 44044 kB
SwapCached: 0 kB
Active: 11728 kB
Inactive: 40108 kB
Active(anon): 6896 kB
Inactive(anon): 36156 kB
Active(file): 4832 kB
Inactive(file): 3952 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 6016 kB
Mapped: 3532 kB
Shmem: 37044 kB
Slab: 15320 kB
SReclaimable: 8128 kB
SUnreclaim: 7192 kB
KernelStack: 392 kB
PageTables: 396 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 62264 kB
Committed_AS: 48376 kB
VmallocTotal: 1048372 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB

free

          total        used        free      shared  buff/cache   available

Mem: 124528 44748 33952 37044 45828 17832
Swap: 0 0 0

Setting "vm.min_free_kbytes=0" helps indeed. The default appears to be 16384 (16 MB). However, according to "free" it has 33 MB of "free" memory, that should be enough to complete the task. But then it also prints 37 MB "shared" and 45 MB cached, which should all count as "available" memory, so this 17 MB "available" memory looks kinda confusing. On a "normal" desktop system free+shared+buff/cache usually equals "available", but not on an embedded system?

In any case, this did not happen with 18.06 and earlier releases for my device (then with "ar71xx/generic" installed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant