Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#1967 - bug make download will hang forever if networking flush/unstable #6864

Open
openwrt-bot opened this issue Nov 24, 2018 · 0 comments
Labels

Comments

@openwrt-bot
Copy link

openalmeida:

Supply the following if possible: - "all", Device problem occurs on - "master branch", Software versions of OpenWrt/LEDE release, packages, etc. - "said below", Steps to reproduce

when i call make download, it hangs/dead,
after debug (tools: htop and strace, and read the sources),
i found it because of blocking socket.

details below:

  1. the python script scripts/dl_github_archive.py
# https://github.com/openwrt/openwrt/blob/master/scripts/dl_github_archive.py#L402
fileobj = urllib2.urlopen(req, context=sslcontext)
# this will make a blocked socket connection
# if no timeout param, it will use global setting, which is None, it means blocked forever.
# this is bad smell, especially unstable networking.

# ref: https://docs.python.org/3/library/socket.html?highlight=socket#socket-timeouts

running situation:

`
/mnt/op/build/scripts/dl_github_archive.py \
--dl-dir="/mnt/op/build/dl" \
--url="https://github.com/kvalo/ath10k-firmware.git" \
--version="d366b80d41309531c0c98b26625f23e318f33e31" \
--subdir="ath10k-firmware-2018-10-10-d366b80d" \
--source="ath10k-firmware-2018-10-10-d366b80d.tar.xz" \
--hash="6285b9f9aa5a541b8258750f36955a6b307903a87cedd842b73dbfdd2bef8986"
`

which bloked/dead because of networking issue.

if patch that line, like this
`
fileobj = urllib2.urlopen(req, timeout=30, context=sslcontext)
`

it wont block forever even networking lost.

  1. as a proof, the perl script scritps/download.pl has timeout and retry param.
# https://github.com/openwrt/openwrt/blob/master/scripts/download.pl#L85-L86
? (qw(curl -f --connect-timeout 20 --retry 5 --location --insecure), shellwords($ENV{CURL_OPTIONS} || ''), $url)
: (qw(wget --tries=5 --timeout=20 --no-check-certificate --output-document=-), shellwords($ENV{WGET_OPTIONS} || ''), $url)

so, add timeout and retry option for make download please,
and apply to all process of downloading stuff operation (
include the 2 scripts above and others, e.g git clone etc.
).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant