Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#2164 - When compiling with zero packages configured, the compilation hangs forever with a call to "cat" #7354

Closed
openwrt-bot opened this issue Mar 6, 2019 · 10 comments
Labels

Comments

@openwrt-bot
Copy link

mike-meshplusplus:

Supply the following if possible:

  • Device problem occurs on
  • Software versions of OpenWrt/LEDE release, packages, etc.
  • Steps to reproduce
git clone --depth 1 https://github.com/openwrt/openwrt.git openwrt-reproduce-package-cat-bug cd openwrt-reproduce-package-cat-bug cp $attached_file_.config .config make V=sc

The build process hangs after these log lines:

find .tmp_versions -name '*.mod' | xargs -r grep -h '\.ko$' | sort -u | sed 's/\.ko$/.o/' | scripts/mod/modpost -o ./Module.symvers -S -s -T - make[5]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x/linux-4.14.104' touch /home/jonesmz/openwrt-reproduce-package-cat-bug/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x/linux-4.14.104/.modules make -C image compile TARGET_BUILD= make[5]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/target/linux/ipq806x' make[5]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. make[5]: Nothing to be done for 'compile'. make[5]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/target/linux/ipq806x/image' make[4]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/target/linux/ipq806x' make[3]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/target/linux' time: target/linux/compile#2.58#0.96#1.99 make[2]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' export MAKEFLAGS= ;make -w -r diffconfig make[2]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' make[2]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' make[3]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/libs/libjson-c' make[3]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/libs/libjson-c' time: package/libs/libjson-c/host-compile#0.16#0.08#0.22 make[3]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/libs/libubox' make[3]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/libs/libubox' time: package/libs/libubox/host-compile#0.68#0.36#0.98 make[3]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/system/opkg' make[2]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' make[3]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug/package/system/opkg' time: package/system/opkg/host-compile#0.55#0.25#0.75 make[2]: Leaving directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' make[2]: Entering directory '/home/jonesmz/openwrt-reproduce-package-cat-bug' make[2]: warning: -jN forced in submake: disabling jobserver mode. rm -rf /home/jonesmz/openwrt-reproduce-package-cat-bug/staging_dir/packages/ipq806x mkdir -p /home/jonesmz/openwrt-reproduce-package-cat-bug/staging_dir/packages/ipq806x

And the following process tree

/bin/bash - make -j25 V=sc - /bin/sh -c _limit=`ulimit -n` ; ............. - make -w -r world - make -j1 package/install - bash -c cat 2>/dev/null - cat

The attached .config file has a minimal configuration where as few packages as possible selected. In this case, I think I've managed to get it down to zero packages. I believe that what's failing is this line:

./package/Makefile: $(call opkg_package_files,$(foreach pkg,$(shell cat $(PACKAGE_INSTALL_FILES) 2>/dev/null),$(pkg)$(call GetABISuffix,$(pkg))))

Where if this line is executed when $PACKAGE_INSTALL_FILES expands to nothing, then the "cat" program expects to read data from standard in, which will wait forever.

An easy workaround is to add /dev/null to the list of files to cat, such as

./package/Makefile: $(call opkg_package_files,$(foreach pkg,$(shell cat /dev/null $(PACKAGE_INSTALL_FILES) 2>/dev/null),$(pkg)$(call GetABISuffix,$(pkg))))

But you still end up with failures later, such as opkg being given an empty list of packages to install.

@openwrt-bot
Copy link
Author

mike-meshplusplus:

Wow, my formatting got absolutely destroyed. Sorry about that.

I apparently can't edit the bug, but can resubmit if that is needed

@openwrt-bot
Copy link
Author

yousong:

Regardless of how you managed to not select a single package, even if the build made its way to the end, the rootfs will be empty and not useful anyway. I do not think we should support that.

In addition, I haven't tried to build with the attached config yet, but "make defconfig" will make some CONFIG_PACKAGE_xx builtin.

@openwrt-bot
Copy link
Author

mike-meshplusplus:

I made the configuration by hand to assist in tracking down another problem that I was having. However, it's been nearly 8 months since I started investigating, and I no longer recall the details.

I suggest that if making an empty configuration is not supported, that the make process errors out with a meaningful error message prior to having trouble much later down the line, in this case with the "cat" program being stuck forever.

@openwrt-bot
Copy link
Author

yousong:

Not a fan of such check myself. Maybe it's nice to have an explicit error message there to avoid future changes accidentally allows such empty config. For the moment I am fine with its behavior when fed with empty package list to install.

@openwrt-bot
Copy link
Author

ynezz:

I suggest that if making an empty configuration is not supported, that the make process errors out with a meaningful error message prior to having trouble much later down the line, in this case with the "cat" program being stuck forever.

Well, I think, that the big fat RED warning is good enough:

WARNING: your configuration is out of sync. Please run make menuconfig, oldconfig or defconfig!

@openwrt-bot
Copy link
Author

mike-meshplusplus:

@petr Štetiar

you're getting that warning because the config that I provided was generated against the master branch 8 months ago. The available packages have changed significantly since then.

Nevertheless, that's not an acceptable warning to cover this condition.

If the configuration being out of sync is going to result in a hard error, then it should be an error, and the build should promptly halt.

@openwrt-bot
Copy link
Author

mike-meshplusplus:

@yousong Zhou

even if the build made its way to the end, the rootfs will be empty and not useful anyway.

I would actually appreciate the ability to create an empty rootfs. It would be a useful outcome for some of the automated testing capabilities that my team is working on.

@openwrt-bot
Copy link
Author

yousong:

Hi Michael, I think the issue reported here is not too big of a concern. It happened solely because it's intentional. I think it's not part of the design principles the build system geared toward preventing its users shooting themselves in the feet this way.

That said, the better path forward would be you posting a patch to the mailing with your use case detailed there. It's more productive this way.

Thanks for the heads-up.

@openwrt-bot
Copy link
Author

ynezz:

The available packages have changed significantly since then.

I don't think so.

curl -s https://bugs.openwrt.org/index.php?getfile=678 > .config; make defconfig 2> /dev/null; grep ^CONFIG_PACKAGE .config # # configuration written to .config # CONFIG_PACKAGE_getrandom=y CONFIG_PACKAGE_libc=y CONFIG_PACKAGE_libgcc=y CONFIG_PACKAGE_urandom-seed=y CONFIG_PACKAGE_urngd=y CONFIG_PACKAGE_libubox=y

Nevertheless, that's not an acceptable warning to cover this condition.

Making this warning hard stop would prevent other valid use cases. It's just warning, that you're probably doing something wrong, telling you, that you're supposed to add make defconfig into your build step.

Quoting from [[https://openwrt.org/docs/guide-developer/build-system/use-buildsystem#defconfig|Build system usage - defconfig]] documentation:

It also checks the dependencies and will add possibly missing necessary dependencies. This can be used to “expand” a short .config recipe (like diffconfig output, possible even pruned further) to a full .config that the make process accepts.

@openwrt-bot
Copy link
Author

ynezz:

I would actually appreciate the ability to create an empty rootfs.

The build system is flexible enough, so I believe, that you can bend it to your needs. You simply can't use custom .config to achieve that goal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant