Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#2096 - A second 'make' always rebuilds something #6951

Open
openwrt-bot opened this issue Feb 2, 2019 · 7 comments
Open

FS#2096 - A second 'make' always rebuilds something #6951

openwrt-bot opened this issue Feb 2, 2019 · 7 comments
Labels

Comments

@openwrt-bot
Copy link

rdiez:

After a full OpenWrt Git HEAD build finishes, running "make" repeatedly again always rebuilds something. Another user is reporting that a subsequent build takes 5 minutes on his Fedora 27 desktop, which I am also seeing on my oldish Ubuntu 16.04.5 laptop.

Some more information is in the original mailing list discussion. This is the first message in the relevant thread:

http://lists.infradead.org/pipermail/openwrt-devel/2019-January/015648.html

Please find attached a build script to reproduce this problem, and a full build log file from my laptop.

Look for "Running the second make..." in the log file. Afterwards, things like these are being rebuilt, which probably should not:

make[6]: Entering directory '/home/rdiez/rdiez/freifunk/openwrt/git-repo/build_dir/target-x86_64_musl/uci-2018-08-11-4c8b4d6e'
[ 5%] Building C object CMakeFiles/uci.dir/libuci.c.o
[ 10%] Building C object CMakeFiles/uci.dir/file.c.o

@openwrt-bot
Copy link
Author

ynezz:

I'm not able to reproduce it on fresh VPS(Scaleway,Vultr) running Fedora 28/29, Ubuntu 14.04/16.04/18.04. I see times under 1m for the 2nd make run. Since I'm not able to reproduce it on 5 different hosts, I think, that it would be really helpful if you could provide Dockerfile which could show this 2nd make problem.

@openwrt-bot
Copy link
Author

rdiez:

I have not learnt Docker yet, and I am an OpenWrt newbie. Before I invest more time in this matter:

  1. Did you try the exact script I uploaded to this bug?

  2. Does the log file in this bug provide no clues at all about what could be triggering the rebuild?

  3. A build time under 1 minute could be considered very long if nothing has changed and the system has loads of RAM and an SSD disk. Normally, it takes almost no time to re-run GNU Make on a set of files that has not changed at all. Do you have an idea why it does not take just 2-3 seconds on your VPS systems?

@openwrt-bot
Copy link
Author

ynezz:

You don't need to use Docker, but as you can see, I'm not able to reproduce the issue with provided build script on fresh Ubuntu 16.04 VPS, which probably means, that this 5 minutes 2nd build issue is only related to your build host.

I'm aware about similar issue on Fedora 29 http://palmtree.beeroclock.net/~karlp/bugs/owrt-rebuilds/ which I'm not able to reproduce on fresh Fedora 29 VPS either.

In other words, if we can't reproduce it, the problem doesn't exist. I just think, that Docker could provide a good reproducible (from scratch) build environment, and is quite easy to setup.

  1. Yes, but I've removed the download step (waste of expensive CPU/RAM resources) and I've added timing to the build steps
  2. No, I'm sorry but I don't have time to inspect build logs.
  3. Feel free to improve it. I think, that once you understand how complex the build system is, you'll probably understand, that 2-3 seconds is a nonsense.

@openwrt-bot
Copy link
Author

rdiez:

I'll take a closer look.

In the meantime, could you provide a build log? This way, I can compare the build log outputs and find out what my machine is rebuilding that your VMs are not.

How long you do you think that rebuilding nothing should take? In the Freifunk community, we are often building firmwares for all routers and all cities. After every little change, such long build times add up. This is killing developer productivity. We are all volunteers, so this is very much wasting our personal time.

@openwrt-bot
Copy link
Author

ynezz:

I don't have the build log, I've deleted the VPS after the build has finished. It's quite easy to spin up new VPS or KVM machine with clean Ubuntu 16.04 system for testing.

For example https://www.hetzner.de/cloud offers VPS with 8 vCPU, 32GB RAM, NVMe SSD disk which can do build from scratch relatively fast, in around 30 minutes (basic ar71xx config). This VPS costs 0.05 EUR per hour, which I find relatively cheap and fast option for testing.

BTW we are all volunteers, so feel free to send patches which will improve the (re)build times for all of us.

You've probably missed it, but here http://palmtree.beeroclock.net/~karlp/bugs/owrt-rebuilds/vm.html you can find 2nd build times for Ubuntu 16.04 running in the VM.

@openwrt-bot
Copy link
Author

rdiez:

I am finding it hard to understand the OpenWrt build system internals. Is there any documentation about it?

In any case, I have rewritten the timestamp.pl script to help me see what is going on. My version is here:

https://github.com/rdiez/Tools/tree/master/Timestamp

You may want to replace the OpenWrt script with mine. The current one is tricky, to say the least.

@openwrt-bot
Copy link
Author

lucize:

I think that util-linux is always recompiling, I saw doing that almost every time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant