Right now when you would just restart the builder, we would just
- stop tracking the background
docker run call
- after restart, pick up the build again from the queue (or another builder does)
- and start from scratch.
This means some time after a "simple" restart, we still have the old build running and consuming resources (cpu, memory, disk).
The current workaround is
- lock the queue
- wait until all in-progress builds are finished
- restart the server processes
- unlock the queue
What woud actually be the best option is:
- extend https://github.com/rust-lang/rustwide so it supports cancelling an in-progress build (probably just
SIGTERM or SIGKILL the docker run process.
- change the builder so it can listen to the signals,
- update the in-progress records in the
builds table
- kill the in-progress builds on shutdown
See also #3250
Right now when you would just restart the builder, we would just
docker runcallThis means some time after a "simple" restart, we still have the old build running and consuming resources (cpu, memory, disk).
The current workaround is
What woud actually be the best option is:
SIGTERMorSIGKILLthedocker runprocess.buildstableSee also #3250