logilab-devtools #29035 parallel builds [in-progress]

it would be nice if lgp build could run jobs in parallel (using the multiprocessing stdlib module for example).

For example lgp build -j4 could run up to four build jobs in parallel.

prioritynormal
typeenhancement
done in<not specified>
load left0.000
closed by<not specified>

Workflow history

from state (1)to statecommentdateUser
openin-progress

Use of the multiprocessing module to dispatch build jobs in parallel.

Initial value is set to the number of core cpu (thanks to cpu_count()).

Note: build should be done in tmpfs to prevent new I/O bound limit (because of the parallel chroot creations).

2011/05/04 22:19 UTC 

Comments

  • 2010/05/12 15:11

    another possibility is to rely on an external tool such as Condor to exploit a pool of CPUs.

add comment

Ticket #29035 - latest update on 2011/05/16, created on 2010/05/12 by Nicolas Chauvat