forked from OSchip/llvm-project
1b3c146acb
Summary: This drastically reduces lit test execution startup time on Windows. Our previous strategy was to manually create one Process per job and manage the worker pool ourselves. Instead, let's use the worker pool provided by multiprocessing. multiprocessing.Pool(jobs) returns almost immediately, and initializes the appropriate number of workers, so they can all start executing tests immediately. This avoids the ramp-up period that the old implementation suffers from. This appears to speed up small test runs. Here are some timings of the llvm-readobj tests on Windows using the various execution strategies: # multiprocessing.Pool: $ for i in `seq 1 3`; do tim python ./bin/llvm-lit.py -sv ../llvm/test/tools/llvm-readobj/ --use-process-pool |& grep real: ; done real: 0m1.156s real: 0m1.078s real: 0m1.094s # multiprocessing.Process: $ for i in `seq 1 3`; do tim python ./bin/llvm-lit.py -sv ../llvm/test/tools/llvm-readobj/ --use-processes |& grep real: ; done real: 0m6.062s real: 0m5.860s real: 0m5.984s # threading.Thread: $ for i in `seq 1 3`; do tim python ./bin/llvm-lit.py -sv ../llvm/test/tools/llvm-readobj/ --use-threads |& grep real: ; done real: 0m9.438s real: 0m10.765s real: 0m11.079s I kept the old code to launch processes in case this change doesn't work on all platforms that LLVM supports, but at some point I would like to remove both the threading and old multiprocessing execution strategies. Reviewers: modocache, rafael Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31677 llvm-svn: 299560 |
||
|---|---|---|
| .. | ||
| bindings | ||
| cmake | ||
| docs | ||
| examples | ||
| include | ||
| lib | ||
| projects | ||
| resources | ||
| runtimes | ||
| test | ||
| tools | ||
| unittests | ||
| utils | ||
| .arcconfig | ||
| .clang-format | ||
| .clang-tidy | ||
| .gitignore | ||
| CMakeLists.txt | ||
| CODE_OWNERS.TXT | ||
| CREDITS.TXT | ||
| LICENSE.TXT | ||
| LLVMBuild.txt | ||
| README.txt | ||
| RELEASE_TESTERS.TXT | ||
| configure | ||
| llvm.spec.in | ||
README.txt
Low Level Virtual Machine (LLVM) ================================ This directory and its subdirectories contain source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and runtime environments. LLVM is open source software. You may freely distribute it under the terms of the license agreement found in LICENSE.txt. Please see the documentation provided in docs/ for further assistance with LLVM, and in particular docs/GettingStarted.rst for getting started with LLVM and docs/README.txt for an overview of LLVM's documentation setup. If you are writing a package for LLVM, see docs/Packaging.rst for our suggestions.