Summary:
Whith the number of projects growing, it is important to be able to
filter them in a more convenient way than by names. It is especially
important for benchmarks, when it is not viable to analyze big
projects 20 or 50 times in a row.
Because of this reason, this commit adds a notion of sizes and a
filtering interface that puts a limit on a maximum size of the project
to analyze or benchmark.
Sizes assigned to the projects in this commit, do not directly
correspond to the number of lines or files in the project. The key
factor that is important for the developers of the analyzer is the
time it takes to analyze the project. And for this very reason,
"size" basically helps to cluster projects based on their analysis
time.
Differential Revision: https://reviews.llvm.org/D83942
Summary:
Not all projects in the project map file might have newer results
for updating, we should handle this situation gracefully.
Additionally, not every user of the test system would want storing
reference results in git. For this reason, git functionality is now
optional.
Differential Revision: https://reviews.llvm.org/D84303
Summary:
This commit includes a couple of changes:
* Benchmark selected projects by analyzing them multiple times
* Compare two benchmarking results and visualizing them on one chart
* Organize project build logging, so we can use the same code
in benchmarks
Differential Revision: https://reviews.llvm.org/D83539
Summary:
Docker on its own has a pretty convenient way to run shell.
This method, however, requires target container to be currently running,
which is not a usual scenario for the test system. For this purpose,
it is better to have a simple way to run the container, shell it, and
clean up at the end of it all. New option `--shell` does exactly this.
Differential Revision: https://reviews.llvm.org/D81598
Summary:
If the user has only python2 installed and wants to use
the dockerized testing system, it is now totally OK.
Differential Revision: https://reviews.llvm.org/D81595
Summary:
Forward results of every command executed in docker. The actual commands
and their error codes are more informative than python stacktraces.
Differential Revision: https://reviews.llvm.org/D81593
Summary:
It provides a simpler interface for testing within docker.
This way the user is not required to no how to use `docker run` and
its options.
Differential Revision: https://reviews.llvm.org/D81572
Valeriy Savchenko