a64b205ae7
[analyzer] [testing] Inside CmpRuns.py output also print the filename of the first item in the path
...
Differential Revision: https://reviews.llvm.org/D57783
llvm-svn: 353228
2019-02-05 22:26:57 +00:00
b748c0e696
Portable Python script across Python version
...
Make scripts more future-proof by importing most __future__ stuff.
Differential Revision: https://reviews.llvm.org/D55208
llvm-svn: 349504
2018-12-18 16:07:37 +00:00
d458974c45
Portable Python script across Python version
...
In Python3, dict.items, dict.keys, dict.values, zip, map and filter no longer return lists, they create generator instead.
The portability patch consists in forcing an extra `list` call if the result is actually used as a list.
`map` are replaced by list comprehension and `filter` by filtered list comprehension.
Differential Revision: https://reviews.llvm.org/D55197
llvm-svn: 349501
2018-12-18 16:04:21 +00:00
3744de522c
Portable Python script across Python version
...
In Python2, division between integer yields an integer, while it yields a float in Python3.
Use a combination of from __future__ import division and // operator to get a portable behavior.
Differential Revision: https://reviews.llvm.org/D55204
llvm-svn: 349455
2018-12-18 08:38:50 +00:00
c0ebe773cd
Portable Python script across Python version
...
Using from __future__ import print_function it is possible to have a compatible behavior of `print(...)` across Python version.
Differential Revision: https://reviews.llvm.org/D55213
llvm-svn: 349454
2018-12-18 08:36:33 +00:00
1f88dc5aee
Portable Python script across Python version
...
Queue module as been renamed into queue in Python3
Differential Revision: https://reviews.llvm.org/D55202
llvm-svn: 349009
2018-12-13 07:44:19 +00:00
09616bdb4a
Portable Python script across version
...
Have all classes derive from object: that's implicitly the default in Python3,
it needs to be done explicilty in Python2.
Differential Revision: https://reviews.llvm.org/D55121
llvm-svn: 348127
2018-12-03 12:12:48 +00:00
3de410848c
Portable Python script across Python version
...
Python2 supports the two following equivalent construct
raise ExceptionType, exception_value
and
raise ExceptionType(exception_value)
Only the later is supported by Python3.
Differential Revision: https://reviews.llvm.org/D55195
llvm-svn: 348126
2018-12-03 12:11:21 +00:00
6a2a197f95
[analyzer] [testing] Compute data on path length, compute percentiles
...
Differential Revision: https://reviews.llvm.org/D52844
llvm-svn: 344990
2018-10-23 01:30:26 +00:00
d3b0846e7a
[analyzer] [tests] Hotfix: missing space
...
llvm-svn: 343643
2018-10-02 22:31:44 +00:00
ac98683423
[analyzer] [tests] Allow specifying entire -analyze-config on the command line, make sure it's always propagated
...
Differential Revision: https://reviews.llvm.org/D52801
llvm-svn: 343636
2018-10-02 21:19:23 +00:00
47e5493b1d
[analyzer] [testing] Pass through an extra argument for specifying extra analyzer options
...
Differential Revision: https://reviews.llvm.org/D52585
llvm-svn: 343158
2018-09-27 01:10:59 +00:00
95363e378a
[analyzer] Remove traces of ubigraph visualization
...
Ubigraph project has been dead since about 2008, and to the best of my
knowledge, no one was using it.
Previously, I wasn't able to launch the existing binary at all.
Differential Revision: https://reviews.llvm.org/D51655
llvm-svn: 341601
2018-09-06 23:07:47 +00:00
1f5fe1132e
[analyzer] [tests] Do not be verbose by default when updating reference results.
...
llvm-svn: 339183
2018-08-07 21:14:35 +00:00
8f6d65c685
[analyzer] [tests] Add an option for showing statistics after running tests.
...
Do not show statistics by default.
llvm-svn: 338323
2018-07-30 23:01:47 +00:00
13d3748e7c
[analyzer] [tests] Style fixes for testing harness.
...
llvm-svn: 338322
2018-07-30 23:01:20 +00:00
6e4ddf49d8
[analyzer] [tests] Pass clang executable path to prefix-less executor scripts.
...
llvm-svn: 336124
2018-07-02 17:10:40 +00:00
5c23d6ab3c
[analyzer] [tests] Allow the tested project to specify it's own analyzer wrapper
...
llvm-svn: 336023
2018-06-29 22:05:32 +00:00
a262cf3146
[analyzer] [tests] Fix 80 column violation in SATestBuild.py
...
llvm-svn: 336022
2018-06-29 22:05:13 +00:00
afc62b7032
[analyzer] Fix string not being formatted with extra arguments
...
Signed-off-by: Mikhail Ramalho <mikhail.ramalho@gmail.com>
llvm-svn: 335739
2018-06-27 14:39:41 +00:00
30130b78c5
[analyzer] [tests] Include statistics in tests.
...
llvm-svn: 335685
2018-06-26 23:17:35 +00:00
690a99a5fb
Fix a (possible) division by zero check in the CmpRuns script
...
I missed updating the check in r333375
llvm-svn: 333529
2018-05-30 11:17:55 +00:00
8af2e690f3
Introduces --stats-only option to only show changes in statistics.
...
llvm-svn: 333375
2018-05-28 15:40:39 +00:00
711058165c
[analyzer] [testing] Be less verbose by default in integration testing.
...
llvm-svn: 328752
2018-03-29 01:23:54 +00:00
dc0abcb87b
[analyzer] [tests] Create a directory for the log file
...
llvm-svn: 326408
2018-03-01 02:18:54 +00:00
3f560daf37
[analyzer] [tests] Write to logfile instead of stdout while updating
...
reference results
llvm-svn: 326295
2018-02-28 01:55:23 +00:00
b7120c944b
[analyzer] [tests] Update CmpRuns to write to stdout correctly in multithreaded environment
...
llvm-svn: 325070
2018-02-13 23:36:01 +00:00
192d9a186f
[analyzer] [tests] Fix a typo in analyzer testing script.
...
Incorrect option instance construction.
llvm-svn: 324946
2018-02-12 22:13:01 +00:00
3959041d4e
[analyzer] [tests] Fixing an error after non-atomic cherry-pick
...
llvm-svn: 324762
2018-02-09 18:48:31 +00:00
fc782a341a
[analyzer] [tests] [NFC] Remove a fragile tightly-coupled component emulating parser output
...
...when we can just use the real parser instead.
Differential Revision: https://reviews.llvm.org/D43098
llvm-svn: 324759
2018-02-09 18:39:47 +00:00
f37d3a5f84
[analyzer] [tests] Test different projects concurrently
...
Differential Revision: https://reviews.llvm.org/D43031
llvm-svn: 324652
2018-02-08 21:22:42 +00:00
986dd45a78
[analyzer] [tests] Show function name in CmpRuns output
...
Combined with enabled flag for stable filenames, this greatly simplifies
finding the offending report.
Differential Revision: https://reviews.llvm.org/D42831
llvm-svn: 324362
2018-02-06 17:22:09 +00:00
78809e53cc
[analyzer] [tests] Fix crash in SATestBuild.py
...
llvm-svn: 324114
2018-02-02 18:27:14 +00:00
f37c07c313
[analyzer] [tests] [NFC] Remove dead code from CmpRuns
...
Indeed, "CHANGE" is not a thing yet, and we should probably not carry
around dead code which does not do anything apart from confusing the
reader.
Differential Revision: https://reviews.llvm.org/D42819
llvm-svn: 324027
2018-02-01 22:40:01 +00:00
b7043222bf
[analyzer] [tests] Add an option to show the histogram of path differences between the analyzer runs
...
Differential Revision: https://reviews.llvm.org/D42778
llvm-svn: 324021
2018-02-01 22:25:18 +00:00
dece62a772
[analyzer] [tests] Show the number of removed/added bug reports
...
Differential Revision: https://reviews.llvm.org/D42718
llvm-svn: 323941
2018-02-01 02:38:42 +00:00
9ed25edc66
[analyzer] Use stable filenames in analyzer testing infrastructure
...
Makes finding the right file in test results easier.
Differential Revision: https://reviews.llvm.org/D42445
llvm-svn: 323697
2018-01-29 21:45:07 +00:00
56939f7e75
[NFC] fix trivial typos in comments
...
"the the" -> "the"
llvm-svn: 323078
2018-01-22 07:44:38 +00:00
3c128cb1e6
[analyzer] [tests] Remove empty folders in reference results, do not store diffs.txt
...
Storing diffs.txt is now redundant, as we simply dump the CmpRuns output
to stdout (it is saved in CI and tends to be small).
Not generating those files enables us to remove empty folders, which
confuse git, as it would not add them with reference results.
llvm-svn: 316948
2017-10-30 19:40:33 +00:00
0a6dba792f
[Analyzer] [Tests] Fixing typo from the previous commit.
...
Can not open a non-existent file with r+.
llvm-svn: 316808
2017-10-27 22:52:36 +00:00
e58044d432
[Analyzer] [Tests] Dump the output of scan-build to stdout on failure.
...
Eliminates extra lookup step during debugging.
llvm-svn: 316806
2017-10-27 22:39:54 +00:00
ff555ce798
[Analyzer] [Tests] Write analyzers crashes to stdout, and not to a separate file
...
With this change it would be sufficient to look at CI console to see the
failure.
llvm-svn: 316687
2017-10-26 19:00:22 +00:00
65839bd429
[Analyzer] [Tests] Consistently use exit codes. Use code=42 to signify different results
...
llvm-svn: 316632
2017-10-26 01:13:22 +00:00
a932c8745c
[Analyzer] [Tests] Do not discard output from CmpRuns.py when running integration tests
...
Contrary to the deleted comment, in most cases CmpRuns.py produces a
fairly small amount of output, which is useful to see straight away to
see what has changed when executing the integration tests.
llvm-svn: 316618
2017-10-25 21:49:46 +00:00
bf92c44616
[Analyzer] [Tests] Minor refactor of testing infrastructure:
...
Move utilities functions into a separate file to make comprehension
easier.
llvm-svn: 316535
2017-10-24 23:52:48 +00:00
318cd1f263
[Analyzer] [Tests] Remove temporary fields from generated reference results.
...
Pointer to HTML diagnostics is removed (as it is not stored) as well as
the version (as it would be available from the commit message).
llvm-svn: 316534
2017-10-24 23:52:46 +00:00
43f683cb33
[Analyzer] Fix bug in testing scripts, which always marked result as failure.
...
llvm-svn: 316522
2017-10-24 22:24:13 +00:00
7d36cd7223
[Analyzer] Remove dead code from testing scripts
...
Differential Revision: https://reviews.llvm.org/D38488
llvm-svn: 315489
2017-10-11 18:42:39 +00:00
1b51cbddf4
[Analyzer Tests] Run static analyzer integration tests until the end,
...
Do not stop at the first failure.
Differential Revision: https://reviews.llvm.org/D38589
llvm-svn: 314992
2017-10-05 17:32:06 +00:00
9944babb65
[Analyzer Tests] Fix misc bugs in analyzer reference results updater.
...
llvm-svn: 314960
2017-10-05 01:02:20 +00:00
George Karpenkov