95363e378a
[analyzer] Remove traces of ubigraph visualization
...
Ubigraph project has been dead since about 2008, and to the best of my
knowledge, no one was using it.
Previously, I wasn't able to launch the existing binary at all.
Differential Revision: https://reviews.llvm.org/D51655
llvm-svn: 341601
2018-09-06 23:07:47 +00:00
1f5fe1132e
[analyzer] [tests] Do not be verbose by default when updating reference results.
...
llvm-svn: 339183
2018-08-07 21:14:35 +00:00
8f6d65c685
[analyzer] [tests] Add an option for showing statistics after running tests.
...
Do not show statistics by default.
llvm-svn: 338323
2018-07-30 23:01:47 +00:00
13d3748e7c
[analyzer] [tests] Style fixes for testing harness.
...
llvm-svn: 338322
2018-07-30 23:01:20 +00:00
6e4ddf49d8
[analyzer] [tests] Pass clang executable path to prefix-less executor scripts.
...
llvm-svn: 336124
2018-07-02 17:10:40 +00:00
5c23d6ab3c
[analyzer] [tests] Allow the tested project to specify it's own analyzer wrapper
...
llvm-svn: 336023
2018-06-29 22:05:32 +00:00
a262cf3146
[analyzer] [tests] Fix 80 column violation in SATestBuild.py
...
llvm-svn: 336022
2018-06-29 22:05:13 +00:00
afc62b7032
[analyzer] Fix string not being formatted with extra arguments
...
Signed-off-by: Mikhail Ramalho <mikhail.ramalho@gmail.com>
llvm-svn: 335739
2018-06-27 14:39:41 +00:00
30130b78c5
[analyzer] [tests] Include statistics in tests.
...
llvm-svn: 335685
2018-06-26 23:17:35 +00:00
690a99a5fb
Fix a (possible) division by zero check in the CmpRuns script
...
I missed updating the check in r333375
llvm-svn: 333529
2018-05-30 11:17:55 +00:00
8af2e690f3
Introduces --stats-only option to only show changes in statistics.
...
llvm-svn: 333375
2018-05-28 15:40:39 +00:00
711058165c
[analyzer] [testing] Be less verbose by default in integration testing.
...
llvm-svn: 328752
2018-03-29 01:23:54 +00:00
dc0abcb87b
[analyzer] [tests] Create a directory for the log file
...
llvm-svn: 326408
2018-03-01 02:18:54 +00:00
3f560daf37
[analyzer] [tests] Write to logfile instead of stdout while updating
...
reference results
llvm-svn: 326295
2018-02-28 01:55:23 +00:00
b7120c944b
[analyzer] [tests] Update CmpRuns to write to stdout correctly in multithreaded environment
...
llvm-svn: 325070
2018-02-13 23:36:01 +00:00
192d9a186f
[analyzer] [tests] Fix a typo in analyzer testing script.
...
Incorrect option instance construction.
llvm-svn: 324946
2018-02-12 22:13:01 +00:00
3959041d4e
[analyzer] [tests] Fixing an error after non-atomic cherry-pick
...
llvm-svn: 324762
2018-02-09 18:48:31 +00:00
fc782a341a
[analyzer] [tests] [NFC] Remove a fragile tightly-coupled component emulating parser output
...
...when we can just use the real parser instead.
Differential Revision: https://reviews.llvm.org/D43098
llvm-svn: 324759
2018-02-09 18:39:47 +00:00
f37d3a5f84
[analyzer] [tests] Test different projects concurrently
...
Differential Revision: https://reviews.llvm.org/D43031
llvm-svn: 324652
2018-02-08 21:22:42 +00:00
986dd45a78
[analyzer] [tests] Show function name in CmpRuns output
...
Combined with enabled flag for stable filenames, this greatly simplifies
finding the offending report.
Differential Revision: https://reviews.llvm.org/D42831
llvm-svn: 324362
2018-02-06 17:22:09 +00:00
78809e53cc
[analyzer] [tests] Fix crash in SATestBuild.py
...
llvm-svn: 324114
2018-02-02 18:27:14 +00:00
f37c07c313
[analyzer] [tests] [NFC] Remove dead code from CmpRuns
...
Indeed, "CHANGE" is not a thing yet, and we should probably not carry
around dead code which does not do anything apart from confusing the
reader.
Differential Revision: https://reviews.llvm.org/D42819
llvm-svn: 324027
2018-02-01 22:40:01 +00:00
b7043222bf
[analyzer] [tests] Add an option to show the histogram of path differences between the analyzer runs
...
Differential Revision: https://reviews.llvm.org/D42778
llvm-svn: 324021
2018-02-01 22:25:18 +00:00
dece62a772
[analyzer] [tests] Show the number of removed/added bug reports
...
Differential Revision: https://reviews.llvm.org/D42718
llvm-svn: 323941
2018-02-01 02:38:42 +00:00
9ed25edc66
[analyzer] Use stable filenames in analyzer testing infrastructure
...
Makes finding the right file in test results easier.
Differential Revision: https://reviews.llvm.org/D42445
llvm-svn: 323697
2018-01-29 21:45:07 +00:00
56939f7e75
[NFC] fix trivial typos in comments
...
"the the" -> "the"
llvm-svn: 323078
2018-01-22 07:44:38 +00:00
3c128cb1e6
[analyzer] [tests] Remove empty folders in reference results, do not store diffs.txt
...
Storing diffs.txt is now redundant, as we simply dump the CmpRuns output
to stdout (it is saved in CI and tends to be small).
Not generating those files enables us to remove empty folders, which
confuse git, as it would not add them with reference results.
llvm-svn: 316948
2017-10-30 19:40:33 +00:00
0a6dba792f
[Analyzer] [Tests] Fixing typo from the previous commit.
...
Can not open a non-existent file with r+.
llvm-svn: 316808
2017-10-27 22:52:36 +00:00
e58044d432
[Analyzer] [Tests] Dump the output of scan-build to stdout on failure.
...
Eliminates extra lookup step during debugging.
llvm-svn: 316806
2017-10-27 22:39:54 +00:00
ff555ce798
[Analyzer] [Tests] Write analyzers crashes to stdout, and not to a separate file
...
With this change it would be sufficient to look at CI console to see the
failure.
llvm-svn: 316687
2017-10-26 19:00:22 +00:00
65839bd429
[Analyzer] [Tests] Consistently use exit codes. Use code=42 to signify different results
...
llvm-svn: 316632
2017-10-26 01:13:22 +00:00
a932c8745c
[Analyzer] [Tests] Do not discard output from CmpRuns.py when running integration tests
...
Contrary to the deleted comment, in most cases CmpRuns.py produces a
fairly small amount of output, which is useful to see straight away to
see what has changed when executing the integration tests.
llvm-svn: 316618
2017-10-25 21:49:46 +00:00
bf92c44616
[Analyzer] [Tests] Minor refactor of testing infrastructure:
...
Move utilities functions into a separate file to make comprehension
easier.
llvm-svn: 316535
2017-10-24 23:52:48 +00:00
318cd1f263
[Analyzer] [Tests] Remove temporary fields from generated reference results.
...
Pointer to HTML diagnostics is removed (as it is not stored) as well as
the version (as it would be available from the commit message).
llvm-svn: 316534
2017-10-24 23:52:46 +00:00
43f683cb33
[Analyzer] Fix bug in testing scripts, which always marked result as failure.
...
llvm-svn: 316522
2017-10-24 22:24:13 +00:00
7d36cd7223
[Analyzer] Remove dead code from testing scripts
...
Differential Revision: https://reviews.llvm.org/D38488
llvm-svn: 315489
2017-10-11 18:42:39 +00:00
1b51cbddf4
[Analyzer Tests] Run static analyzer integration tests until the end,
...
Do not stop at the first failure.
Differential Revision: https://reviews.llvm.org/D38589
llvm-svn: 314992
2017-10-05 17:32:06 +00:00
9944babb65
[Analyzer Tests] Fix misc bugs in analyzer reference results updater.
...
llvm-svn: 314960
2017-10-05 01:02:20 +00:00
a807660918
[Analyzer] Make testing scripts flake8 compliant
...
Differential Review: https://reviews.llvm.org/D38213
llvm-svn: 314692
2017-10-02 17:59:12 +00:00
af76b4a553
[Analyzer] Add nullability to the list of tested checkers in SATestBuild
...
Differential Revision: https://reviews.llvm.org/D38162
llvm-svn: 314572
2017-09-30 00:05:24 +00:00
439a95bde7
[Analyzer] Fix minor errors in python scripts.
...
llvm-svn: 314042
2017-09-22 22:58:46 +00:00
aff33385d2
[Analyzer] Static analyzer script for updating reference results
...
The script updates reference results from the previous run, and stages them.
Differential Revision: https://reviews.llvm.org/D38157
llvm-svn: 313950
2017-09-22 01:43:12 +00:00
3abfc3b0d7
[Analyzer] Refactor analyzer testing scripts.
...
- Exporting needed function for future reuse.
- Idiomatic python: using with `file as f` instead of `try/finally`.
- Fixing some indentation issues.
- No need to reinvent python `multiprocessing.getCPUCount()`
- Removing a function parameter which is always the same under all invocations.
- Adding some docstrings.
Differential Revision: https://reviews.llvm.org/D38156
llvm-svn: 313949
2017-09-22 01:41:16 +00:00
be6c329515
[Analyzer] Use CC environment variable to select analyzer path in SATestBuild.
...
This change is required to easily test the given checkout of the analyzer,
rather than the one bundled with a system compiler.
Differential Revision: https://reviews.llvm.org/D38155
llvm-svn: 313927
2017-09-21 22:12:49 +00:00
2d155092cb
[Analyzer] Add simple help to SATestAdd.py
...
Differential Revision: https://reviews.llvm.org/D38003
llvm-svn: 313924
2017-09-21 21:47:33 +00:00
7a256839d9
[Analyzer] Remove dead code from CmpRuns.py.
...
Differential Revision: https://reviews.llvm.org/D38003
llvm-svn: 313923
2017-09-21 21:47:13 +00:00
0dfc6f0809
[analyzer] SATestBuild.py: Treat '#' as comment in projectMap.csv
...
Treat lines in projectMap.csv that start with '#' as comments. This enables a
workflow where projects can be temporarily disabled with a comment describing
when they should be turned back on.
Differential Revision: https://reviews.llvm.org/D24709
llvm-svn: 281880
2016-09-19 01:36:40 +00:00
9ea8033d1b
[analyzer] SATestBuild.py: Remove html and log when producing reference results.
...
The html reports are huge -- every issue in a given file results in a separate
copy of the source code, in HTML form, for the file. This gets very large
quickly and it doesn't make sense to check this into a reference repository.
Also remove the log when generating reference results because it can leak
absolute path names. We still keep both the html and the log around when
producing non-reference results.
llvm-svn: 258594
2016-01-23 01:09:07 +00:00
86f61a9a62
[analyzer] SATestBuild.py: Handle spaces in path passed to --use-analyzer.
...
I missed this one in r258493.
llvm-svn: 258517
2016-01-22 18:45:22 +00:00
ab95cd2794
[analyzer] Update SATestBuild.py to handle spaces in paths.
...
The Jenkins workspace on the new Green Dragon builder for the static analyzer has spaces
in its path.
llvm-svn: 258493
2016-01-22 07:08:06 +00:00
George Karpenkov