Commit Graph

45 Commits

Author SHA1 Message Date
Valeriy Savchenko c98872e3a3 [analyzer] Modernize analyzer's Python scripts
Summary:
Fix read/write in binary format, which crashes Python 3.
Additionally, clean up redundant (as for Python 3) code and
fix a handful of flake8 warnings.

Differential Revision: https://reviews.llvm.org/D79932
2020-05-14 15:55:37 +03:00
George Karpenkov a64b205ae7 [analyzer] [testing] Inside CmpRuns.py output also print the filename of the first item in the path
Differential Revision: https://reviews.llvm.org/D57783

llvm-svn: 353228
2019-02-05 22:26:57 +00:00
Serge Guelton d458974c45 Portable Python script across Python version
In Python3, dict.items, dict.keys, dict.values, zip, map and filter no longer return lists, they create generator instead.

The portability patch consists in forcing an extra `list` call if the result is actually used as a list.
`map` are replaced by list comprehension and `filter` by filtered list comprehension.

Differential Revision: https://reviews.llvm.org/D55197

llvm-svn: 349501
2018-12-18 16:04:21 +00:00
Serge Guelton 3744de522c Portable Python script across Python version
In Python2, division between integer yields an integer, while it yields a float in Python3.
Use a combination of from __future__ import division and // operator to get a portable behavior.

Differential Revision: https://reviews.llvm.org/D55204

llvm-svn: 349455
2018-12-18 08:38:50 +00:00
Serge Guelton c0ebe773cd Portable Python script across Python version
Using from __future__ import print_function it is possible to have a compatible behavior of `print(...)` across Python version.

Differential Revision: https://reviews.llvm.org/D55213

llvm-svn: 349454
2018-12-18 08:36:33 +00:00
Serge Guelton 09616bdb4a Portable Python script across version
Have all classes derive from object: that's implicitly the default in Python3,
it needs to be done explicilty in Python2.

Differential Revision: https://reviews.llvm.org/D55121

llvm-svn: 348127
2018-12-03 12:12:48 +00:00
George Karpenkov 6a2a197f95 [analyzer] [testing] Compute data on path length, compute percentiles
Differential Revision: https://reviews.llvm.org/D52844

llvm-svn: 344990
2018-10-23 01:30:26 +00:00
George Karpenkov 8f6d65c685 [analyzer] [tests] Add an option for showing statistics after running tests.
Do not show statistics by default.

llvm-svn: 338323
2018-07-30 23:01:47 +00:00
George Karpenkov 13d3748e7c [analyzer] [tests] Style fixes for testing harness.
llvm-svn: 338322
2018-07-30 23:01:20 +00:00
Mikhail R. Gadelha 690a99a5fb Fix a (possible) division by zero check in the CmpRuns script
I missed updating the check in r333375

llvm-svn: 333529
2018-05-30 11:17:55 +00:00
Mikhail R. Gadelha 8af2e690f3 Introduces --stats-only option to only show changes in statistics.
llvm-svn: 333375
2018-05-28 15:40:39 +00:00
George Karpenkov b7120c944b [analyzer] [tests] Update CmpRuns to write to stdout correctly in multithreaded environment
llvm-svn: 325070
2018-02-13 23:36:01 +00:00
George Karpenkov 3959041d4e [analyzer] [tests] Fixing an error after non-atomic cherry-pick
llvm-svn: 324762
2018-02-09 18:48:31 +00:00
George Karpenkov fc782a341a [analyzer] [tests] [NFC] Remove a fragile tightly-coupled component emulating parser output
...when we can just use the real parser instead.

Differential Revision: https://reviews.llvm.org/D43098

llvm-svn: 324759
2018-02-09 18:39:47 +00:00
George Karpenkov 986dd45a78 [analyzer] [tests] Show function name in CmpRuns output
Combined with enabled flag for stable filenames, this greatly simplifies
finding the offending report.

Differential Revision: https://reviews.llvm.org/D42831

llvm-svn: 324362
2018-02-06 17:22:09 +00:00
George Karpenkov 78809e53cc [analyzer] [tests] Fix crash in SATestBuild.py
llvm-svn: 324114
2018-02-02 18:27:14 +00:00
George Karpenkov f37c07c313 [analyzer] [tests] [NFC] Remove dead code from CmpRuns
Indeed, "CHANGE" is not a thing yet, and we should probably not carry
around dead code which does not do anything apart from confusing the
reader.

Differential Revision: https://reviews.llvm.org/D42819

llvm-svn: 324027
2018-02-01 22:40:01 +00:00
George Karpenkov b7043222bf [analyzer] [tests] Add an option to show the histogram of path differences between the analyzer runs
Differential Revision: https://reviews.llvm.org/D42778

llvm-svn: 324021
2018-02-01 22:25:18 +00:00
George Karpenkov dece62a772 [analyzer] [tests] Show the number of removed/added bug reports
Differential Revision: https://reviews.llvm.org/D42718

llvm-svn: 323941
2018-02-01 02:38:42 +00:00
George Karpenkov a807660918 [Analyzer] Make testing scripts flake8 compliant
Differential Review: https://reviews.llvm.org/D38213

llvm-svn: 314692
2017-10-02 17:59:12 +00:00
George Karpenkov 3abfc3b0d7 [Analyzer] Refactor analyzer testing scripts.
- Exporting needed function for future reuse.
 - Idiomatic python: using with `file as f` instead of `try/finally`.
 - Fixing some indentation issues.
 - No need to reinvent python `multiprocessing.getCPUCount()`
 - Removing a function parameter which is always the same under all invocations.
 - Adding some docstrings.

Differential Revision: https://reviews.llvm.org/D38156

llvm-svn: 313949
2017-09-22 01:41:16 +00:00
George Karpenkov 7a256839d9 [Analyzer] Remove dead code from CmpRuns.py.
Differential Revision: https://reviews.llvm.org/D38003

llvm-svn: 313923
2017-09-21 21:47:13 +00:00
Gabor Horvath efec16307c [analyzer] Bug identification
This patch adds hashes to the plist and html output to be able to identfy bugs
for suppressing false positives or diff results against a baseline. This hash
aims to be resilient for code evolution and is usable to identify bugs in two
different snapshots of the same software. One missing piece however is a 
permanent unique identifier of the checker that produces the warning. Once that
issue is resolved, the hashes generated are going to change. Until that point
this feature is marked experimental, but it is suitable for early adoption.

Differential Revision: http://reviews.llvm.org/D10305 

Original patch by: Bence Babati!

llvm-svn: 251011
2015-10-22 11:53:04 +00:00
Ted Kremenek 3a0678e33c [analyzer] Apply whitespace cleanups by Honggyu Kim.
llvm-svn: 246978
2015-09-08 03:50:52 +00:00
Gabor Horvath c3177f2ed4 [Static Analyzer] Make the paths relative from the project root when generating reference results in the test suite.
llvm-svn: 241706
2015-07-08 18:39:31 +00:00
Gabor Horvath 93fde94d81 [Static Analyzer] Minor improvements to SATest.
Differential Revision: http://reviews.llvm.org/D10812

llvm-svn: 241073
2015-06-30 15:31:17 +00:00
Jordan Rose b042cc7822 [analyzer] CmpRuns.py: Accept single files as input.
This allows us to compare two direct invocations of the analyzer on a
single source file without having to wrap the output plists in their
own directories.

llvm-svn: 177804
2013-03-23 01:21:26 +00:00
Anna Zaks fab9bb6890 [analyzer] CmpRuns: move the clang_version info into the run (out of
report)

llvm-svn: 168105
2012-11-15 22:42:44 +00:00
Anna Zaks 9821e57c09 [analyzer] Ensure that CmpRuns recursively walks the output directory.
llvm-svn: 167981
2012-11-14 21:32:16 +00:00
Anna Zaks 639b40484e [analyzer] CmpRuns: cleanup APIs, allow processing of plists with no
clang_version, fix the compare routine.

llvm-svn: 166129
2012-10-17 21:09:26 +00:00
Anna Zaks 93283d0933 [analyzer] Expose clang_version in CmpRuns
llvm-svn: 166048
2012-10-16 19:36:39 +00:00
Anna Zaks c80313b888 [analyzer] CmpRuns - cleaned up the API, added relative path getter
llvm-svn: 165995
2012-10-15 22:48:21 +00:00
Anna Zaks e12365bc27 [analyzer] Ted fixed the plist-html issue, so the assert can go away.
llvm-svn: 162598
2012-08-24 21:07:49 +00:00
Anna Zaks af3ae3cbbd [analyzer] Unbreak the static analyzer buildbot.
We no longer associate the correct HTML file with plist diagnostic.
Fall out from r162028.

llvm-svn: 162071
2012-08-17 01:09:13 +00:00
Anna Zaks 45a992b5f0 [analyzer] CmpRuns should include file name in the issue identifier.
This prevents us from treating the issues from different files with the
same function names and same offsets as the same.

The issue identifier now includes the file name. Also added a way to
strip off the root directories form the source file names.

llvm-svn: 161150
2012-08-02 00:41:40 +00:00
Jordan Rose 5920a1bdd1 Un-break standalone invocations of CmpRuns.py.
Fallout from CmpRuns.py API changes in r160314.

llvm-svn: 160800
2012-07-26 20:03:51 +00:00
Anna Zaks 9b7d7140ef [analyzer] Make CmpRuns external-user friendly.
CmpRuns can be used for static analyzer bug report comparison. However,
we want to make sure external users do not rely on the way bugs are
represented (plist files). Make sure that we have a user
friendly/documented API for CmpRuns script.

llvm-svn: 160314
2012-07-16 20:21:42 +00:00
Anna Zaks d60367b893 [analyzer] Use "issue hash" in CmpRuns; followup on r158180
(For the future: It would be more efficient to produce a hash key with
the embedded function info inside the compiler.)

llvm-svn: 158187
2012-06-08 01:50:49 +00:00
Anna Zaks b3d896d333 [analyzer] Add the HTML file to the SATest diagnostic diff.
(Uses the functionality which has been in CmpRuns long before.)

llvm-svn: 148868
2012-01-24 21:57:35 +00:00
Anna Zaks 895c312404 [analyzer] Testing: make diagnostic diffs more informative (add
diagnostic message).

llvm-svn: 148867
2012-01-24 21:57:32 +00:00
Anna Zaks 767d356f87 [analyzer] Testing: Display the number of reported differences in the main log.
llvm-svn: 144113
2011-11-08 19:56:31 +00:00
Anna Zaks 84726dd72c [analyzer] Relax the cmp criteria for analyzer reports.
Until we find out a way to easily find out what changed by looking at the logs.

llvm-svn: 143789
2011-11-05 05:20:56 +00:00
Anna Zaks ea8f5725c5 [analyzer] CmpRuns can now optionally delete empty reports.
llvm-svn: 139550
2011-09-12 22:40:36 +00:00
Anna Zaks b80d836137 [analyzer] CmpRuns.cmpScanBuildResults() should be easy to call from other modules.
llvm-svn: 139543
2011-09-12 21:32:41 +00:00
Anna Zaks 03f3598ff1 Rename CmpRuns into CmpRuns.py so that it could be used as a module.
llvm-svn: 139540
2011-09-12 21:07:18 +00:00