Combined with enabled flag for stable filenames, this greatly simplifies
finding the offending report.
Differential Revision: https://reviews.llvm.org/D42831
llvm-svn: 324362
Indeed, "CHANGE" is not a thing yet, and we should probably not carry
around dead code which does not do anything apart from confusing the
reader.
Differential Revision: https://reviews.llvm.org/D42819
llvm-svn: 324027
Storing diffs.txt is now redundant, as we simply dump the CmpRuns output
to stdout (it is saved in CI and tends to be small).
Not generating those files enables us to remove empty folders, which
confuse git, as it would not add them with reference results.
llvm-svn: 316948
Contrary to the deleted comment, in most cases CmpRuns.py produces a
fairly small amount of output, which is useful to see straight away to
see what has changed when executing the integration tests.
llvm-svn: 316618
Pointer to HTML diagnostics is removed (as it is not stored) as well as
the version (as it would be available from the commit message).
llvm-svn: 316534
- Exporting needed function for future reuse.
- Idiomatic python: using with `file as f` instead of `try/finally`.
- Fixing some indentation issues.
- No need to reinvent python `multiprocessing.getCPUCount()`
- Removing a function parameter which is always the same under all invocations.
- Adding some docstrings.
Differential Revision: https://reviews.llvm.org/D38156
llvm-svn: 313949
This change is required to easily test the given checkout of the analyzer,
rather than the one bundled with a system compiler.
Differential Revision: https://reviews.llvm.org/D38155
llvm-svn: 313927
Treat lines in projectMap.csv that start with '#' as comments. This enables a
workflow where projects can be temporarily disabled with a comment describing
when they should be turned back on.
Differential Revision: https://reviews.llvm.org/D24709
llvm-svn: 281880
The html reports are huge -- every issue in a given file results in a separate
copy of the source code, in HTML form, for the file. This gets very large
quickly and it doesn't make sense to check this into a reference repository.
Also remove the log when generating reference results because it can leak
absolute path names. We still keep both the html and the log around when
producing non-reference results.
llvm-svn: 258594
Currently the SATestBuild.py and SATestAdd.py buildbot scripts expect project
sources to be checked into the project repository. This commit changes these
scripts to additionally support a model where project sources are downloaded
rather than checked into the repository. Sometimes projects may need to be
modified (for example, to support a newer versions of clang), so the updated scripts
also allow for an optional patch file that will be applied to the downloaded
project source before analysis.
To support this workflow, this commit changes the expected layout of
a project in the repository. The project-specific helper scripts will stay
in the root of each project directory, but the benchmark source itself (if
checked into the repo) should now be stored in a subdirectory named
'CachedSource':
project_name/
cleanup_run_static_analyzer.sh [optional]
run_static_analyzer.cmd [required]
download_project.sh [optional]
CachedSource/ [optional]
changes_for_analyzer.patch [optional]
If the 'CachedSource' source directory is not present, the download script will
be executed. This script should download the project source into 'CachedSource'.
Then, if 'changes_for_analyzer.patch' is present its changes will
be applied to a copy of 'CachedSource' before analysis.
Differential Revision: http://reviews.llvm.org/D14345
llvm-svn: 252410
This patch adds hashes to the plist and html output to be able to identfy bugs
for suppressing false positives or diff results against a baseline. This hash
aims to be resilient for code evolution and is usable to identify bugs in two
different snapshots of the same software. One missing piece however is a
permanent unique identifier of the checker that produces the warning. Once that
issue is resolved, the hashes generated are going to change. Until that point
this feature is marked experimental, but it is suitable for early adoption.
Differential Revision: http://reviews.llvm.org/D10305
Original patch by: Bence Babati!
llvm-svn: 251011
Move the logic looking for additional checkers in the SA_ADDITIONAL_CHECKERS
environmental variable from SATestBuild's main() to runScanBuild(). This allows
SATestAdd.py to use the variable as well. Without it, we won't include
additional checkers when building reference results for the build bot.
Differential Revision: http://reviews.llvm.org/D12891
llvm-svn: 247767
Update the static analyzer buildbot script to set -isysroot to the OS X SDK path
when analyzing preprocessed files on OS X.
Differential Revision: http://reviews.llvm.org/D12769
llvm-svn: 247617
When running a make-based command, SATestBuild tries to append a -jN flag
with an appropriate N to run the build in parallel. However, it failed
to take into account that each line read includes a trailing newline
(unless it is the last line of a file without a trailing newline), which
resulted in the "-jN" appearing on a line on its own.
llvm-svn: 190164
The Logs directory isn't used for testing, so it's filtered out ahead of
time. However, there's then no reason to include it in version control at
all. Don't error if it's not present.
llvm-svn: 183689
As mentioned in the previous commit message, the use-after-free and
double-free warnings for 'delete' are worth enabling even while the
leak warnings still have false positives.
llvm-svn: 178891
This allows us to compare two direct invocations of the analyzer on a
single source file without having to wrap the output plists in their
own directories.
llvm-svn: 177804
SATestBuild expects to compare output directories for each invocation of
scan-build that it runs, but scan-build clears out empty directories by
default. We were coincidentally not getting that behavior until r173294.
llvm-svn: 173383
Before, SATestBuild unilaterally added '-j<n>' to every project built with
'make'. Now, we check and see if there's a -j option already specified, which
allows a project to explicitly be marked '-j1'.
llvm-svn: 168603
This has been broken for a while, but the branch was never being taken.
(We were trying to do 'str + floatVal'; now we do 'str % intVal' and use
the '%d' format.)
llvm-svn: 168174
This prevents us from treating the issues from different files with the
same function names and same offsets as the same.
The issue identifier now includes the file name. Also added a way to
strip off the root directories form the source file names.
llvm-svn: 161150
CmpRuns can be used for static analyzer bug report comparison. However,
we want to make sure external users do not rely on the way bugs are
represented (plist files). Make sure that we have a user
friendly/documented API for CmpRuns script.
llvm-svn: 160314
Rename the pre_run_static_analyzer script into cleanup_run_static_analyzer
and use it to clean up after the reference build - which is done as part of
adding a new project to the system.
llvm-svn: 143568
George Karpenkov