The Android sanitizer tests are currently some of the most difficult
to run correctly, requiring at least 3 build directories which have
to be configured in just the right way and built in the correct order
(see e.g. [1] and the functions that it calls).
This patch adds a check-hwasan target which greatly simplifies running
the hwasan tests for gn users, taking advantage of its support for
multiple toolchains. With this the tests can be run simply by setting
an NDK path and running "ninja check-hwasan" with a compatible Android
device connected. The Linux/x86_64 and Android/aarch64 targets are
tested in parallel.
[1] https://github.com/llvm/llvm-zorg/blob/master/zorg/buildbot/builders/sanitizers/buildbot_android.sh
Differential Revision: https://reviews.llvm.org/D56713
llvm-svn: 351277
The path to the resource directory will end up being used in several
more places once the support for running check-hwasan lands. This
moves the definition to a central location so that it can be used
from those places.
Differential Revision: https://reviews.llvm.org/D56700
llvm-svn: 351255
While here, add a use_lld flag and default it to true when using
clang on non-mac.
Differential Revision: https://reviews.llvm.org/D56710
llvm-svn: 351248
This makes e.g. ToolChain::isCrossCompiling() in
clang/lib/Driver/ToolChain.cpp return the correct result
if the compiler was cross-compiled. This change also affects
llvm_default_target_triple, so cross-compiled compilers default to
targeting the cross-compilation target, which makes more sense than
the host that the compiler was compiled on.
This change will also be necessary in order for the correct triples
to appear in generated lit files for non-native targets.
Differential Revision: https://reviews.llvm.org/D56696
llvm-svn: 351168
Add an asm tool (will be required for building sanitizer_common on
x64) and set a soname for DSOs so that anything that links against
them gets the correct DT_NEEDED.
Differential Revision: https://reviews.llvm.org/D56705
llvm-svn: 351167
Some of the sanitizer runtime code needs to be built with RTTI;
this allows that code to opt in to RTTI.
Differential Revision: https://reviews.llvm.org/D56627
llvm-svn: 351155
Since people weren't enthused about moving the .gn file to the toplevel in
D56419, here's a script to make gn at least somewhat more pleasant to invoke
(useful for gn clean, gn args --list, gn desc, etc).
Differential Revision: https://reviews.llvm.org/D56565
llvm-svn: 351064
This shortcut mechanism for creating types was added 10 years ago, but
has seen almost no uptake since then, neither internally nor in
external projects.
The very small number of characters saved by using it does not seem
worth the mental overhead of an additional type-creation API, so,
delete it.
Differential Revision: https://reviews.llvm.org/D56573
llvm-svn: 351020
I didn't break all that much during upstreaming, just needs two small fixes:
- fix spelling of MCJITTests.def file
- make libLTO a shared_library to put it in bin/ on Windows where it is in the
CMake build too
Differential Revision: https://reviews.llvm.org/D56630
llvm-svn: 351004
This makes it possible to build llvm-symbolizer for Android, which
is one of the prerequisites for running the sanitizer tests on Android.
Differential Revision: https://reviews.llvm.org/D56577
llvm-svn: 350979
Also change the toolchain description to use current_os instead of
host_os so that the template can be used for cross builds, and add
a current_os to the win toolchain to match the unix toolchain.
Differential Revision: https://reviews.llvm.org/D56576
llvm-svn: 350977
This makes the generated files go to the right place when using a non-default toolchain.
Differential Revision: https://reviews.llvm.org/D56427
llvm-svn: 350963
Most significantly, this makes bin/llvm-lit executable so that it
can be run in the usual way.
Differential Revision: https://reviews.llvm.org/D56423
llvm-svn: 350688
Convert the output of "git rev-parse --short HEAD" to a string before
substituting it into the output file. Without this the output file
will look like this on Python 3:
#define LLVM_REVISION "git-b'6a4895a025f'"
Differential Revision: https://reviews.llvm.org/D56459
llvm-svn: 350686
The PowerPC target itself is similar to the X86 target in https://reviews.llvm.org/rL348903
The llvm-exegesis unittests bits are similar to the corresponding AArch64 in https://reviews.llvm.org/rL350499
The whole patch is very similar to the WebAssembly target being added in https://reviews.llvm.org/rL350628
Also add a dep from tools/llvm-exegesis/lib to the AArch64 subdir, which I
failed to do in r350499.
The motivation for this target is solely that it has a unit test and I want to
enable the GN<->CMake unittest syncing check for llvm.
Differential Revision: https://reviews.llvm.org/D56416
llvm-svn: 350629
The WebAssembly target itself is similar to the X86 target in https://reviews.llvm.org/rL348903
The unittests bits are similar to the corresponding AArch64 in https://reviews.llvm.org/rL350499
The motivation for this target is solely that it has a unit test and I want to
enable the GN<->CMake unittest syncing check for llvm. (After this, only the
PowerPC target is needed and I can turn it on.)
Differential Revision: https://reviews.llvm.org/D56374
llvm-svn: 350628
The -o flag means something different to ar than what appears to be
intended here. Also, llvm-ar doesn't accept the flag in this position.
Differential Revision: https://reviews.llvm.org/D56426
llvm-svn: 350604
This is slightly ugly for three reasons:
- The shlib needs to go next to the binary to be found on all platforms, so the
build files refer to target_out_dir
- The explicit -fPIC flag needed on the shared lib side, and the -rdynamic flag
needed on the host side, on Linux
- Plugins that refer to LLVM code and assume that the host will resolve them
don't work on Windows -- PluginsTests won't test anything on Windows (but
DynamicLibraryTests will, since the dll here doesn't call LLVM code)
If we get lots more of these plugin / plugin host targets it might make sense
to add a template for them. But for now, these are the last ones we need.
(We're at 6 plugin hosts, 2 of them tests, and at 6 shared libraries, 2 of them
tests as well. clang is a plugin host by default in the CMake build but not
(yet?) in the GN build.)
Differential Revision: https://reviews.llvm.org/D56330
llvm-svn: 350454
I'm pretty unhappy this patch: DebugInfoPDBTests uses an API that requires some
magic txt file to be next to the unit test executable that stores the absolute
path to the LLVM source root.
The choices here are:
1. Don't use the unittest() template for DebugInfoPDBTests and set output_dir
for unit tests in two places (the gni file for every test but this one, and the
BUILD.gn file for this specific test).
2. Add another unittest_foo() template variation for this one test.
I went with the former, and added a comment to the template to look out for
this.
(The CMake build has the same issue.)
Differential Revision: https://reviews.llvm.org/D56324
llvm-svn: 350447
Adds build files for //llvm/unittest/[A-D].
Also teach sync_source_lists_from_cmake.py to not complain about missing
BUILD.gn files for CMakeLists.txt files that just call add_subdirectory()
without calling add_.+_unittest, like e.g.
llvm/unittests/Target/CMakeLists.txt.
(Omits CodeGen/GlobalISel and DebugInfo/PDB because their build files are somewhat interesting, and this patch is already on the larger side.)
Differential Revision: https://reviews.llvm.org/D56212
llvm-svn: 350411
With this, check-llvm runs and passes all of clang's lit tests. It doesn't run
any of its unit tests yet.
This is the only change in the GN build patch series that needs a change to a
file outside of llvm/utils/gn: llvm/test/tools/llvm-config/booleans.test checks
the result of llvm-config --build-system for some reason, so I'm updating the
test to accept "gn" as valid output in addition to "cmake". (The alternative
would be to let the gn build self-identify as cmake, which seems worse.)
Like with check-clang and check-lld, running just ninja -C out/gn will build
all prerequisites needed to run tests, but it won't run the tests (so that the
build becomes clean after one build). Running ninja -C out/gn check-llvm will
build prerequisites if needed and run the tests. The check-llvm target never
becomes clean and runs tests every time.
Differential Revision: https://reviews.llvm.org/D56195
llvm-svn: 350410
Not used by anything yet, but will be needed to make check-llvm run ld64's libLTO plugin tests.
Differential Revision: https://reviews.llvm.org/D56317
llvm-svn: 350409
Nothing pulls them in yet, but they will be needed for check-llvm.
LineEditor depends on libedit, so create a gn/build/lib for it, following the
usual pattern.
Differential Revision: https://reviews.llvm.org/D56316
llvm-svn: 350407
Also add a fuzzer() template for defining fuzzers that's similar to
add_llvm_fuzzer in the CMake build, and a build file for dependency
llvm/lib/FuzzMutate.
Also make `assert(defined(...` error strings a bit more self-consistent.
Differential Revision: https://reviews.llvm.org/D56194
llvm-svn: 350238
These two plugins are loaded into a host process that contains all LLVM
symbols, so they don't link against anything. This required minor readjustments
to the tablegen() setup of IR.
Needed for check-llvm.
Differential Revision: https://reviews.llvm.org/D56204
llvm-svn: 350234
Also add build files for dependencies llvm/lib/ExecutionEngine/{Interpreter,Orc}
Needed for check-llvm.
Differential Revision: https://reviews.llvm.org/D56193
llvm-svn: 350226
Needed for check-llvm.
This is the last target reading llvm_install_binutils_symlinks.
Differential Revision: https://reviews.llvm.org/D56190
llvm-svn: 350215
Also add build file for dependency llvm/lib/XRay.
Needed for check-llvm.
(yaml-bench is an llvm/util, not an llvm/tool.)
Differential Revision: https://reviews.llvm.org/D56163
llvm-svn: 350211
Also add build file for dependency llvm/lib/ExecutionEngine/MCJIT.
The exegesis stuff is pretty hairy and knows a lot about Target internals (in
general, not specifically in the GN build). I put the llvm-tblgen -gen-exegesis
call in llvm/tools/llvm-exegesis/lib/X86, instead of in llvm/lib/Target/X86
where it is in CMake land, and asked on D52932 why it's in that place in the
CMake build.
Needed for check-llvm.
Differential Revision: https://reviews.llvm.org/D56167
llvm-svn: 350184
Also add a build file for clang/lib/ASTMatchers/Dynamic, which is only needed
by tests (and clang/tools/extra).
Also make llvm/utils/gn/build/sync_source_lists_from_cmake.py check that every
CMakeLists.txt file below {lld,clang}/unittests has a corresponding BUILD.gn
file, so we notice if new test binaries get added (since the failure mode for
missing GN build files for tests is just the tests silently not running in the
GN build).
Also add a unittest() macro for defining unit test targets, and add a lengthy
comment there about where the unit test binaries go and why.
With this, the build files for //clang are complete.
Differential Revision: https://reviews.llvm.org/D56116
llvm-svn: 350171
With this, check-clang runs and passes all of clang's lit tests. It doesn't run
any of its unit tests yet.
Like with check-lld, running just ninja -C out/gn will build all prerequisites
needed to run tests, but it won't run the tests (so that the build becomes
clean after one build). Running ninja -C out/gn check-clang will build
prerequisites if needed and run the tests. The check-clang target never becomes
clean and runs tests every time.
Differential Revision: https://reviews.llvm.org/D56095
llvm-svn: 350108
GN wants the NOSORT line to be the first line of a comment block, not the last
line.
I sent https://gn-review.googlesource.com/c/gn/+/3560 to support having it in
the last line too, but since it will be a while until everyone has that change
even if it's expected, use the form that works today.
Differential Revision: https://reviews.llvm.org/D56065
llvm-svn: 350060
libclang is somewhat incomplete. It's just enough to get check-clang to pass,
but that requires it to be pretty complete. The biggest thing is that it's not
built as a shared library on Linux. The libclang/BUILD.gn file has a comment
with details on what else is missing.
Differential Revision: https://reviews.llvm.org/D56059
llvm-svn: 350054
Verified by comparing the output of `otool -P bin/clang` between the GN and the
CMake build.
Differential Revision: https://reviews.llvm.org/D55984
llvm-svn: 349992
The intent is to add the build file for clang/lib/StaticAnalyzer/Frontend; everything else is pulled in by that.
Differential Revision: https://reviews.llvm.org/D55978
llvm-svn: 349986
The check-lld target was missing the dependency on llvm-nm and llvm-objdump in that case.
Differential Revision: https://reviews.llvm.org/D55941
llvm-svn: 349836
Mostly boring, except for the spurious dependency on StaticAnalyzer/Checkers --
see comments in the code.
Differential Revision: https://reviews.llvm.org/D55927
llvm-svn: 349832
Nothing really interesting. One thing to consider is where the clang_tablegen()
invocations that generate files that are private to a library should be. The
CMake build puts them in clang/include/clang/Parse (in this case), but maybe
putting them right in clang/lib/Parse/BUILD.gn makes mor sense. (For
clang_tablegen() calls that generate .inc files used by the public headers,
putting the call in the public BUILD file makes sense.)
For now, I've put the build file in the public header folder, since that
matches CMake and what I did in the last 2 clang patches, but I'm not sure I
like this.
Differential Revision: https://reviews.llvm.org/D55925
llvm-svn: 349831
And add build files for gtest.
With this, the build files for //lld are complete.
Differential Revision: https://reviews.llvm.org/D55838
llvm-svn: 349704
Also add a build file for llvm-lit, which in turn needs llvm/tools/llvm-config.
With this, check-lld runs and passes all of lld's lit tests. It doesn't run any
of its unit tests yet.
Running just ninja -C out/gn will build all prerequisites needed to run tests,
but it won't run the tests (so that the build becomes clean after one build).
Running ninja -C out/gn check-lld will build prerequisites if needed and run
the tests. The check-lld target never becomes clean and runs tests every time.
llvm-config's build file is a bit gnarly: Everything not needed to run tests is
basically stubbed out. Also, to generate LibraryDependencies.inc we shell out
to llvm-build at build-time. It would be much nicer to get the library
dependencies by using the dependency data the GN build contains
(http://llvm-cs.pcc.me.uk/gen/tools/llvm-config/LibraryDependencies.inc#1).
Differential Revision: https://reviews.llvm.org/D55836
llvm-svn: 349702
Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.
Also adds an action to write include/clang/Config/config.h and
include/clang/Basic/Version.inc.
Differential Revision: https://reviews.llvm.org/D55847
(The previous commit of this contained unrelated changes, so I reverted the
whole previous commit and I'm now landing only what I intended to land.)
llvm-svn: 349679
Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.
Also adds an action to write include/clang/Config/config.h and
include/clang/Basic/Version.inc.
Differential Revision: https://reviews.llvm.org/D55847
llvm-svn: 349677
This relands r330742:
"""
Let TableGen write output only if it changed, instead of doing so in cmake.
Removes one subprocess and one temp file from the build for each tablegen
invocation.
No intended behavior change.
"""
In particular, if you see rebuilds after this change that you didn't see
before this change, that's unintended and it's fine to revert this change
again (but let me know).
r330742 got reverted because some people reported that llvm-tblgen ran on every
build after it. This could happen if the depfile output got deleted without
deleting the main .inc output. To fix, make TableGen always write the depfile,
but keep writing the main .inc output only if it has changed. This matches what
we did in cmake before.
Differential Revision: https://reviews.llvm.org/D55842
llvm-svn: 349624
Also add build files for deps DebugInfo/Symbolize, ToolDrivers/dll-tool.
Also add gn/build/libs/xar (needed by llvm-objdump).
Also delete an incorrect part of the symlink description in //BUILD.gn (it used
to be true before I made the symlink step write a stamp file; now it's no
longer true).
These are all binaries needed by check-lld that need symlinks.
Differential Revision: https://reviews.llvm.org/D55743
llvm-svn: 349486
This is slightly involved, see the comments in the code.
The GN build now builds a functional lld!
Differential Revision: https://reviews.llvm.org/D55606
llvm-svn: 349096
On Windows, we won't go into the `host_os != "win"` block, so `defines`
won't have been defined, and we'll run into an undefined identifier
error when we try to later append to it. Unconditionally define it at
the start and append to it everywhere else.
Differential Revision: https://reviews.llvm.org/D55617
llvm-svn: 348993
Version.inc.in processing has a potentially interesting part which I've punted
on for now (LLD_REVISION and LLD_REPOSITORY are set to empty strings for now).
lld now builds in the gn build. But no symlinks to it are created yet, so it
can't be meaningfully run yet.
Differential Revision: https://reviews.llvm.org/D55593
llvm-svn: 348945
The diff in targets.gni is due to me running `gn format` on all .gn and .gni
files.
llvm_enable_dia_sdk is in a gni file because I'm going to have to read it when
writing the lit invocations for check-llvm and check-lld. I've never had the
DIA sdk installed locally so I never tested building with it enabled -- it
probably doesn't Just Work and needs some path to diaguids.lib. We can finish
that once somebody needs it.
Differential Revision: https://reviews.llvm.org/D55591
llvm-svn: 348908
The tablegen setup for Target/X86 is a bit different from the CMake build: In
the CMake build, Target/X86/CMakeLists.txt has a single tablegen target that
does everything. But some of the generated files are only used privately by a
subproject, so in the GN build some of the tablegen invocations are
smaller-scoped, mostly for build cleanliness. (It helps also a tiny bit with
build parallelism since now e.g. the cpp files in MCTargetDesc can build after
just 3 .inc files are generated instead of being blocked on all 13. But it's
not a big win, since things depending on Target still need to wait for all 11,
even though all .inc file use is internal to lib/Target.)
Also add a build file for llc, since now all its dependencies have build files.
Differential Revision: https://reviews.llvm.org/D55524
llvm-svn: 348903
Tweak write_cmake_config.py to also handle variable references looking @FOO@
(matching CMake's configure_file() function), and make it replace '\' 'n' in
values with a newline literal since there's no good portable way of passing a
real newline literal on a command line.
Use that to process all the .def.in files in llvm/include/Config and add
llvm/lib/Target/BUILD.gn, which (indirectly, through llvm-c/Target.h) includes
them.
Differential Revision: https://reviews.llvm.org/D55184
llvm-svn: 348503
No behavior change, just makes the script match the other scripts in
llvm/utils/gn/build.
Differential Revision: https://reviews.llvm.org/D55183
llvm-svn: 348190
Before, #cmakedefine FOO resulted in #define FOO with a trailing space if FOO
was set to something truthy. Make it so that it's just #define FOO without a
trailing space.
No functional difference.
Differential Revision: https://reviews.llvm.org/D55172
llvm-svn: 348107
Before, the script had a bunch of special cases for #cmakedefine and
#cmakedefine01 and then did general variable substitution. Now, the script
always does general variable substitution for all lines and handles the special
cases afterwards.
This has no observable effect for the inputs we use, but is easier to explain
and slightly easier to implement.
Also mention to link to CMake's configure_file() in the docstring.
(The new behavior doesn't quite match CMake on lines like #cmakedefine ${FOO},
but nobody does that.)
Differential Revision: https://reviews.llvm.org/D55171
llvm-svn: 348106
Also adds a boring build file for llvm/lib/BinaryFormat (needed by llvm/lib/IR).
lib/IR marks Attributes and IntrinsicsEnum as public_deps (because IR's public
headers include the generated .inc files), so projects depending on lib/IR will
implicitly depend on them being generated. As a consequence, most targets won't
have to explicitly list a dependency on these tablegen steps (contrast with
intrinsics_gen in the cmake build).
This doesn't yet have the optimization where tablegen's output is only updated
if it's changed.
Differential Revision: https://reviews.llvm.org/D55028#inline-486755
llvm-svn: 347927
Also fix a missing file in lib/Support/BUILD.gn found by the script.
The script is very stupid and assumes that CMakeLists.txt follow the standard
LLVM CMakeLists.txt formatting with one cpp source file per line. Despite its
simplicity, it works well in practice.
It would be nice if it also checked deps and maybe automatically applied its
suggestions.
Differential Revision: https://reviews.llvm.org/D54930
llvm-svn: 347925
Adds build files for:
- llvm/lib/DebugInfo/CodeView
- llvm/lib/DebugInfo/MSF
- llvm/lib/MC
- llvm/lib/TableGen
- llvm/utils/TableGen
All the build files just list sources and deps and are uninteresting.
Differential Revision: https://reviews.llvm.org/D54931
llvm-svn: 347702
The comments at the top of
llvm/utils/gn/secondary/llvm/include/llvm/Config/BUILD.gn and
llvm/utils/gn/build/write_cmake_config.py should explain the main bits
happening in this patch. The main parts here are that these headers are
generated at build time, not gn time, and that currently they don't do any
actual feature checks but just hardcode most things based on the current OS,
which seems to work well enough. If this stops being enough, the feature checks
should each be their own action writing the result to somewhere, and the config
write step should depend on those checks (so that they can run in parallel and
as part of the build) -- utils/llvm/gn/README.rst already has some more words
on that in "Philosophy".
(write_cmake_config.py is also going to be used to write clang's
clang/include/clang/Config/config.h)
This also adds a few files for linking to system libraries in a consistent way
if needed in llvm/utils/gn/build/libs (and moves pthread to that model).0
I'm also adding llvm/utils/gn/secondary/llvm/lib/Target/targets.gni in this
patch because $native_arch is needed for writing llvm-config.h -- the rest of
it will be used later, when the build files for llvm/lib/Target get added. That
file describes how to select which archs to build.
As a demo, also add a build file for llvm-undname and make it the default build
target (it depends on everything that can currently be built).
Differential Revision: https://reviews.llvm.org/D54678
llvm-svn: 347636
See "GN build roundtable summary; adding GN build files to the repo" on
llvm-dev and cfe-dev for discussion.
In particular, this build is completely unsupported. People adding new files to
LLVM are not expected to update the GN build files, and reviewers are not
supposed to request the gn build files to be updated.
This adds just enough to be able to build llvm/lib/Demangle. It requires using
a monorepo.
This adds a few build config options you can set in args.gn
(`gn args out/foo --list` for all):
- is_debug = true to enable debug builds (defaults to release)
- llvm_enable_assertions to toggle assertions (defaults to true)
- clang_base_path, if set an absolute path to a locally-built clang to be used
as host compiler
Differential Revision: https://reviews.llvm.org/D54345
llvm-svn: 347128
Peter Collingbourne