Commit Graph

143 Commits

Author SHA1 Message Date
Nico Weber 87d4ea2433 [gn build] Make an explicit `use_lld = true` on mac use lld.darwinnew
use_lld defaults to true on non-mac if clang_base_path is set (i.e.
the host compiler is a locally-built clang). On mac, the lld Mach-O
port used to be unusable, but ld64.lld.darwinnew is close to usable.
When explicitly setting `use_lld = true` in a GN build on a mac host,
check-lld passes, two check-clang tests fail, and a handful check-llvm
tests fail (the latter all due to -flat_namespace not yet being implemented).
2021-01-09 14:03:52 -05:00
Nico Weber 51a292d994 [gn build] Switch copy_bundle_data from pax to cpio
This will hopefully fix the build not becoming clean when using Ninja
1.9+. Ninja 1.9 enabled high-resolution time stamps, but pax doesn't
correctly set high-resolution timestamps on its output.

See https://github.com/nico/hack/blob/master/notes/copydir.md for a
detailed writeup of problem and alternatives.
2020-12-30 13:59:03 -05:00
Nico Weber 7e33fd9ce2 [gn build] Link with -Wl,--gdb-index when linking with LLD
For full-debug-info (is_debug=true / symbol_level=2 builds), this makes
linking 15% slower, but gdb startup 1500% faster (for lld: link time
3.9s->4.4s, gdb load time >30s->2s).

For link time, I ran

    bench.py -o {noindex,index}.txt \
        sh -c 'rm out/gn/bin/lld && ninja -C out/gn lld'

and then `ministat noindex.txt index.txt`:

```
x noindex.txt
+ index.txt
    N           Min           Max        Median           Avg        Stddev
x   5      3.784461     4.0200169     3.8452811     3.8754988   0.089902595
+   5       4.32496     4.6058481     4.3361208     4.4141198    0.12288267
Difference at 95.0% confidence
	0.538621 +/- 0.15702
	13.8981% +/- 4.05161%
	(Student's t, pooled s = 0.107663)
```

For gdb load time I loaded the crash in PR48392 with

    gdb -ex r --args ../out/gn/bin/ld64.lld.darwinnew @response.txt

and just stopped the time until the crash got displayed with a stopwatch
a few times. So the speedup there is less precise, but it's so
pronounced that that's ok (loads ~instantly with the patch, takes a very
long time without it).

Only doing this for LLD because I haven't tried it with other linkers.

Differential Revision: https://reviews.llvm.org/D92844
2020-12-17 15:39:00 -05:00
Arthur Eubanks c1f30e5817 [gn build] Add symbol_level to adjust debug info level
is_debug by default makes symbol_level = 2 and !is_debug means by
default symbol_level = 0.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D92958
2020-12-17 09:20:53 -08:00
Nico Weber cacb1a9f93 [gn build] sync script: try to make sync script even clearer
Turns out startswith() takes an optional start parameter :)

No behavior change.
2020-12-01 14:35:27 -05:00
Nico Weber 43afba0677 [gn build] sync script: try to make a loop clearer
We want to find 'sources = [', but only if it's not the prefix
of 'sources = []' -- we're looking for a non-empty list.

No behavior change.
2020-11-25 21:16:22 -05:00
Nico Weber 12c2646feb [gn build] sync script: only compute tokloc when adding files
No behavior change, but maybe a bit clearer.
2020-11-25 21:05:13 -05:00
Nico Weber 48d1017245 [gn build] sync script: handle multiple source lists if only deleting
With this, changes like b534beabee can be merged automatically.
2020-11-25 20:57:55 -05:00
Nico Weber 9194aa8867 [gn build] modify hotfix in 17df195f70 to work with all enabled targets
llvm-config output with the gn build is just good enough to make
tests pass, but llvm-config wants all .a files it knows about to
actually exist. So let it know about fewer .a files that don't
exist if not all targets are enabled.
2020-11-23 21:28:54 -05:00
Arthur Eubanks 67e0f791c9 [gn build] Use forward slashes for goma directory
gn generates improper compile_commands.json files by not escaping
backslashes.
2020-11-17 17:10:36 -08:00
Nico Weber ffaa859b72 [gn build] more hotfix after 17df195f70 to unbreak llvm-config tests 2020-11-13 10:03:05 -05:00
Nico Weber 17df195f70 [gn build] Hotfix to unbreak build after 9218ff50f9 2020-11-13 09:26:43 -05:00
Nico Weber 4a96b2e75f [gn build] Add libclang_rt.ios.a, libclang_rt.iossim.a to the build
It's built with just-built clang, like all other compiler-rt parts
in the GN build.

This requires adding some cross build support to the mac toolchain.

Also add explicit mmacosx-version-min and miphoneos-version-min
flags to the build.

ios.a is only built with the arm64 slice, iossim.a only with the
x86_64 slice for now. (The latter should maybe become host_cpu
when Arm Macs become a common iOS development platform.)

With this, it's possible to build chromium/iOS with a GN-built LLVM.

Differential Revision: https://reviews.llvm.org/D89260
2020-10-12 16:21:04 -04:00
Arthur Eubanks b29573b672 [gn build] Support building with ThinLTO
Differential Revision: https://reviews.llvm.org/D88584
2020-10-01 13:48:31 -07:00
Arthur Eubanks 11a75e6c92 [gn build] Allow option to build with asan/tsan/ubsan
Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D88056
2020-09-23 11:24:38 -07:00
Nico Weber 9b346f974e [gn build] Do not sync filenames containing variable references 2020-09-18 15:34:33 -04:00
Nico Weber 15c378f6e6 [gn build] unconfuse sync script about "sources = []" in clang/lib/Headers/BUILD.gn 2020-09-16 14:50:29 -04:00
Arthur Eubanks a1caa30297 [gn build] Add support for expensive checks
Reviewed By: hans, MaskRay

Differential Revision: https://reviews.llvm.org/D86007
2020-08-18 09:53:39 -07:00
Mark Mentovai 92d5839297 [gn build] mac: use frameworks instead of libs where appropriate
As of GN 3028c6a426a4, the hack that transformed "libs" ending in
".framework" from -l arguments to -framework arguments has been removed.
Instead, "frameworks" must be used, and the toolchain must provide
support.

Differential Revision: https://reviews.llvm.org/D84219
2020-08-06 18:58:57 -04:00
Nico Weber b6902d977a [gn build] Make syncing to single-line source files work after 1afd889d0b 2020-07-27 12:44:47 -04:00
Nico Weber 1afd889d0b [gn build] Make sync_source_lists_from_cmake handle one-line sources lines
sync_source_lists_from_cmake now also looks for source files in
`sources += [ "foo.cc" ]` lines, which allows us to remove most
`# Make `gn format` not collapse this` comments.

(sync_source_lists_from_cmake doesn't look for `foo_headers += [...]`
still, so the comment is still needed in two places for that.)

No intentional behavior change.
2020-07-17 11:53:42 -04:00
Nico Weber 90a35bd074 [gn build] fix link of libclang_rt.asan_osx_dynamic.dylib if command line tools are not installed 2020-07-04 20:26:39 -04:00
Nico Weber 05c5ff5ab0 [gn build] make stage2_unix_toolchain set clang_base_path
This fixes the build of compiler-rt on macOS when _not_ using
clang_base_path in args.gn: Xcode clang knows where to find the
SDK, but regular clang doesn't and needs a -isysroot parameter.
We correctly add that parameter when clang_base_path is set,
but else we omit it. If clang_base_path was not set, we also
didn't add the flag for stage2_unix_toolchain() when we build
compiler-rt with just-built clang.

Make stage2_unix_toolchain() use clang_base_path instead of setting
cc / cxx. It's less code, and it gets things like this right.
2020-07-04 19:36:09 -04:00
Mitch Phillips e26b25f8b1 [HWASan] Add sizeof(global) in report even if symbols missing.
Summary: Refactor the current global header iteration to be callback-based, and add a feature that reports the size of the global variable during reporting. This allows binaries without symbols to still report the size of the global variable, which is always available in the HWASan globals PT_NOTE metadata.

Reviewers: eugenis, pcc

Reviewed By: pcc

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D80599
2020-06-09 13:02:13 -07:00
Mitch Phillips 9bca45bd45 Rebase. 2020-06-09 13:01:40 -07:00
Mitch Phillips 2ecf32fb35 remove redundant comment about Android. 2020-06-09 13:01:40 -07:00
Mitch Phillips 1bfb5b8e36 Address Peter's comments. 2020-06-09 13:01:40 -07:00
Mitch Phillips 184b437699 Move DSO dependencies inside the group. 2020-06-09 13:01:40 -07:00
Mitch Phillips 9e9142cbb9 Patch up issues with GN builds (pthread / libz)
Summary:
Fixes up two small issues with the gn build.

 1 - Ensures that the correct ldflag `-pthread` is provided, not just linking the library.
 2 - Ensures that libraries are linked in the same group as the dependencies. This fixes a problem where system libraries (libc) are involved in a link-order dependency that's not being fulfilled.

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80591
2020-06-09 13:01:40 -07:00
Nico Weber 3871ad366b [gn build] port some -Wno flags for gcc from the cmake build. 2020-06-04 08:10:29 -04:00
Nico Weber a946cb1813 [gn build] Fix link with ld.bfd and gold.
lld's symbol resolution algorithm makes it not depend on
the order of object files and libraries, but ld.bfd and
gold require listing dependencies later on the link line.
Put {{libs}} after {{inputs}} so that e.g. -lpthreads
appears after the object files, not before it.

Differential Revision: https://reviews.llvm.org/D81035
2020-06-03 17:06:30 -04:00
Nico Weber 1502941c1c [gn build] Try harder to unbreak Windows build after f8e833a501. 2020-05-19 17:14:08 -04:00
Nico Weber f8cccd126b [gn build] Try to unbreak Windows build after f8e833a501. 2020-05-19 16:14:04 -04:00
Sébastien Marchand f8e833a501 [gn build] Add a flag zlib_path to provide the path to zlib on Windows.
Also, automatically set llvm_enable_zlib to true when zlib_path is set.

Differential Revision: https://reviews.llvm.org/D80042
2020-05-19 16:00:54 -04:00
Nico Weber 4ce375a8f2 [gn build] minor tweaks to sync script
- only complain about duplicate 'sources' lines if they contain
  more than 1 elements each (before, 0 and 1 element lists were
  counted too). In practice, this only has an effect for
  clang/lib/Headers/BUILD.gn

- make the '# NOSORT' diag actually work. This too only affects
  clang/lib/Headers/BUILD.gn.

In aggregate, changes to clang/lib/Headers/BUILD.gn still can't
be auto-merged, but for a slighly better reason.
2020-03-30 22:07:23 -04:00
Nico Weber a235215415 fix a comment grammar-o 2020-03-30 14:40:15 -04:00
Reid Kleckner d75be4c06a [gn] Use ghash if using clang & LLD together to make PDBs
I noticed my links were a bit slower on Windows than usual.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D75774
2020-03-09 15:23:54 -07:00
Nico Weber 187204cd74 [gn build] use \bfoo\b instead of \<foo\> in sync script
\<foo\> is more correct, but since we use shell=True on Windows,
the < and > get interpreted as redirection operators.

Rather than adding cmd escaping, just use \bfoo\b, which is Good
Enough Often Enough.
2020-02-19 12:32:02 -05:00
Nico Weber bde0e320bc [gn build] revert e8e078c8bf
Now that I've updated ancient goma clients on the bots, this should
work. (Internal goma bug: b/139410332, fixed months ago.)
2020-02-19 11:11:25 -05:00
Nico Weber e8e078c8bf [gn build] use -Xclang form for fdebug-comp-dir for now
The driver version of the flag seems to confuse goma.
2020-02-16 21:30:27 -05:00
Nico Weber 37d1120572 [gn build] unbreak win build by removing flags that only work with a sysroot 2020-02-15 06:25:21 -05:00
Nico Weber ead0b76382 [gn build] Make build locally deterministic
This follows http://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html
to make the GN build locally deterministic.

With this, I've built lld at two different build paths on my Windows box and got
identical binaries. (I'd expect the same to happen on Linux, and with other
binaries.)

This doesn't have the bits to get universal determinism yet.

Differential Revision: https://reviews.llvm.org/D74519
2020-02-14 21:55:33 -05:00
Nico Weber 09153ab9d2 [windows] Add /Gw to compiler flags
This is like -fdata-sections, and it's not part of /O2 by default for some reason.

In the cmake build, reduces the size of clang.exe from 70,358,016 bytes to 69,982,720 bytes.

clang-format.exe goes from 3,703,296 bytes to 3,331,072 bytes.

Differential Revision: https://reviews.llvm.org/D74573
2020-02-14 15:15:00 -05:00
Nico Weber 0bd6be9c3d [gn build] Fix sync script on renames like "Foo.cpp" -> "LLVMFoo.cpp"
Before, the script used `git log -SFoo.cpp` to find a commit where
the number of occurrences of "Foo.cpp" changed -- but since
a patch with

  + LLVMFoo.cpp
  - Foo.cpp

contains the same number of instances of "Foo.cpp", the script
incorrectly skipped this type of rename.

As fix, look for '\bFoo\.cpp\b' instead and pass --pickaxe-regex
so that we can grep for word boundaries.

To test, check out 7531a5039f (which renamed in llvm/lib/IR
RemarkStreamer.cpp to LLVMRemarkStreamer.cpp) and look at the output of
the script.  Before this change, it correctly assigned the addition
of LLVMRemarkStreamer.cpp to 7531a5039f but incorrectly assigned
the removal of RemarkStreamer.cpp to b8a847c.  With this, it
correctly assigns both to 7531a5039f.
2020-02-13 09:26:47 -05:00
Nico Weber 442d8e7a91 [gn build] add a FIXME about using /Gw on win 2020-01-29 19:12:08 -05:00
Nico Weber bfcfa53c3d Revert "[gn build] [win] produce symbolized stack frames in release builds too"
This reverts commit fd98eccf98.
Seems to have no effect, need to try it locally for a bit first.
2020-01-22 12:54:19 -05:00
Nico Weber fd98eccf98 [gn build] [win] produce symbolized stack frames in release builds too 2020-01-22 12:36:38 -05:00
Nico Weber 1645f5e496 [gn build] replace llvm_allow_tardy_revision with llvm_append_vc_rev
Previously, the gn build would create VCSRevision.h / VCSVersion.h
files with some LLD_REVISION / LLVM_REVISION / CLANG_REVISION but
by default wouldn't add a dependency on .git/logs/HEAD so that
the step doesn't rerun after every branch switch or every pull.

That's bad for deterministic builds, and having --version print
some arbitrarily old revision isn't great either.

Instead, move to the model that the cmake build (now) uses fairly
consistently: If llvm_append_vc_rev is set, include the revision,
else don't.

Since the GN build is focused on developers, set llvm_append_vc_rev
to false instead of true by default (different from the cmake build),
so that things don't rebuild after every branch switch and every
pull.

While here, also remove some pre-monorepo code.

Differential Revision: https://reviews.llvm.org/D72859
2020-01-16 19:05:07 -05:00
Nico Weber 5caa121295 [gn build] include revision information in lld --version output 2020-01-16 13:10:41 -05:00
Nico Weber 527281a843 [gn build] re-run "gn format" with trunk gn 2020-01-15 13:37:38 -05:00