Commit Graph

17 Commits

Author SHA1 Message Date
Nick Terrell 43ff634772 [Support] Optimize SHA1 implementation
* Add inline to the helper functions because gcc-9 won't inline all of
  them without the hint. I've avoided `__attribute__((always_inline))`
  because gcc and clang will inline without it, and improves
  compatibility.
* Replace the byte-by-byte copy in update() with endian::readbe32()
  since perf reports that 1/2 of the time is spent copying into the
  buffer before this patch.

When lld uses --build-id=sha1 it spends 30-45% of CPU in SHA1 depending on the binary (not wall-time since it is parallel). This patch speeds up SHA1 by a factor of 2 on clang-8 and 3 on gcc-6. This leads to a >10% improvement in overall linking time.

lld-speed-test benchmarks run on an Intel i9-9900k with Turbo disabled on CPU 0 compiled with clang-9. Stats recorded with `perf stat -r 5`. All inputs are using `--build-id=sha1`.

| Input | Before (seconds) | After (seconds) |
| --- | --- | --- |
| chrome | 2.14 | 1.82 (-15%) |
| chrome-icf | 2.56 | 2.29 (-10%) |
| clang | 0.65 | 0.53 (-18%) |
| clang-fsds | 0.69 | 0.58 (-16%) |
| clang-gdb-index | 21.71 | 19.3 (-11%) |
| gold | 0.42 | 0.34 (-19%) |
| gold-fsds | 0.431 | 0.355 (-17%) |
| linux-kernel | 0.625 | 0.575 (-8%) |
| llvm-as | 0.045 | 0.039 (-14%) |
| llvm-as-fsds | 0.035 | 0.039 (-11%) |
| mozilla | 11.3 | 9.8  (-13%) |
| mozilla-gc | 11.84 | 10.36 (-12%) |
| mozilla-O0 | 8.2 | 5.84 (-28%) |
| scylla | 5.59 | 4.52 (-19%) |

Reviewed By: ruiu, MaskRay

Differential Revision: https://reviews.llvm.org/D69295
2019-11-11 22:14:28 -08:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Eric Christopher 3390a8f63d Fix header comment on SHA1 code.
llvm-svn: 327086
2018-03-09 00:23:35 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Rui Ueyama c464fadc64 Fix builbots.
llvm-svn: 287735
2016-11-23 03:58:12 +00:00
Rui Ueyama 877c26c844 Add convenient functions to compute hashes of byte vectors.
In many sitautions, you just want to compute a hash for one chunk
of data. This patch adds convenient functions for that purpose.

Differential Revision: https://reviews.llvm.org/D26988

llvm-svn: 287726
2016-11-23 00:46:09 +00:00
Rui Ueyama e5669cecde Attempt to fix big-endian buildbots.
llvm-svn: 287476
2016-11-20 01:41:28 +00:00
Rui Ueyama 567d9c4b8f Style fix. NFC.
llvm-svn: 287475
2016-11-20 01:15:56 +00:00
Rui Ueyama 218072a989 Fix buildbot.
llvm-svn: 287474
2016-11-20 01:13:22 +00:00
Rui Ueyama fe33661ab0 SHA1: unroll loop in hashBlock.
This code is taken from public domain.
https://github.com/jsonn/src/blob/trunk/common/lib/libc/hash/sha1/sha1.c

I wrote a sha1 command and ran it on my Xeon E5-2680 v2 2.80GHz machine.
Here is a result. The new hash function is 37% faster than before.

 Performance counter stats for './llvm-sha1-old /ssd/build/bin/lld' (10 runs):

       6640.503687 task-clock (msec)         #    1.001 CPUs utilized            ( +-  0.03% )
                54 context-switches          #    0.008 K/sec                    ( +-  5.03% )
                 5 cpu-migrations            #    0.001 K/sec                    ( +- 31.73% )
           183,803 page-faults               #    0.028 M/sec                    ( +-  0.00% )
    18,527,954,113 cycles                    #    2.790 GHz                      ( +-  0.03% )
     4,993,237,485 stalled-cycles-frontend   #   26.95% frontend cycles idle     ( +-  0.11% )
   <not supported> stalled-cycles-backend
    50,217,149,423 instructions              #    2.71  insns per cycle
                                             #    0.10  stalled cycles per insn  ( +-  0.00% )
     6,094,322,337 branches                  #  917.750 M/sec                    ( +-  0.00% )
        11,778,239 branch-misses             #    0.19% of all branches          ( +-  0.01% )

       6.634017401 seconds time elapsed                                          ( +-  0.03% )

 Performance counter stats for './llvm-sha1-new /ssd/build/bin/lld' (10 runs):

       4167.062720 task-clock (msec)         #    1.001 CPUs utilized            ( +-  0.02% )
                52 context-switches          #    0.012 K/sec                    ( +- 16.45% )
                 7 cpu-migrations            #    0.002 K/sec                    ( +- 32.20% )
           183,804 page-faults               #    0.044 M/sec                    ( +-  0.00% )
    11,626,611,958 cycles                    #    2.790 GHz                      ( +-  0.02% )
     4,491,897,976 stalled-cycles-frontend   #   38.63% frontend cycles idle     ( +-  0.05% )
   <not supported> stalled-cycles-backend
    24,320,180,617 instructions              #    2.09  insns per cycle
                                             #    0.18  stalled cycles per insn  ( +-  0.00% )
     1,574,674,576 branches                  #  377.886 M/sec                    ( +-  0.00% )
        11,769,693 branch-misses             #    0.75% of all branches          ( +-  0.00% )

       4.163251552 seconds time elapsed                                          ( +-  0.02% )

Differential Revision: https://reviews.llvm.org/D26890

llvm-svn: 287473
2016-11-20 01:03:22 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Duncan P. N. Exon Smith 91d3cfed78 Revert "Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes."
This reverts commit r265454 since it broke the build.  E.g.:

  http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-incremental_build/22413/

llvm-svn: 265459
2016-04-05 20:45:04 +00:00
Eugene Zelenko 1760dc2a23 Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes.
Some Include What You Use suggestions were used too.

Use anonymous namespaces in source files.

Differential revision: http://reviews.llvm.org/D18778

llvm-svn: 265454
2016-04-05 20:19:49 +00:00
Mehdi Amini 180441f09a Fix S390 big endian detection
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265109
2016-04-01 05:12:24 +00:00
Mehdi Amini 4cd5702578 Add support for computing SHA1 in LLVM
Provide a class to generate a SHA1 from a sequence of bytes, and
a convenience raw_ostream adaptor.
This will be used to provide a "build-id" by hashing the Module
block when writing bitcode. ThinLTO will use this information for
incremental build.

Reapply r265094 which was reverted in r265102 because it broke
MSVC bots (constexpr is not supported).

http://reviews.llvm.org/D16325

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265107
2016-04-01 04:30:16 +00:00
Mehdi Amini 85fb9e058e Revert "Add support for computing SHA1 in LLVM"
This reverts commit r265096, r265095, and r265094.
Windows build is broken, and the validation does not pass.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265102
2016-04-01 03:03:21 +00:00
Mehdi Amini 3689ae14eb Add support for computing SHA1 in LLVM
Provide a class to generate a SHA1 from a sequence of bytes, and
a convenience raw_ostream adaptor.
This will be used to provide a "build-id" by hashing the Module
block when writing bitcode. ThinLTO will use this information for
incremental build.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265094
2016-04-01 01:29:54 +00:00