Commit Graph

25 Commits

Author SHA1 Message Date
Michal Gorny 929ce27cc7 [test] Skip ThinLTO cache tests requiring atime setting on NetBSD
Skip the ThinLTO cache tests on NetBSD.  They require 'touch' being
able to alter atime of files, while NetBSD inhibits atime updates
when filesystem is mounted noatime.

Differential Revision: https://reviews.llvm.org/D55273

llvm-svn: 348355
2018-12-05 11:15:50 +00:00
James Henderson 99031b79a6 [ThinLTO]Expose cache entry expiration time option in llvm-lto and fix a test
Two cases in a ThinLTO test were passing for the wrong reasons, since
rL340374. The tests were supposed to be testing that files were being
pruned due to the cache size, but they were in fact being pruned because
they were older than the default expiration period of 1 week.

This change fixes the tests by explicitly setting the expiration time to
the maximum value. This required the option to be exposed in llvm-lto.

By assigning all files in the cache a similar time, it is possible to see
that the newest files are still being kept, and that we aren't passing
for the wrong reason again. In the event that the entry expiration were
to expire for them, then the test would start failing, because these
files would be removed too.

Reviewed by: rnk, inglorion

Differential Revision: https://reviews.llvm.org/D51992

llvm-svn: 343687
2018-10-03 13:00:20 +00:00
James Henderson e29e40854b Reland r342233: [ThinLTO] Allow setting of maximum cache size with 64-bit number
The original was reverted due to an apparent build-bot test failure,
but it looks like this is just a flaky test.

Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.

The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.

We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.

Reviewed by: mehdi_amini, tejohnson, steven_wu

Differential Revision: https://reviews.llvm.org/D52023

llvm-svn: 342366
2018-09-17 10:21:26 +00:00
James Henderson 13f426304f Revert r342233.
This caused LLD test failures, which I've been unable to reproduce.

Reverting to allow for further investigation next week.

llvm-svn: 342244
2018-09-14 16:48:47 +00:00
James Henderson 48c0688a36 [ThinLTO]Allow setting of maximum cache size with 64-bit number
Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.

The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.

We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.

Reviewed by: mehdi_amini, tejohnson, steven_wu

Differential Revision: https://reviews.llvm.org/D52023

llvm-svn: 342233
2018-09-14 12:51:19 +00:00
Stella Stamenova 79a40ebba5 [lit, python] Remove quotes around %python in cache.ll
Summary: We needed quotes around %python before to make python work correctly (on Windows) if the path contains spaces. I recently made a change so that %python now inherently has quotes, so now adding quotes around %python makes the test fail because the quotes cancel each other.

Reviewers: asmith, inglorion

Subscribers: mehdi_amini, eraman, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51244

llvm-svn: 340753
2018-08-27 16:33:13 +00:00
Bob Haarman 481d224b67 [Support][CachePruning] prune least recently accessed files first
Summary:
Before this change, pruning order was based on size. This changes it
to be based on time of last use instead, preferring to keep recently
used files and prune older ones.

Reviewers: pcc, rnk, espindola

Reviewed By: rnk

Subscribers: emaste, arichardson, hiraditya, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51062

llvm-svn: 340374
2018-08-22 00:52:16 +00:00
Stella Stamenova cc2404c01d [lit, python] Always add quotes around the python path in lit
Summary:
The issue with the python path is that the path to python on Windows can contain spaces. To make the tests always work, the path to python needs to be surrounded by quotes.

This change updates several configuration files which specify the path to python as a substitution and also remove quotes from existing tests.

Reviewers: asmith, zturner, alexshap, jakehehrlich

Reviewed By: zturner, alexshap, jakehehrlich

Subscribers: mehdi_amini, nemanjai, eraman, kbarton, jakehehrlich, steven_wu, dexonsmith, stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50206

llvm-svn: 339073
2018-08-06 22:37:44 +00:00
Andrew Ng 089303d8ff [ThinLTO] Update ThinLTO cache file atimes when on Windows
ThinLTO cache file access times are used for expiration based pruning
and since Vista, file access times are not updated by Windows by
default:

https://blogs.technet.microsoft.com/filecab/2006/11/07/disabling-last-access-time-in-windows-vista-to-improve-ntfs-performance

This means on Windows, cache files are currently being pruned from
creation time. This change manually updates cache files that are
accessed by ThinLTO, when on Windows.

Patch by Owen Reynolds.

Differential Revision: https://reviews.llvm.org/D47266

llvm-svn: 336276
2018-07-04 14:17:10 +00:00
Aaron Smith e5ee89c0d3 [lit] Fix several Python 2/3 compatibility issues and tests
- In Python 2.x, basestring is the base string type, but in 
  Python 3.x basestring is not defined and instead str includes 
  unicode strings.

- When Python is in a path that includes spaces, it needs to 
  be specified with quotes in the test files for it to run.

- The cache.ll test relies on files of a specific size being 
  created by Python, but on some versions of Windows the 
  files that are created by the current code are one byte 
  larger than expected. To fix the test, update file creation 
  to always make files of the expected size.

Patch by Stella Stamenova!

llvm-svn: 329466
2018-04-07 00:21:28 +00:00
Ekaterina Romanova b8aeec46b7 [ThinLTO] Added a couple of C LTO API interfaces to control the cache policy.
- thinlto_codegen_set_cache_size_bytes to control the absolute size of cache directory. 
- thinlto_codegen_set_cache_size_files the size and amount of files in cache directory. 
These functions have been supported in C++ LTO API for a long time, but were absent in C LTO API.

Differential Revision: https://reviews.llvm.org/D42446

llvm-svn: 326537
2018-03-02 03:51:27 +00:00
Ekaterina Romanova d345f73939 Allow 0 to be a valid value pruning interval in C LTO API. Value 0 will cause garbage collector to run. This matches the behavior in C++ LTO API.
llvm-svn: 325303
2018-02-15 23:29:21 +00:00
Ben Dunbobbin 9ecb8b548c [Support][CachePruning] Disable cache pruning regression fix
borked by: rL284966 (see: https://reviews.llvm.org/D25730).

Previously, Interval was unsigned (see: CachePruning.h), replacing the type with std::chrono::seconds (which is signed) causes a regression in behaviour because the c-api intends negative values to translate to large positive intervals to *effectively* disable the pruning (see comments on: setCachePruningInterval()).

Differential Revision: https://reviews.llvm.org/D41231

llvm-svn: 321077
2017-12-19 14:42:38 +00:00
Peter Collingbourne 7faa60c406 llvm-lto2: Move the LTO::run() action behind a subcommand.
Move LTO::run() to a "run" subcommand so that we can introduce new subcommands
for testing different parts of the LTO implementation.

This doesn't use llvm::cl subcommands because it doesn't appear to be currently
possible to pass an argument not associated with a subcommand to a subcommand
(e.g. -lto-use-new-pm, -mcpu=yonah).

Differential Revision: https://reviews.llvm.org/D31410

llvm-svn: 299967
2017-04-11 18:12:00 +00:00
Peter Collingbourne 25a17ba4c7 Support, LTO: When pruning a directory, ignore files matching a prefix.
This is a safeguard against data loss if the user specifies a directory
that is not a cache directory. Teach the existing cache pruning clients
to create files with appropriate names.

Differential Revision: https://reviews.llvm.org/D31109

llvm-svn: 298271
2017-03-20 16:41:57 +00:00
Peter Collingbourne ab76a19afb LTO: When creating a local cache, create the cache directory if it does not already exist.
Differential Revision: https://reviews.llvm.org/D30519

llvm-svn: 296726
2017-03-02 02:02:38 +00:00
Mehdi Amini f82bda0a7a ThinLTO: don't perform incremental LTO on module without a hash
Clang always emit a hash for ThinLTO, but as other frontend are
starting to use ThinLTO, this could be a serious bug.

Differential Revision: https://reviews.llvm.org/D25379

llvm-svn: 283655
2016-10-08 04:44:23 +00:00
Mehdi Amini adc0e26bef [ThinLTO] Add caching to the new LTO API
Add the ability to plug a cache on the LTO API.
I tried to write such that a linker implementation can
control the cache backend. This is intrusive and I'm
not totally happy with it, but I can't figure out a
better design right now.

Differential Revision: https://reviews.llvm.org/D23599

llvm-svn: 279576
2016-08-23 21:30:12 +00:00
NAKAMURA Takumi 2c8500996d Re-enable llvm/test/ThinLTO/X86/cache.ll.
This reverts;
  r269548, "XFAIL ThinLTO Caching test on Windows."
  r269561, "Rework r269548, "XFAIL ThinLTO Caching test on Windows.", not to use XFAIL, for now."

llvm-svn: 269567
2016-05-14 14:28:17 +00:00
NAKAMURA Takumi c50a2a93ae Rework r269548, "XFAIL ThinLTO Caching test on Windows.", not to use XFAIL, for now.
It was passing (and is XPASSing) with --host=linux --target=win32.

llvm-svn: 269561
2016-05-14 12:47:40 +00:00
Mehdi Amini 66862c2797 XFAIL ThinLTO Caching test on Windows.
I have no idea what's going on on Windows here.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269548
2016-05-14 05:38:58 +00:00
Mehdi Amini ab4a8b6ca3 Add testing in llvm-lto for ThinLTO caching.
Trying to improve code coverage for `make check`

From: mehdi_amini <mehdi_amini@91177308-0d34-0410-b5e6-96231b3b80d8>
llvm-svn: 269545
2016-05-14 05:16:41 +00:00
Mehdi Amini 34b0241b81 Revert "Add testing in llvm-lto for ThinLTO caching."
This reverts commit r269538 and r269542.
"rename()" is expected to fail across filesystems, will handle this.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269543
2016-05-14 05:07:44 +00:00
Mehdi Amini e19c794741 Increase verbosity in the test output to help debugging windows issues
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269542
2016-05-14 05:01:36 +00:00
Mehdi Amini dec0e54d58 Add testing in llvm-lto for ThinLTO caching.
Trying to improve code coverage for `make check`

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269538
2016-05-14 04:41:26 +00:00