Commit Graph

28 Commits

Author SHA1 Message Date
Simon Pilgrim a30f5fd94d CachePruning.h - reduce StringRef.h to Optional.h include. NFC
We only need to include Optional.h, forward declare StringRef and move the StringRef.h include down to CachePruning.cpp.
2020-05-08 14:06:53 +01:00
Fangrui Song d9b948b6eb Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC
F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800
2019-08-05 05:43:48 +00:00
Fangrui Song a0f9c4f72c [CachePruning] Simplify comparator
llvm-svn: 358843
2019-04-21 06:17:40 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Bob Haarman 481d224b67 [Support][CachePruning] prune least recently accessed files first
Summary:
Before this change, pruning order was based on size. This changes it
to be based on time of last use instead, preferring to keep recently
used files and prune older ones.

Reviewers: pcc, rnk, espindola

Reviewed By: rnk

Subscribers: emaste, arichardson, hiraditya, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51062

llvm-svn: 340374
2018-08-22 00:52:16 +00:00
Nicola Zaghen d34e60ca85 Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624

llvm-svn: 332240
2018-05-14 12:53:11 +00:00
Ben Dunbobbin bb534b15a9 [ThinLTO][CachePruning] explicitly disable pruning
In https://reviews.llvm.org/rL321077 and https://reviews.llvm.org/D41231 I fixed a regression in the c-api which prevented the pruning from being *effectively* disabled.

However this approach, helpfully recommended by @labath, is cleaner.
It is also nice to remove the weasel words about effectively disabling from the api comments.

Differential Revision: https://reviews.llvm.org/D41497

llvm-svn: 321376
2017-12-22 18:32:15 +00:00
Ben Dunbobbin 9ecb8b548c [Support][CachePruning] Disable cache pruning regression fix
borked by: rL284966 (see: https://reviews.llvm.org/D25730).

Previously, Interval was unsigned (see: CachePruning.h), replacing the type with std::chrono::seconds (which is signed) causes a regression in behaviour because the c-api intends negative values to translate to large positive intervals to *effectively* disable the pruning (see comments on: setCachePruningInterval()).

Differential Revision: https://reviews.llvm.org/D41231

llvm-svn: 321077
2017-12-19 14:42:38 +00:00
Peter Collingbourne 048ac83973 CachePruning: Allow limiting the number of files in the cache directory.
The default limit is 1000000 but it can be configured with a cache
policy. The motivation is that some filesystems (notably ext4) have
a limit on the number of files that can be contained in a directory
(separate from the inode limit).

Differential Revision: https://reviews.llvm.org/D40327

llvm-svn: 318857
2017-11-22 18:27:31 +00:00
Ben Dunbobbin cac52140bc [Support][CachePruning] Fix regression in pruning interval
Fixed broken comparison.
borked by: rL284966 (see: https://reviews.llvm.org/D25730).

Differential Revision: https://reviews.llvm.org/D40119

This is a second attempt to commit this.
The first attempt broke lld and gold tests that had been written against
the incorrect behaivour.

llvm-svn: 318524
2017-11-17 14:42:18 +00:00
Rafael Espindola ba3a87d898 Revert "[Support][CachePruning] Fix regression in pruning interval"
This reverts commit r318397.

It broke tools/gold/X86/cache.ll.

llvm-svn: 318419
2017-11-16 17:00:48 +00:00
Ben Dunbobbin 7f426869ba [Support][CachePruning] Fix regression in pruning interval
Fixed broken comparison.
borked by: rL284966 (see: https://reviews.llvm.org/D25730).

Differential Revision: https://reviews.llvm.org/D40119

llvm-svn: 318397
2017-11-16 13:15:56 +00:00
Peter Collingbourne 0dfdb44797 Support: Have directory_iterator::status() return FindFirstFileEx/FindNextFile results on Windows.
This allows clients to avoid an unnecessary fs::status() call on each
directory entry. Because the information returned by FindFirstFileEx
is a subset of the information returned by a regular status() call,
I needed to extract a base class from file_status that contains only
that information.

On my machine, this reduces the time required to enumerate a ThinLTO
cache directory containing 520k files from almost 4 minutes to less
than 2 seconds.

Differential Revision: https://reviews.llvm.org/D38716

llvm-svn: 315378
2017-10-10 22:19:46 +00:00
Peter Collingbourne 15ab1720c3 Fix a misleading indentation warning.
llvm-svn: 306130
2017-06-23 17:17:47 +00:00
Peter Collingbourne 30aaa2f3f6 Make the size specification for cache_size_bytes case insensitive.
llvm-svn: 306129
2017-06-23 17:13:51 +00:00
Peter Collingbourne 8d29223386 Add a ThinLTO cache policy for controlling the maximum cache size in bytes.
This is useful when an upper limit on the cache size needs to be
controlled independently of the amount of the amount of free space.

One use case is a machine with a large number of cache directories
(e.g. a buildbot slave hosting a large number of independent build
jobs). By imposing an upper size limit on each cache directory,
users can more easily estimate the server's capacity.

Differential Revision: https://reviews.llvm.org/D34547

llvm-svn: 306126
2017-06-23 17:05:03 +00:00
Peter Collingbourne 25a17ba4c7 Support, LTO: When pruning a directory, ignore files matching a prefix.
This is a safeguard against data loss if the user specifies a directory
that is not a cache directory. Teach the existing cache pruning clients
to create files with appropriate names.

Differential Revision: https://reviews.llvm.org/D31109

llvm-svn: 298271
2017-03-20 16:41:57 +00:00
Peter Collingbourne 255c6e1bae Fix pessimising moves.
llvm-svn: 297928
2017-03-16 03:54:38 +00:00
Peter Collingbourne d1eac7bc36 Support: Add a cache pruning policy parser.
The idea is that the policy string fully specifies the policy and is portable
between clients.

Differential Revision: https://reviews.llvm.org/D31020

llvm-svn: 297927
2017-03-16 03:42:00 +00:00
Peter Collingbourne cead56fb22 Support: Simplify the CachePruning API. NFCI.
Change the function that implements the pruning into a free function that
takes the policy as a struct argument.

Differential Revision: https://reviews.llvm.org/D31009

llvm-svn: 297907
2017-03-15 22:54:18 +00:00
Pavel Labath 757ca886cd Remove TimeValue usage from llvm/Support
Summary:
This is a follow-up to D25416. It removes all usages of TimeValue from
llvm/Support library (except for the actual TimeValue declaration), and replaces
them with appropriate usages of std::chrono. To facilitate this, I have added
small utility functions for converting time points and durations into appropriate
OS-specific types (FILETIME, struct timespec, ...).

Reviewers: zturner, mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25730

llvm-svn: 284966
2016-10-24 10:59:17 +00:00
Vassil Vassilev 2ec8b1506a Missing includes.
llvm-svn: 281450
2016-09-14 08:55:18 +00:00
Pawel Bylica 4cc795501f CachePruning: correct comment about file order. NFC
Summary: Actually the list of cached files is sorted by file size, not by last accessed time. Also remove unused file access time param for a helper function.

Reviewers: joker-eph, chandlerc, davide

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21639

llvm-svn: 273852
2016-06-27 08:46:23 +00:00
NAKAMURA Takumi f5e36ad863 CachePruning.cpp: Don't use errno.
llvm-svn: 269565
2016-05-14 14:21:39 +00:00
Mehdi Amini 721800d438 CachePruning: early exit if no path supplied
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266965
2016-04-21 06:43:45 +00:00
Mehdi Amini 045d4754f5 Add debugging to the cache pruning
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266686
2016-04-18 21:54:00 +00:00
Mehdi Amini e9a04ae141 CachePruning: fix typo, we accumulate file size here, not time
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266685
2016-04-18 21:53:55 +00:00
Mehdi Amini 27814980a3 Add Cache Pruning support
Incremental LTO will usea cache to store object files.
This patch handles the pruning part of the cache, exposing
a few knobs:

- Pruning interval: the implementation keeps a "timestamp" file in the
  directory and will scan it only after a given interval since the
  last modification of the timestamp file. This is for performance
  purpose, we don't want to scan continuously the folder.
- Entry expiration: this is the time after which a file that hasn't
  been used is remove from the cache.
- Maximum size: expressed in percentage of the available disk space,
  it helps to avoid that we blow up the disk space.

http://reviews.llvm.org/D18422

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265209
2016-04-02 03:28:26 +00:00