Summary:
Remove the generic error nadling policies and handle each allocator error
explicitly. Although more verbose, it allows for more comprehensive, precise
and actionable allocator related failure reports.
This finishes up the series of changes of the particular sanitizer
allocators, improves the internal allocator error reporting and removes
now unused policies.
Reviewers: vitalybuka, cryptoad
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48328
llvm-svn: 335147
Summary:
- Reland rL324263, this time allowing for a compile-time decision as to whether
or not use the 32-bit division. A single test is using a class map covering
a maximum size greater than 4GB, this can be checked via the template
parameters, and allows SizeClassAllocator64PopulateFreeListOOM to pass;
- `MaxCachedHint` is always called on a class id for which we have already
computed the size, but we still recompute `Size(class_id)`. Change the
prototype of the function to work on sizes instead of class ids. This also
allows us to get rid of the `kBatchClassID` special case. Update the callers
accordingly;
- `InitCache` and `Drain` will start iterating at index 1: index 0 contents are
unused and can safely be left to be 0. Plus we do not pay the cost of going
through an `UNLIKELY` in `MaxCachedHint`, and touching memory that is
otherwise not used;
- `const` some variables in the areas modified;
- Remove an spurious extra line at the end of a file.
Reviewers: alekseyshl, tl0gic, dberris
Reviewed By: alekseyshl, dberris
Subscribers: dberris, kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D43088
llvm-svn: 324906
Summary:
Here are a few improvements proposed for the local cache:
- `InitCache` always read from `per_class_[1]` in the fast path. This was not
ideal as we are working with `per_class_[class_id]`. The latter offers the
same property we are looking for (eg: `max_count != 0` means initialized),
so we might as well use it and keep our memory accesses local to the same
`per_class_` element. So change `InitCache` to take the current `PerClass`
as an argument. This also makes the fast-path assembly of `Deallocate` a lot
more compact;
- Change the 32-bit `Refill` & `Drain` functions to mimic their 64-bit
counterparts, by passing the current `PerClass` as an argument. This saves
some array computations;
- As far as I can tell, `InitCache` has no place in `Drain`: it's either called
from `Deallocate` which calls `InitCache`, or from the "upper" `Drain` which
checks for `c->count` to be greater than 0 (strictly). So remove it there.
- Move the `stats_` updates to after we are done with the `per_class_` accesses
in an attempt to preserve locality once more;
- Change some `CHECK` to `DCHECK`: I don't think the ones changed belonged in
the fast path and seemed to be overly cautious failsafes;
- Mark some variables as `const`.
The overall result is cleaner more compact fast path generated code, and some
performance gains with Scudo (and likely other Sanitizers).
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42851
llvm-svn: 324257
Summary:
Currently `TransferBatch` are located within the same memory regions as
"regular" chunks. This is not ideal for security: they make for an interesting
target to overwrite, and are not protected by the frontend (namely, Scudo).
To solve this, we re-introduce `kUseSeparateSizeClassForBatch` for the 32-bit
Primary allowing for `TransferBatch` to end up in their own memory region.
Currently only Scudo would use this new feature, the default behavior remains
unchanged. The separate `kBatchClassID` was used for a brief period of time
previously but removed when the 64-bit ended up using the "free array".
Reviewers: alekseyshl, kcc, eugenis
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D37082
llvm-svn: 311891
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).
Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34433
llvm-svn: 305972
Summary:
`SizeClassForTransferBatch` is expensive and is called for every `CreateBatch`
and `DestroyBatch`. Caching it means `kNumClasses` calls in `InitCache`
instead. This should be a performance gain if more than `kNumClasses / 2`
batches are created and destroyed during the lifetime of the local cache.
I have chosen to fully remove the function and putting the code in `InitCache`,
which is a debatable choice.
In single threaded benchmarks leveraging primary backed allocations, this turns
out to be a sizeable gain in performances (greater than 5%). In multithreaded
benchmarks leveraging everything, it is less significant but still an
improvement (about 1%).
Reviewers: kcc, dvyukov, alekseyshl
Reviewed By: dvyukov
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D32365
llvm-svn: 301184
Summary:
Allocator::ClassIdToSize() is not free and calling it in every
Allocate/Deallocate has noticeable impact on perf.
Reapplying D31991 with the appropriate fixes.
Reviewers: cryptoad
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D32024
llvm-svn: 300216
Summary:
Allocator::ClassIdToSize() is not free and calling it in every
Allocate/Deallocate has noticeable impact on perf.
Reviewers: eugenis, kcc
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D31991
llvm-svn: 300107
Alex Shlyapnikov