maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/compiler-rt/lib/scudo/scudo_tsd_shared.inc

39 lines
1.1 KiB
PHP
Raw Normal View History

[scudo] Scudo thread specific data refactor, part 3 Summary: Previous parts: D38139, D38183. In this part of the refactor, we abstract the Linux vs Android TSD dissociation in favor of a Exclusive vs Shared one, allowing for easier platform introduction and configuration. Most of this change consist of shuffling the files around to reflect the new organization. We introduce `scudo_platform.h` where platform specific definition lie. This involves the TSD model and the platform specific allocator parameters. In an upcoming CL, those will be configurable via defines, but we currently stick with conservative defaults. Reviewers: alekseyshl, dvyukov Reviewed By: alekseyshl, dvyukov Subscribers: srhines, llvm-commits, mgorny Differential Revision: https://reviews.llvm.org/D38244 llvm-svn: 314224
2017-09-27 01:20:02 +08:00
//===-- scudo_tsd_shared.inc ------------------------------------*- C++ -*-===//
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
///
[scudo] Scudo thread specific data refactor, part 3 Summary: Previous parts: D38139, D38183. In this part of the refactor, we abstract the Linux vs Android TSD dissociation in favor of a Exclusive vs Shared one, allowing for easier platform introduction and configuration. Most of this change consist of shuffling the files around to reflect the new organization. We introduce `scudo_platform.h` where platform specific definition lie. This involves the TSD model and the platform specific allocator parameters. In an upcoming CL, those will be configurable via defines, but we currently stick with conservative defaults. Reviewers: alekseyshl, dvyukov Reviewed By: alekseyshl, dvyukov Subscribers: srhines, llvm-commits, mgorny Differential Revision: https://reviews.llvm.org/D38244 llvm-svn: 314224
2017-09-27 01:20:02 +08:00
/// Scudo shared TSD fastpath functions implementation.
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
///
//===----------------------------------------------------------------------===//
[scudo] Scudo thread specific data refactor, part 3 Summary: Previous parts: D38139, D38183. In this part of the refactor, we abstract the Linux vs Android TSD dissociation in favor of a Exclusive vs Shared one, allowing for easier platform introduction and configuration. Most of this change consist of shuffling the files around to reflect the new organization. We introduce `scudo_platform.h` where platform specific definition lie. This involves the TSD model and the platform specific allocator parameters. In an upcoming CL, those will be configurable via defines, but we currently stick with conservative defaults. Reviewers: alekseyshl, dvyukov Reviewed By: alekseyshl, dvyukov Subscribers: srhines, llvm-commits, mgorny Differential Revision: https://reviews.llvm.org/D38244 llvm-svn: 314224
2017-09-27 01:20:02 +08:00
#ifndef SCUDO_TSD_H_
# error "This file must be included inside scudo_tsd.h."
#endif // SCUDO_TSD_H_
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
[scudo] Scudo thread specific data refactor, part 3 Summary: Previous parts: D38139, D38183. In this part of the refactor, we abstract the Linux vs Android TSD dissociation in favor of a Exclusive vs Shared one, allowing for easier platform introduction and configuration. Most of this change consist of shuffling the files around to reflect the new organization. We introduce `scudo_platform.h` where platform specific definition lie. This involves the TSD model and the platform specific allocator parameters. In an upcoming CL, those will be configurable via defines, but we currently stick with conservative defaults. Reviewers: alekseyshl, dvyukov Reviewed By: alekseyshl, dvyukov Subscribers: srhines, llvm-commits, mgorny Differential Revision: https://reviews.llvm.org/D38244 llvm-svn: 314224
2017-09-27 01:20:02 +08:00
#if !SCUDO_TSD_EXCLUSIVE
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
[scudo] Fix improper TSD init after TLS destructors are called Summary: Some of glibc's own thread local data is destroyed after a user's thread local destructors are called, via __libc_thread_freeres. This might involve calling free, as is the case for strerror_thread_freeres. If there is no prior heap operation in the thread, this free would end up initializing some thread specific data that would never be destroyed properly (as user's pthread destructors have already been called), while still being deallocated when the TLS goes away. As a result, a program could SEGV, usually in __sanitizer::AllocatorGlobalStats::Unregister, where one of the doubly linked list links would refer to a now unmapped memory area. To prevent this from happening, we will not do a full initialization from the deallocation path. This means that the fallback cache & quarantine will be used if no other heap operation has been called, and we effectively prevent the TSD being initialized and never destroyed. The TSD will be fully initialized for all other paths. In the event of a thread doing only frees and nothing else, a TSD would never be initialized for that thread, but this situation is unlikely and we can live with that. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37697 llvm-svn: 312939
2017-09-12 03:59:40 +08:00
ALWAYS_INLINE void initThreadMaybe(bool MinimalInit = false) {
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
if (LIKELY(*get_android_tls_ptr()))
return;
[scudo] Fix improper TSD init after TLS destructors are called Summary: Some of glibc's own thread local data is destroyed after a user's thread local destructors are called, via __libc_thread_freeres. This might involve calling free, as is the case for strerror_thread_freeres. If there is no prior heap operation in the thread, this free would end up initializing some thread specific data that would never be destroyed properly (as user's pthread destructors have already been called), while still being deallocated when the TLS goes away. As a result, a program could SEGV, usually in __sanitizer::AllocatorGlobalStats::Unregister, where one of the doubly linked list links would refer to a now unmapped memory area. To prevent this from happening, we will not do a full initialization from the deallocation path. This means that the fallback cache & quarantine will be used if no other heap operation has been called, and we effectively prevent the TSD being initialized and never destroyed. The TSD will be fully initialized for all other paths. In the event of a thread doing only frees and nothing else, a TSD would never be initialized for that thread, but this situation is unlikely and we can live with that. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37697 llvm-svn: 312939
2017-09-12 03:59:40 +08:00
initThread(MinimalInit);
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
}
[scudo] Scudo thread specific data refactor, part 1 Summary: We are going through an overhaul of Scudo's TSD, to allow for new platforms to be integrated more easily, and make the code more sound. This first part is mostly renaming, preferring some shorter names, correcting some comments. I removed `getPrng` and `getAllocatorCache` to directly access the members, there was not really any benefit to them (and it was suggested by Dmitry in D37590). The only functional change is in `scudo_tls_android.cpp`: we enforce bounds to the `NumberOfTSDs` and most of the logic in `getTSDAndLockSlow` is skipped if we only have 1 TSD. Reviewers: alekseyshl, dvyukov, kcc Reviewed By: dvyukov Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D38139 llvm-svn: 313987
2017-09-22 23:35:37 +08:00
ScudoTSD *getTSDAndLockSlow();
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
[scudo] Scudo thread specific data refactor, part 1 Summary: We are going through an overhaul of Scudo's TSD, to allow for new platforms to be integrated more easily, and make the code more sound. This first part is mostly renaming, preferring some shorter names, correcting some comments. I removed `getPrng` and `getAllocatorCache` to directly access the members, there was not really any benefit to them (and it was suggested by Dmitry in D37590). The only functional change is in `scudo_tls_android.cpp`: we enforce bounds to the `NumberOfTSDs` and most of the logic in `getTSDAndLockSlow` is skipped if we only have 1 TSD. Reviewers: alekseyshl, dvyukov, kcc Reviewed By: dvyukov Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D38139 llvm-svn: 313987
2017-09-22 23:35:37 +08:00
ALWAYS_INLINE ScudoTSD *getTSDAndLock() {
ScudoTSD *TSD = reinterpret_cast<ScudoTSD *>(*get_android_tls_ptr());
[scudo] Scudo thread specific data refactor, part 2 Summary: Following D38139, we now consolidate the TSD definition, merging the shared TSD definition with the exclusive TSD definition. We introduce a boolean set at initializaton denoting the need for the TSD to be unlocked or not. This adds some unused members to the exclusive TSD, but increases consistency and reduces the definitions fragmentation. We remove the fallback mechanism from `scudo_allocator.cpp` and add a fallback TSD in the non-shared version. Since the shared version doesn't require one, this makes overall more sense. There are a couple of additional cosmetic changes: removing the header guards from the remaining `.inc` files, added error string to a `CHECK`. Question to reviewers: I thought about friending `getTSDAndLock` in `ScudoTSD` so that the `FallbackTSD` could `Mutex.Lock()` directly instead of `lock()` which involved zeroing out the `Precedence`, which is unused otherwise. Is it worth doing? Reviewers: alekseyshl, dvyukov, kcc Reviewed By: dvyukov Subscribers: srhines, llvm-commits Differential Revision: https://reviews.llvm.org/D38183 llvm-svn: 314110
2017-09-25 23:12:08 +08:00
CHECK(TSD && "No TSD associated with the current thread!");
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
// Try to lock the currently associated context.
[scudo] Scudo thread specific data refactor, part 1 Summary: We are going through an overhaul of Scudo's TSD, to allow for new platforms to be integrated more easily, and make the code more sound. This first part is mostly renaming, preferring some shorter names, correcting some comments. I removed `getPrng` and `getAllocatorCache` to directly access the members, there was not really any benefit to them (and it was suggested by Dmitry in D37590). The only functional change is in `scudo_tls_android.cpp`: we enforce bounds to the `NumberOfTSDs` and most of the logic in `getTSDAndLockSlow` is skipped if we only have 1 TSD. Reviewers: alekseyshl, dvyukov, kcc Reviewed By: dvyukov Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D38139 llvm-svn: 313987
2017-09-22 23:35:37 +08:00
if (TSD->tryLock())
return TSD;
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
// If it failed, go the slow path.
[scudo] Scudo thread specific data refactor, part 1 Summary: We are going through an overhaul of Scudo's TSD, to allow for new platforms to be integrated more easily, and make the code more sound. This first part is mostly renaming, preferring some shorter names, correcting some comments. I removed `getPrng` and `getAllocatorCache` to directly access the members, there was not really any benefit to them (and it was suggested by Dmitry in D37590). The only functional change is in `scudo_tls_android.cpp`: we enforce bounds to the `NumberOfTSDs` and most of the logic in `getTSDAndLockSlow` is skipped if we only have 1 TSD. Reviewers: alekseyshl, dvyukov, kcc Reviewed By: dvyukov Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D38139 llvm-svn: 313987
2017-09-22 23:35:37 +08:00
return getTSDAndLockSlow();
[scudo] Add Android support Summary: This change adds Android support to the allocator (but doesn't yet enable it in the cmake config), and should be the last fragment of the rewritten change D31947. Android has more memory constraints than other platforms, so the idea of a unique context per thread would not have worked. The alternative chosen is to allocate a set of contexts based on the number of cores on the machine, and share those contexts within the threads. Contexts can be dynamically reassigned to threads to prevent contention, based on a scheme suggested by @dvyuokv in the initial review. Additionally, given that Android doesn't support ELF TLS (only emutls for now), we use the TSan TLS slot to make things faster: Scudo is mutually exclusive with other sanitizers so this shouldn't cause any problem. An additional change made here, is replacing `thread_local` by `THREADLOCAL` and using the initial-exec thread model in the non-Android version to prevent extraneous weak definition and checks on the relevant variables. Reviewers: kcc, dvyukov, alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D32649 llvm-svn: 302300
2017-05-06 05:38:22 +08:00
}
[scudo] Scudo thread specific data refactor, part 3 Summary: Previous parts: D38139, D38183. In this part of the refactor, we abstract the Linux vs Android TSD dissociation in favor of a Exclusive vs Shared one, allowing for easier platform introduction and configuration. Most of this change consist of shuffling the files around to reflect the new organization. We introduce `scudo_platform.h` where platform specific definition lie. This involves the TSD model and the platform specific allocator parameters. In an upcoming CL, those will be configurable via defines, but we currently stick with conservative defaults. Reviewers: alekseyshl, dvyukov Reviewed By: alekseyshl, dvyukov Subscribers: srhines, llvm-commits, mgorny Differential Revision: https://reviews.llvm.org/D38244 llvm-svn: 314224
2017-09-27 01:20:02 +08:00
#endif // !SCUDO_TSD_EXCLUSIVE