There was a segfault in the stubs library in posix_memalign because
of a bad parameter. The fix is to send address of the pointer as a
parameter. Also added check of result of posix_memalign.
Patch by Andrey Churbanov.
Differential Revision: http://reviews.llvm.org/D21529
llvm-svn: 273276
This change appends the process id to the KMP_STATS_FILE (if specified) which
enables MPI processes to output their stats to separate files.
Differential Revision: http://reviews.llvm.org/D21386
llvm-svn: 273273
Change hwloc discovery algorithm to print topology for only accessible
resources, and report uniformity correspondingly, similar to what other topology
discovery algorithms do. Fixes minor inconsistency in total topology reported
and resources used for threads binding in case hwloc used.
Patch by Andrey Churbanov.
Differential Revision: http://reviews.llvm.org/D21389
llvm-svn: 272952
This patch allows a user to enable Hwloc on windows. There are three main
changes in here:
1.kmp.h - Move definitions/declarations out of KMP_OS_WINDOWS guard (our windows
implementation of affinity) because they need to be defined when
KMP_USE_HWLOC is on as well.
2.teach __kmp_set_system_affinity, __kmp_get_system_affinity,
__kmp_get_proc_group, and __kmp_affinity_bind_thread how to use hwloc.
3.teach CMake how to include hwloc when building Windows
Another minor change in here is to make sure that anything under KMP_USE_HWLOC
is also guarded by KMP_AFFINITY_SUPPORTED as well. This is to prevent Mac
builds from requiring anything from Hwloc.
Differential Revision: http://reviews.llvm.org/D21441
llvm-svn: 272951
With single thread using __kmpc_omp_wait_deps segfaults in OpenMP runtime.
Offloading with depend also encounters this problem when we generate
kmpc_omp_wait_deps instead of kmpc_omp_task_with_deps.
Patch by Alex Duran
Differential Revision: http://reviews.llvm.org/D21384
llvm-svn: 272949
Cleanup: fixed missing memory cleanup in couple of corner cases. Fixes possible
memory leak in some corner cases
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21355
llvm-svn: 272946
Improved performance of ittnotify calls by request from ittnotify
owner: calls to __itt_string_handle_create made unique (it was
called multiple times).
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21353
llvm-svn: 272945
Deprecate KMP_PLACE_THREADS and rename it to KMP_HW_SUBSET due to confusion
about its purpose and function among users. KMP_HW_SUBSET is an environment
variable which allows users to easily pick a subset of the hardware topology to
use. e.g., KMP_HW_SUBSET=30c,2t means use 30 cores, 2 threads per core.
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21340
llvm-svn: 272937
Added argv array check/allocation for parallel directly nested inside the teams
construct, as new coming Fortran codegen passes parameters directly into
kmpc_fork_call missing same parameters in kmpc_fork_teams (earlier codegen
passed to parallel the subset of parameter passed to teams, and thus
no check/allocation needed).
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21336
llvm-svn: 272935
Currently, there is a big overhead in reporting of loop metadata through
ittnotify. The pair of functions: __kmp_str_loc_init/__kmp_str_loc_free are
replaced with strchr/atoi calls. Thus, a lot of time consuming actions are
skipped - many memory allocations/deallocations, heavy string duplication, etc.
The loop metadata only needs line and column info from the source string, so no
allocations and string splitting actually needed.
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21309
llvm-svn: 272698
Cleanup - unused code removal.
TODO: consider to remove (replace with flag class methods)
also kmp_wait_64 and kmp_release_64 routines.
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21332
llvm-svn: 272697
OpenMP 4.1 is now OpenMP 4.5. Any mention of 41 or 4.1 is replaced with
45 or 4.5. Also, if the CMake option LIBOMP_OMP_VERSION is 41, CMake warns that
41 is deprecated and to use 45 instead.
llvm-svn: 272687
Fix for bugzilla https://llvm.org/bugs/show_bug.cgi?id=26602. Removed functions
body consisted of the only KMP_ASSERT(0) statement. Thus possible runtime crash
converted to compile-time error, which looks preferable (faster possible error
detection).
TODO: consider C++11 static assert as an alternative, that could
make the diagnostics better.
Patch by Andrey Churbanov
Differential Revision: http://reviews.llvm.org/D21304
llvm-svn: 272590
Remove static specifier from var fullMask and remove kmp_get_fullMask() routine.
When iterating through procs in a mask, always check if proc is in fullMask
(this check was missing in a few places).
Patch by Brian Bliss.
Differential Revision: http://reviews.llvm.org/D21300
llvm-svn: 272589
If either current_task or new_task is untied then skip task scheduling
constraint checks, because untied tasks are not affected by the task
scheduling constraints.
Differential Revision: http://reviews.llvm.org/D21196
llvm-svn: 272570
The problem scenario is the following:
A dynamic library, libfoo.so, depends on libomp.so (it creates parallel region
and calls some omp functions). An application has a loop where it dynamically
loads libfoo.so, calls the function from it, unloads libfoo.so. After several
loop iterations application crashes with the message about lack of resources
OMP: Error #34: System unable to allocate necessary resources for OMP thread:
The problem is that pthread_kill() was not followed by pthread_join() in case
of terminated thread. This patch fixes this problem for both worker and monitor
threads.
Differential Revision: http://reviews.llvm.org/D21200
llvm-svn: 272567
These changes remove the hwloc_topology_ignore_type function which doesn't exist
in the hwloc 2.0 API. In the existing code, the topology extracted from hwloc
has the cache levels stripped out and then assumes the final stripped topology
follows the typical three-level topology: packages -> cores -> HW threads.
But the code is doing unclean manipulations to determine at what level those
resources are located and also assumes too much about what hwloc is detecting
(there could be intermediate levels in between socket and core for instance).
This new way of extracting the topology doesn't strip out any hardware objects
that hwloc detects. It does not assume the three level topology, and instead
searches for the relevant three levels within the topology for each bit of
information using hwloc interface functions. i.e., the three level topology
subset that our affinity code is interested in is extracted from the hwloc
topology tree directly.
For example, the new __kmp_hwloc_get_nobjs_under_obj function gives the user the
number of cores under a socket reliably without worrying if there are unexpected
objects between the socket object and core object in the hwloc topology
structure. Also, now that all topology information is kept, there are also
possibilities of using the caches/numa nodes to determine more sophisticated
affinity settings in the future.
There is also some cleanup code added for the destruction of the
__kmp_hwloc_topology object.
Differential Revision: http://reviews.llvm.org/D21195
llvm-svn: 272565
The bitmask complement operation doesn't consider the max proc id which means
something like !{0} will be translated to {1,2,3,4,...,600,601,...,1023} on a
Linux system even though there aren't 600 processors on said system. This
change has the complement bitmask and-ed with the fullmask so that it will only
contain valid processors.
Differential Revision: http://reviews.llvm.org/D21245
llvm-svn: 272561
Refactored __kmp_execute_tasks_template to shorten and remove code redundancy.
The original code for __kmp_execute_tasks_template was very redundant with
large sections of repeated code that needed to be kept consistent, and goto
statements that made the control flow difficult to discern. This refactoring
removes all gotos and redundancy.
Patch by Terry Wilmarth
Differential Revision: http://reviews.llvm.org/D20879
llvm-svn: 272286
MSVC doesn't allow std::atomic<>s in a union since they don't have trivial
copy constructor. Replacing them with e.g. std::atomic_int works, but that
breaks the GCC build on Linux, because then calls to e.g. std::atomic_load_explicit
fail, as they expect a real std::atomic<> pointer.
Fixing this with an #ifdef to unbreak the build for now.
llvm-svn: 272271
As I replaced no-op TCR_4 with actual code, compiler complained while building debug build.
This patch moves 'cast to int' to the correct place.
Extension to Differential Revision: http://reviews.llvm.org/D19880
llvm-svn: 271377
This patch replaces use of compiler builtin atomics with
C++11 atomics for ticket locks implementation. Ticket locks
are used in critical places of the runtime, e.g. in the tasking
mechanism.
The main reason this change was introduced is the problem
with work stealing function on ARM architecture which suffered
from nasty race condition. It turned out that the root cause of
the problem lies in the way ticket locks are implemented. Changing
compiler builtins into C++11 atomics solves the problem.
Two assertions were added into kmp_tasking.c which are useful
for detecting early symptoms of something wrong going on with
work stealing, which were among the possible outcomes of the
race condition.
Differential Revision: http://reviews.llvm.org/D19878
llvm-svn: 271324
This patch implements the new kmp_sch_static_balanced_chunked schedule kind that
the compiler will generate when it encounters schedule(simd: static). It just
adds the new constant and the new switch case __kmp_for_static_init.
Patch by Alex Duran.
Differential Revision: http://reviews.llvm.org/D20699
llvm-svn: 271320
When an asynchronous offload task is completed, COI calls the runtime to queue
a "destructor task". When the task deques are full, a dead-lock situation
arises where the OpenMP threads are inside but cannot progress because the COI
thread is stuck inside the runtime trying to find a slot in a deque.
This patch implements the solution where the task deques doubled in size when
a task is being queued from a COI thread.
Differential Revision: http://reviews.llvm.org/D20733
llvm-svn: 271319
The problem is the lack of dispatch buffers when thousands of loops with nowait,
about 10 iterations each, are executed by hundreds of threads. We only have
built-in 7 dispatch buffers, but there is a need in dozens or hundreds of
buffers.
The problem can be fixed by setting KMP_MAX_DISP_BUF to bigger value. In order
to give users same possibility I changed build-time control into run-time one,
adding API just in case.
This change adds an environment variable KMP_DISP_NUM_BUFFERS and a new API
function kmp_set_disp_num_buffers(int num_buffers).
The KMP_DISP_NUM_BUFFERS envirable works only before serial initialization,
because during the serial initialization we already allocate buffers for the hot
team, so it is too late to change the number of buffers later (or we need to
reallocate buffers for all teams which sounds too complicated). The
kmp_set_defaults() routine does not work for this envirable, because it calls
serial initialization before reading the parameter string. So a new routine,
kmp_set_disp_num_buffers(), is created so that it can set our internal global
variable before the library initialization. If both the envirable and API used
the envirable wins.
Differential Revision: http://reviews.llvm.org/D20697
llvm-svn: 271318
The OMP_PROC_BIND=spread strategy fails to assign the master thread the
correct place partition after the first parallel region. Other threads in the
hot team will remember their place_partition, but the master's place partition
is restored to what it was before entering the parallel region. So when the hot
team is used for subsequent parallel regions, the master has lost this info.
This fix calls __kmp_partition_places to update only the master thread's place
partition in the spread case when there are no other changes to the hot team.
Patch by Terry Wilmarth
Differential Revision: http://reviews.llvm.org/D20539
llvm-svn: 270890
On Blue Gene/Q, having LIBOMP_USE_ITT_NOTIFY support compiled into a
statically-linked binary causes a failure at runtime because dlopen fails.
This patch changes LIBOMP_USE_ITT_NOTIFY to a cacheable configuration setting
that can be disabled.
Patch by John Mellor-Crummey
Differential Revision: http://reviews.llvm.org/D20517
llvm-svn: 270884
Clang no longer restricts itself to generating microtasks with a small number
of arguments, and so an assembly implementation is required to prevent hitting
the parameter limit present in the C implementation. This adds an
implementation for ppc64[le].
llvm-svn: 270821
Most of this is modifications to check for differences before updating data
fields in team struct. There is also some rearrangement of the team struct.
Patch by Diego Caballero
Differential Revision: http://reviews.llvm.org/D20487
llvm-svn: 270468
These changes allow testing on Windows using clang.exe.
There are two main changes:
1. Only link to -lm when it actually exists on the system
2. Create basic versions of pthread_create() and pthread_join() for windows.
They are not POSIX compliant by any stretch but will allow any existing
and future tests to use pthread_create() and pthread_join() for testing
interactions of libomp with os threads.
Differential Revision: http://reviews.llvm.org/D20391
llvm-svn: 270464
KMP_USE_FUTEX preprocessor definition defined in kmp_lock.h is used
inconsequently throughout LLVM libomp code.
* some .c files that use this define do not include kmp_lock.h file,
in effect guarded part of code are never compiled
* some places in code use architecture-depending preprocessor
logic expressions which effectively disable use of Futex for
AArch64 architecture, all these places should use
'#if KMP_USE_FUTEX' instead to avoid any further confusions
* some places use KMP_HAS_FUTEX which is nowhere defined,
KMP_USE_FUTEX should be used instead
Differential Revision: http://reviews.llvm.org/D19629
llvm-svn: 269642
This patch solves 'Too many args to microtask' problem which occurs
while executing lulesh2.0.3 benchmark on AArch64.
To solve this I had to wrtite AArch64 assembly version of
__kmp_invoke_microtask() function, similar to x86 and x86_64
implementations.
Differential Revision: http://reviews.llvm.org/D19879
llvm-svn: 269399
This change adds a new entry point,
kmp_aligned_malloc(size_t size, size_t alignment), an entry point corresponding
to kmp_malloc() but with the capability to return aligned memory as well.
Other allocator routines have been adjusted so that kmp_free() can be used for
freeing memory blocks allocated by any kmp_*alloc() routine, including the new
kmp_aligned_malloc() routine.
Differential Revision: http://reviews.llvm.org/D19814
llvm-svn: 269365
After hot teams were enabled by default, the library started using levels kept
in the team structure. The levels are broken in case foreign thread exits and
puts its team into the pool which is then re-used by another foreign thread.
The broken behavior observed is when printing the levels for each new team, one
gets 1, 2, 1, 2, 1, 2, etc. This makes the library believe that every other
team is nested which is incorrect. What is wanted is for the levels to be
1, 1, 1, etc.
Differential Revision: http://reviews.llvm.org/D19980
llvm-svn: 269363
This reverts a presumaby-unintentional change in:
r268640 - [STATS] Use partitioned timer scheme
and fixes segfaults in an x86_64 debug build of the runtime library.
llvm-svn: 269259
This patch introduces following:
* TCI_* and TCD_* macros for incrementation and decrementation
* Fix for invalid use of TCR_8 in one expression
Differential Revision: http://reviews.llvm.org/D19880
llvm-svn: 268826
This change removes the current timers with ones that partition time properly.
The current timers are nested, so that if a new timer, B, starts when the
current timer, A, is already timing, A's time will include B's. To eliminate
this problem, the partitioned timers are designed to stop the current timer (A),
let the new timer run (B), and when the new timer is finished, restart the
previously running timer (A). With this partitioning of time, a threads' timers
all sum up to the OMP_worker_thread_life time and can now easily show the
percentage of time a thread is spending in different parts of the runtime or
user code.
There is also a new state variable associated with each thread which tells where
it is executing a task. This corresponds with the timers: OMP_task_*, e.g., if
time is spent in OMP_task_taskwait, then that thread executed tasks inside a
#pragma omp taskwait construct.
The changes are mostly changing the MACROs to use the new PARITIONED_* macros,
the new partitionedTimers class and its methods, and new state logic.
Differential Revision: http://reviews.llvm.org/D19229
llvm-svn: 268640
This debug sections's functionality can be replicated using the environment
variable KMP_TOPOLOGY_METHOD with different values and KMP_AFFINITY=verbose
llvm-svn: 267472
This change has the hwloc_bitmap_list_snprintf() function use the entire buffer
to print the mask. There is no need to shorten the buffer length by 7. It only
needs to be shortened by one byte.
llvm-svn: 267470
I have prepared some patches for LLVM OpenMP runtime, mostly addressing
ARMv8 support. Before I upstream them, I must address legal issues that
arose around my planned contribution. I was advised that before I send any
substantial commit, I need to make sure that LICENSE.txt file in the projects
repository contains a statement submitted by ARM, similar to the one provided
by Intel (see "a license agreement from the copyright/patent holders"). This is
the same situation as with top-level LLVM project: ARM has provided the same
statement in http://llvm.org/svn/llvm-project/llvm/trunk/lib/Target/ARM/LICENSE.TXT file.
Patch by Paul Osmialowski
Differential Revision: http://reviews.llvm.org/D19319
llvm-svn: 267446
The trip count calculation was incorrect for loops with large bounds. For example,
for(int i=-2,000,000,000; i < 2,000,000,000; i+=50000000), the trip count
calculation had overflow (trying to calculate 2,000,000,000 + 2,000,000,000 with
signed integers) and wasn't giving the right value. This patch fixes this error
in the runtime by using unsigned integers instead. There is still a bug in the
clang compiler component because it warns that there is overflow in the
test case file when there isn't. This error isn't there for the Intel Compiler.
So for now, the test case is designated as XFAIL.
Differential Revision: http://reviews.llvm.org/D19078
llvm-svn: 266677
Introduced a counter of parts of an untied task submitted for execution. The
counter controls whether all parts of the task are already finished. The
compiler should generate re-submission of partially executed untied task by
itself before exiting of each task part except for the lexical last part.
Differential Revision: http://reviews.llvm.org/D19026
llvm-svn: 266675
Some codes that use TLS fail intermittently because one thread tries to write
TLS values after the TLS key has been destroyed by another thread. This happens
when one thread executes library shutdown (and destroys TLS keys), while another
thread starts to execute the TLS key destructor routine. Before this change, the
kmp_init_runtime flag was checked before calling pthread_* TLS functions, but
this flag is set to FALSE later than the destruction of the TLS keys, which
leads to failure. The fix is to check kmp_init_gtid instead, as this flag is
unset *before* the destruction of TLS keys.
Differential Revision: http://reviews.llvm.org/D19022
llvm-svn: 266674
ittnotify fix for barrier imbalance time in case tasks exist. In the current
implementation, task execution time is included into aggregated time on a
barrier. This fix calculates task execution time and corrects the arrive time
by subtracting the task execution time.
Since __kmp_invoke_task() can not only be called on a barrier, the field
th.th_bar_arrive_time is used to check if the function was called at the
barrier (th.th_bar_arrive_time != 0). So for this check, th_bar_arrive_time
is set to zero right after the value is used on the barrier.
Differential Revision: http://reviews.llvm.org/D19030
llvm-svn: 266332
This change adds back off logic in the test and set lock for better contended
lock performance. It uses a simple truncated binary exponential back off
function. The default back off parameters are tuned for x86.
The main back off logic has a two loop structure where each is controlled by a
user-level parameter:
max_backoff - limits the outer loop number of iterations.
This parameter should be a power of 2.
min_ticks - the inner spin wait loop number of "ticks" which is system
dependent and should be tuned for your system if you so choose.
The "ticks" on x86 correspond to the time stamp counter,
but on other architectures ticks is a timestamp derived
from gettimeofday().
The user can modify these via the environment variable:
KMP_SPIN_BACKOFF_PARAMS=max_backoff[,min_ticks]
Currently, since the default user lock is a queuing lock,
one would have to also specify KMP_LOCK_KIND=tas to use the test-and-set locks.
Differential Revision: http://reviews.llvm.org/D19020
llvm-svn: 266329
This change has OMP_WAIT_POLICY=active to mean that threads will busy-wait in
spin loops and virtually never go to sleep. OMP_WAIT_POLICY=passive now means
that threads will immediately go to sleep inside a spin loop. KMP_BLOCKTIME was
the previous mechanism to specify this behavior via KMP_BLOCKTIME=0 or
KMP_BLOCKTIME=infinite, but the standard OpenMP environment variable should
also be able to specify this behavior.
Differential Revision: http://reviews.llvm.org/D18577
llvm-svn: 265339
#endif was one line too low. If KMP_USE_ADAPTIVE_LOCKS is 0,
then queuing locks would incorrectly use drdpa lock mechanism.
This is a fix for https://llvm.org/bugs/show_bug.cgi?id=26649
llvm-svn: 264934
Removed reference to "ref ct" in a comment, as ref_ct no longer exists. Also
moved the comment to where the task_team is about to be tested if NULL.
llvm-svn: 264786
The problem is that the definition of kmp_cpuinfo_t contains:
char name [3*sizeof (kmp_cpuid_t)]; // CPUID(0x80000002,0x80000003,0x80000004)
and kmp_cpuid_t is only defined when compiling for x86.
Differential Revision: http://reviews.llvm.org/D18245
llvm-svn: 264535
For serialized parallel regions, wrong ids were reported. Now the same code is
used as in kmp_dispatch.cpp which emits the correct ids.
Differential Revision: http://reviews.llvm.org/D18348
llvm-svn: 264266
For non-serialized parallel regions the master thread issued two callbacks:
The first one in kmp_gsupport.c and the second in __kmp_join_call. Therefore
only trigger the callback in kmp_gsupport.c for serialized parallel regions.
Differential Revision: http://reviews.llvm.org/D16716
llvm-svn: 264264
Have Visual Studio use MemoryBarrier() instead of _mm_mfence() and remove
__declspec align attribute from function parameters in kmp_atomic.h
llvm-svn: 264166
Some basic checks next to the implementation should futher lower the
possibility to introduce regressions. (Note that this would have catched
the ordering issue fixed in rL258866 and pointed to rL263940.)
The tests are implementation dependent in one point because they assume that
thread ids are assigned in ascending order. This is not defined by the standard
but currently ensured in libomp. We have to think about another way of ordering
the threads should this ever be subject to change...
Note that this isn't aiming at replacing the implementation independent
test-suite at https://github.com/OpenMPToolsInterface/ompt-test-suite!
Differential Revision: http://reviews.llvm.org/D16715
llvm-svn: 264027
This change logically separates the stats_flags_e::noTotal bit flag from the
stats_flags_e::onlyInMaster and stats_flags_e::noUnits bit flags. If no
TOTAL_foo output is wanted for a particular statistic, the flag must be
explicitly included in that statistic's flags.
Differential Revision: http://reviews.llvm.org/D18198
llvm-svn: 263954
Building libomp using CMake versions < 3.3 caused a link time error. These
errors occurred because when assembling z_Windows_NT-586_asm.asm, the
definitions: OMPT_SUPPORT, _M_AMD64|_M_IA32 weren't defined on the command line.
To fix the problem, the COMPILE_FLAGS property for the assembly file is appended
to instead of the COMPILE_DEFINITIONS property being set. For whatever reason, the
COMPILE_DEFINITIONS property doesn't pick up the definitions for assembly files
for the older CMake versions.
llvm-svn: 263651
This change adds a header to the printout of the statistics which includes the
time, machine name, and processor info if available. This change also includes
some cosmetic changes like using enum casting for timer and counter iteration.
Differential Revision: http://reviews.llvm.org/D18153
llvm-svn: 263580
This change removes synthesized stats and instead has all timers print out a
total which is the aggregate statistics across threads. This is displayed as
"Total_foo" at the end of program. The stats_flags_e::synthesized flag is
removed and the printStats() function is split into two separate functions:
printTimerStats() which can display the aggregate total and printCounterStats().
Differential Revision: http://reviews.llvm.org/D17869
llvm-svn: 263290
From the standard: The taskloop construct specifies that the iterations of one
or more associated loops will be executed in parallel using OpenMP tasks. The
iterations are distributed across tasks created by the construct and scheduled
to be executed.
This initial implementation uses a simple linear tasks distribution algorithm.
Later we can add other algorithms to speedup generation of huge number of tasks
(i.e., tree-like tasks generation should be faster).
This needs to be put into the OpenMP runtime library in order for the
compiler team to develop the compiler side of the implementation.
Differential Revision: http://reviews.llvm.org/D17404
llvm-svn: 262535
Jonathan Peyton