MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
/*
|
|
|
|
* Copyright (C) 2015 Imagination Technologies
|
|
|
|
* Author: Alex Smith <alex.smith@imgtec.com>
|
|
|
|
*
|
|
|
|
* This program is free software; you can redistribute it and/or modify it
|
|
|
|
* under the terms of the GNU General Public License as published by the
|
|
|
|
* Free Software Foundation; either version 2 of the License, or (at your
|
|
|
|
* option) any later version.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include "vdso.h"
|
|
|
|
|
|
|
|
#include <linux/compiler.h>
|
|
|
|
#include <linux/irqchip/mips-gic.h>
|
|
|
|
#include <linux/time.h>
|
|
|
|
|
|
|
|
#include <asm/clocksource.h>
|
|
|
|
#include <asm/io.h>
|
|
|
|
#include <asm/mips-cm.h>
|
|
|
|
#include <asm/unistd.h>
|
|
|
|
#include <asm/vdso.h>
|
|
|
|
|
2017-06-28 23:55:30 +08:00
|
|
|
#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
|
|
|
|
|
|
|
|
static __always_inline long gettimeofday_fallback(struct timeval *_tv,
|
|
|
|
struct timezone *_tz)
|
|
|
|
{
|
|
|
|
register struct timezone *tz asm("a1") = _tz;
|
|
|
|
register struct timeval *tv asm("a0") = _tv;
|
|
|
|
register long ret asm("v0");
|
|
|
|
register long nr asm("v0") = __NR_gettimeofday;
|
|
|
|
register long error asm("a3");
|
|
|
|
|
|
|
|
asm volatile(
|
|
|
|
" syscall\n"
|
|
|
|
: "=r" (ret), "=r" (error)
|
|
|
|
: "r" (tv), "r" (tz), "r" (nr)
|
2017-07-28 00:08:47 +08:00
|
|
|
: "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
|
|
|
|
"$14", "$15", "$24", "$25", "hi", "lo", "memory");
|
2017-06-28 23:55:30 +08:00
|
|
|
|
|
|
|
return error ? -ret : ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
2017-06-28 23:55:29 +08:00
|
|
|
static __always_inline long clock_gettime_fallback(clockid_t _clkid,
|
|
|
|
struct timespec *_ts)
|
|
|
|
{
|
|
|
|
register struct timespec *ts asm("a1") = _ts;
|
|
|
|
register clockid_t clkid asm("a0") = _clkid;
|
|
|
|
register long ret asm("v0");
|
|
|
|
register long nr asm("v0") = __NR_clock_gettime;
|
|
|
|
register long error asm("a3");
|
|
|
|
|
|
|
|
asm volatile(
|
|
|
|
" syscall\n"
|
|
|
|
: "=r" (ret), "=r" (error)
|
|
|
|
: "r" (clkid), "r" (ts), "r" (nr)
|
2017-07-28 00:08:47 +08:00
|
|
|
: "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
|
|
|
|
"$14", "$15", "$24", "$25", "hi", "lo", "memory");
|
2017-06-28 23:55:29 +08:00
|
|
|
|
|
|
|
return error ? -ret : ret;
|
|
|
|
}
|
|
|
|
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
static __always_inline int do_realtime_coarse(struct timespec *ts,
|
|
|
|
const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
u32 start_seq;
|
|
|
|
|
|
|
|
do {
|
|
|
|
start_seq = vdso_data_read_begin(data);
|
|
|
|
|
|
|
|
ts->tv_sec = data->xtime_sec;
|
|
|
|
ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
|
|
|
|
} while (vdso_data_read_retry(data, start_seq));
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static __always_inline int do_monotonic_coarse(struct timespec *ts,
|
|
|
|
const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
u32 start_seq;
|
MIPS: VDSO: Fix conversions in do_monotonic()/do_monotonic_coarse()
Fix incorrect calculation in do_monotonic() and do_monotonic_coarse()
function that in turn caused incorrect values returned by the vdso
version of system call clock_gettime() on mips64 if its system clock
ID parameter was CLOCK_MONOTONIC or CLOCK_MONOTONIC_COARSE.
Consider these variables and their types on mips32 and mips64:
tk->wall_to_monotonic.tv_sec s64, s64 (kernel/vdso.c)
vdso_data.wall_to_mono_sec u32, u32 (kernel/vdso.c)
to_mono_sec u32, u32 (vdso/gettimeofday.c)
ts->tv_sec s32, s64 (vdso/gettimeofday.c)
For mips64 case, u32 vdso_data.wall_to_mono_sec variable is updated
from the 64-bit signed variable tk->wall_to_monotonic.tv_sec
(kernel/vdso.c:76) which is a negative number holding the time passed
from 1970-01-01 to the time boot started. This 64-bit signed value is
currently around 47+ years, in seconds. For instance, let this value
be:
-1489757461
or
11111111111111111111111111111111 10100111001101000001101011101011
By updating 32-bit vdso_data.wall_to_mono_sec variable, we lose upper
32 bits (signed 1's).
to_mono_sec variable is a parameter of do_monotonic() and
do_monotonic_coarse() functions which holds vdso_data.wall_to_mono_sec
value. Its value needs to be added (or subtracted considering it holds
negative value from the tk->wall_to_monotonic.tv_sec) to the current
time passed from 1970-01-01 (ts->tv_sec), which is again something like
47+ years, but increased by the time passed from the boot to the
current time. ts->tv_sec is 32-bit long in case of 32-bit architecture
and 64-bit long in case of 64-bit architecture. Consider the update of
ts->tv_sec (vdso/gettimeofday.c:55 & 167):
ts->tv_sec += to_mono_sec;
mips32 case: This update will be performed correctly, since both
ts->tv_sec and to_mono_sec are 32-bit long and the sign in to_mono_sec
is preserved. Implicit conversion from u32 to s32 will be done
correctly.
mips64 case: This update will be wrong, since the implicit conversion
will not be done correctly. The reason is that the conversion will be
from u32 to s64. This is because to_mono_sec is 32-bit long for both
mips32 and mips64 cases and s64..33 bits of converted to_mono_sec
variable will be zeros.
So, in order to make MIPS64 implementation work properly for
MONOTONIC and MONOTONIC_COARSE clock ids on mips64, the size of
wall_to_mono_sec variable in mips_vdso_data union and respective
parameters in do_monotonic() and do_monotonic_coarse() functions
should be changed from u32 to u64. Because of consistency, this
size change from u32 and u64 is also done for wall_to_mono_nsec
variable and corresponding function parameters.
As far as similar situations for other architectures are concerned,
let's take a look at arm. Arm has two distinct vdso_data structures
for 32-bit & 64-bit cases, and arm's wall_to_mono_sec and
wall_to_mono_nsec are u32 for 32-bit and u64 for 64-bit cases.
On the other hand, MIPS has only one structure (mips_vdso_data),
hence the need for changing the size of above mentioned parameters.
Signed-off-by: Goran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: Miodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: Aleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16638/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-06-28 23:55:28 +08:00
|
|
|
u64 to_mono_sec;
|
|
|
|
u64 to_mono_nsec;
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
|
|
|
|
do {
|
|
|
|
start_seq = vdso_data_read_begin(data);
|
|
|
|
|
|
|
|
ts->tv_sec = data->xtime_sec;
|
|
|
|
ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
|
|
|
|
|
|
|
|
to_mono_sec = data->wall_to_mono_sec;
|
|
|
|
to_mono_nsec = data->wall_to_mono_nsec;
|
|
|
|
} while (vdso_data_read_retry(data, start_seq));
|
|
|
|
|
|
|
|
ts->tv_sec += to_mono_sec;
|
|
|
|
timespec_add_ns(ts, to_mono_nsec);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef CONFIG_CSRC_R4K
|
|
|
|
|
|
|
|
static __always_inline u64 read_r4k_count(void)
|
|
|
|
{
|
|
|
|
unsigned int count;
|
|
|
|
|
|
|
|
__asm__ __volatile__(
|
|
|
|
" .set push\n"
|
|
|
|
" .set mips32r2\n"
|
|
|
|
" rdhwr %0, $2\n"
|
|
|
|
" .set pop\n"
|
|
|
|
: "=r" (count));
|
|
|
|
|
|
|
|
return count;
|
|
|
|
}
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#ifdef CONFIG_CLKSRC_MIPS_GIC
|
|
|
|
|
|
|
|
static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
void __iomem *gic = get_gic(data);
|
|
|
|
u32 hi, hi2, lo;
|
|
|
|
|
|
|
|
do {
|
|
|
|
hi = __raw_readl(gic + GIC_UMV_SH_COUNTER_63_32_OFS);
|
|
|
|
lo = __raw_readl(gic + GIC_UMV_SH_COUNTER_31_00_OFS);
|
|
|
|
hi2 = __raw_readl(gic + GIC_UMV_SH_COUNTER_63_32_OFS);
|
|
|
|
} while (hi2 != hi);
|
|
|
|
|
|
|
|
return (((u64)hi) << 32) + lo;
|
|
|
|
}
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
static __always_inline u64 get_ns(const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
u64 cycle_now, delta, nsec;
|
|
|
|
|
|
|
|
switch (data->clock_mode) {
|
|
|
|
#ifdef CONFIG_CSRC_R4K
|
|
|
|
case VDSO_CLOCK_R4K:
|
|
|
|
cycle_now = read_r4k_count();
|
|
|
|
break;
|
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_CLKSRC_MIPS_GIC
|
|
|
|
case VDSO_CLOCK_GIC:
|
|
|
|
cycle_now = read_gic_count(data);
|
|
|
|
break;
|
|
|
|
#endif
|
|
|
|
default:
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
|
|
|
|
|
|
|
|
nsec = (delta * data->cs_mult) + data->xtime_nsec;
|
|
|
|
nsec >>= data->cs_shift;
|
|
|
|
|
|
|
|
return nsec;
|
|
|
|
}
|
|
|
|
|
|
|
|
static __always_inline int do_realtime(struct timespec *ts,
|
|
|
|
const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
u32 start_seq;
|
|
|
|
u64 ns;
|
|
|
|
|
|
|
|
do {
|
|
|
|
start_seq = vdso_data_read_begin(data);
|
|
|
|
|
|
|
|
if (data->clock_mode == VDSO_CLOCK_NONE)
|
|
|
|
return -ENOSYS;
|
|
|
|
|
|
|
|
ts->tv_sec = data->xtime_sec;
|
|
|
|
ns = get_ns(data);
|
|
|
|
} while (vdso_data_read_retry(data, start_seq));
|
|
|
|
|
|
|
|
ts->tv_nsec = 0;
|
|
|
|
timespec_add_ns(ts, ns);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static __always_inline int do_monotonic(struct timespec *ts,
|
|
|
|
const union mips_vdso_data *data)
|
|
|
|
{
|
|
|
|
u32 start_seq;
|
|
|
|
u64 ns;
|
MIPS: VDSO: Fix conversions in do_monotonic()/do_monotonic_coarse()
Fix incorrect calculation in do_monotonic() and do_monotonic_coarse()
function that in turn caused incorrect values returned by the vdso
version of system call clock_gettime() on mips64 if its system clock
ID parameter was CLOCK_MONOTONIC or CLOCK_MONOTONIC_COARSE.
Consider these variables and their types on mips32 and mips64:
tk->wall_to_monotonic.tv_sec s64, s64 (kernel/vdso.c)
vdso_data.wall_to_mono_sec u32, u32 (kernel/vdso.c)
to_mono_sec u32, u32 (vdso/gettimeofday.c)
ts->tv_sec s32, s64 (vdso/gettimeofday.c)
For mips64 case, u32 vdso_data.wall_to_mono_sec variable is updated
from the 64-bit signed variable tk->wall_to_monotonic.tv_sec
(kernel/vdso.c:76) which is a negative number holding the time passed
from 1970-01-01 to the time boot started. This 64-bit signed value is
currently around 47+ years, in seconds. For instance, let this value
be:
-1489757461
or
11111111111111111111111111111111 10100111001101000001101011101011
By updating 32-bit vdso_data.wall_to_mono_sec variable, we lose upper
32 bits (signed 1's).
to_mono_sec variable is a parameter of do_monotonic() and
do_monotonic_coarse() functions which holds vdso_data.wall_to_mono_sec
value. Its value needs to be added (or subtracted considering it holds
negative value from the tk->wall_to_monotonic.tv_sec) to the current
time passed from 1970-01-01 (ts->tv_sec), which is again something like
47+ years, but increased by the time passed from the boot to the
current time. ts->tv_sec is 32-bit long in case of 32-bit architecture
and 64-bit long in case of 64-bit architecture. Consider the update of
ts->tv_sec (vdso/gettimeofday.c:55 & 167):
ts->tv_sec += to_mono_sec;
mips32 case: This update will be performed correctly, since both
ts->tv_sec and to_mono_sec are 32-bit long and the sign in to_mono_sec
is preserved. Implicit conversion from u32 to s32 will be done
correctly.
mips64 case: This update will be wrong, since the implicit conversion
will not be done correctly. The reason is that the conversion will be
from u32 to s64. This is because to_mono_sec is 32-bit long for both
mips32 and mips64 cases and s64..33 bits of converted to_mono_sec
variable will be zeros.
So, in order to make MIPS64 implementation work properly for
MONOTONIC and MONOTONIC_COARSE clock ids on mips64, the size of
wall_to_mono_sec variable in mips_vdso_data union and respective
parameters in do_monotonic() and do_monotonic_coarse() functions
should be changed from u32 to u64. Because of consistency, this
size change from u32 and u64 is also done for wall_to_mono_nsec
variable and corresponding function parameters.
As far as similar situations for other architectures are concerned,
let's take a look at arm. Arm has two distinct vdso_data structures
for 32-bit & 64-bit cases, and arm's wall_to_mono_sec and
wall_to_mono_nsec are u32 for 32-bit and u64 for 64-bit cases.
On the other hand, MIPS has only one structure (mips_vdso_data),
hence the need for changing the size of above mentioned parameters.
Signed-off-by: Goran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: Miodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: Aleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16638/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-06-28 23:55:28 +08:00
|
|
|
u64 to_mono_sec;
|
|
|
|
u64 to_mono_nsec;
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
|
|
|
|
do {
|
|
|
|
start_seq = vdso_data_read_begin(data);
|
|
|
|
|
|
|
|
if (data->clock_mode == VDSO_CLOCK_NONE)
|
|
|
|
return -ENOSYS;
|
|
|
|
|
|
|
|
ts->tv_sec = data->xtime_sec;
|
|
|
|
ns = get_ns(data);
|
|
|
|
|
|
|
|
to_mono_sec = data->wall_to_mono_sec;
|
|
|
|
to_mono_nsec = data->wall_to_mono_nsec;
|
|
|
|
} while (vdso_data_read_retry(data, start_seq));
|
|
|
|
|
|
|
|
ts->tv_sec += to_mono_sec;
|
|
|
|
ts->tv_nsec = 0;
|
|
|
|
timespec_add_ns(ts, ns + to_mono_nsec);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This is behind the ifdef so that we don't provide the symbol when there's no
|
|
|
|
* possibility of there being a usable clocksource, because there's nothing we
|
|
|
|
* can do without it. When libc fails the symbol lookup it should fall back on
|
|
|
|
* the standard syscall path.
|
|
|
|
*/
|
|
|
|
int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
|
|
|
|
{
|
|
|
|
const union mips_vdso_data *data = get_vdso_data();
|
|
|
|
struct timespec ts;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
ret = do_realtime(&ts, data);
|
|
|
|
if (ret)
|
2017-06-28 23:55:30 +08:00
|
|
|
return gettimeofday_fallback(tv, tz);
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
|
|
|
|
if (tv) {
|
|
|
|
tv->tv_sec = ts.tv_sec;
|
|
|
|
tv->tv_usec = ts.tv_nsec / 1000;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (tz) {
|
|
|
|
tz->tz_minuteswest = data->tz_minuteswest;
|
|
|
|
tz->tz_dsttime = data->tz_dsttime;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2017-06-28 23:55:31 +08:00
|
|
|
#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
|
|
|
|
int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
|
|
|
|
{
|
|
|
|
const union mips_vdso_data *data = get_vdso_data();
|
2017-06-28 23:55:29 +08:00
|
|
|
int ret = -1;
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
|
|
|
|
switch (clkid) {
|
|
|
|
case CLOCK_REALTIME_COARSE:
|
|
|
|
ret = do_realtime_coarse(ts, data);
|
|
|
|
break;
|
|
|
|
case CLOCK_MONOTONIC_COARSE:
|
|
|
|
ret = do_monotonic_coarse(ts, data);
|
|
|
|
break;
|
|
|
|
case CLOCK_REALTIME:
|
|
|
|
ret = do_realtime(ts, data);
|
|
|
|
break;
|
|
|
|
case CLOCK_MONOTONIC:
|
|
|
|
ret = do_monotonic(ts, data);
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
2017-06-28 23:55:29 +08:00
|
|
|
if (ret)
|
|
|
|
ret = clock_gettime_fallback(clkid, ts);
|
|
|
|
|
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime()
Add user-mode implementations of gettimeofday() and clock_gettime() to
the VDSO. This is currently usable with 2 clocksources: the CP0 count
register, which is accessible to user-mode via RDHWR on R2 and later
cores, or the MIPS Global Interrupt Controller (GIC) timer, which
provides a "user-mode visible" section containing a mirror of its
counter registers. This section must be mapped into user memory, which
is done below the VDSO data page.
When a supported clocksource is not in use, the VDSO functions will
return -ENOSYS, which causes libc to fall back on the standard syscall
path.
When support for neither of these clocksources is compiled into the
kernel at all, the VDSO still provides clock_gettime(), as the coarse
realtime/monotonic clocks can still be implemented. However,
gettimeofday() is not provided in this case as nothing can be done
without a suitable clocksource. This causes the symbol lookup to fail
in libc and it will then always use the standard syscall path.
This patch includes a workaround for a bug in QEMU which results in
RDHWR on the CP0 count register always returning a constant (incorrect)
value. A fix for this has been submitted, and the workaround can be
removed after the fix has been in stable releases for a reasonable
amount of time.
A simple performance test which calls gettimeofday() 1000 times in a
loop and calculates the average execution time gives the following
results on a Malta + I6400 (running at 20MHz):
- Syscall: ~31000 ns
- VDSO (GIC): ~15000 ns
- VDSO (CP0): ~9500 ns
[markos.chandras@imgtec.com:
- Minor code re-arrangements in order for mappings to be made
in the order they appear to the process' address space.
- Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef
- Use gic_get_usm_range so we can do the GIC mapping in the
arch/mips/kernel/vdso instead of the GIC irqchip driver]
Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/11338/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 16:57:44 +08:00
|
|
|
return ret;
|
|
|
|
}
|