2019-06-04 16:11:32 +08:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-only
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
/*
|
|
|
|
* Kernel-based Virtual Machine driver for Linux
|
|
|
|
*
|
|
|
|
* This module enables machines with Intel VT-x extensions to run virtual
|
|
|
|
* machines without emulation or binary translation.
|
|
|
|
*
|
|
|
|
* Copyright (C) 2006 Qumranet, Inc.
|
2010-10-06 20:23:22 +08:00
|
|
|
* Copyright 2010 Red Hat, Inc. and/or its affiliates.
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
*
|
|
|
|
* Authors:
|
|
|
|
* Avi Kivity <avi@qumranet.com>
|
|
|
|
* Yaniv Kamay <yaniv@qumranet.com>
|
|
|
|
*/
|
|
|
|
|
2015-03-26 22:39:29 +08:00
|
|
|
#include <kvm/iodev.h>
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-12-16 17:02:48 +08:00
|
|
|
#include <linux/kvm_host.h>
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
#include <linux/kvm.h>
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/errno.h>
|
|
|
|
#include <linux/percpu.h>
|
|
|
|
#include <linux/mm.h>
|
|
|
|
#include <linux/miscdevice.h>
|
|
|
|
#include <linux/vmalloc.h>
|
|
|
|
#include <linux/reboot.h>
|
|
|
|
#include <linux/debugfs.h>
|
|
|
|
#include <linux/highmem.h>
|
|
|
|
#include <linux/file.h>
|
2011-03-24 05:16:23 +08:00
|
|
|
#include <linux/syscore_ops.h>
|
2007-02-12 16:54:47 +08:00
|
|
|
#include <linux/cpu.h>
|
2017-02-03 02:15:33 +08:00
|
|
|
#include <linux/sched/signal.h>
|
2017-02-09 01:51:29 +08:00
|
|
|
#include <linux/sched/mm.h>
|
2017-02-09 01:51:35 +08:00
|
|
|
#include <linux/sched/stat.h>
|
2007-06-08 00:18:30 +08:00
|
|
|
#include <linux/cpumask.h>
|
|
|
|
#include <linux/smp.h>
|
2007-06-28 20:38:16 +08:00
|
|
|
#include <linux/anon_inodes.h>
|
2007-09-10 23:10:54 +08:00
|
|
|
#include <linux/profile.h>
|
2007-09-18 03:57:50 +08:00
|
|
|
#include <linux/kvm_para.h>
|
2007-10-10 01:20:39 +08:00
|
|
|
#include <linux/pagemap.h>
|
2007-10-18 22:59:34 +08:00
|
|
|
#include <linux/mman.h>
|
2008-04-03 03:46:56 +08:00
|
|
|
#include <linux/swap.h>
|
2009-03-12 21:45:39 +08:00
|
|
|
#include <linux/bitops.h>
|
2009-05-08 04:55:13 +08:00
|
|
|
#include <linux/spinlock.h>
|
2009-10-22 20:19:27 +08:00
|
|
|
#include <linux/compat.h>
|
2009-12-24 00:35:21 +08:00
|
|
|
#include <linux/srcu.h>
|
2010-01-28 19:37:56 +08:00
|
|
|
#include <linux/hugetlb.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 16:04:11 +08:00
|
|
|
#include <linux/slab.h>
|
2011-07-27 21:00:48 +08:00
|
|
|
#include <linux/sort.h>
|
|
|
|
#include <linux/bsearch.h>
|
2019-05-17 20:08:53 +08:00
|
|
|
#include <linux/io.h>
|
2019-05-17 16:49:49 +08:00
|
|
|
#include <linux/lockdep.h>
|
2019-11-04 19:22:02 +08:00
|
|
|
#include <linux/kthread.h>
|
2021-06-06 10:10:44 +08:00
|
|
|
#include <linux/suspend.h>
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-06-29 02:15:57 +08:00
|
|
|
#include <asm/processor.h>
|
2014-09-20 07:03:25 +08:00
|
|
|
#include <asm/ioctl.h>
|
2016-12-25 03:46:01 +08:00
|
|
|
#include <linux/uaccess.h>
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2008-05-30 22:05:54 +08:00
|
|
|
#include "coalesced_mmio.h"
|
2010-10-14 17:22:46 +08:00
|
|
|
#include "async_pf.h"
|
2021-02-03 02:57:24 +08:00
|
|
|
#include "mmu_lock.h"
|
2014-09-24 19:02:46 +08:00
|
|
|
#include "vfio.h"
|
2008-05-30 22:05:54 +08:00
|
|
|
|
2009-06-17 20:22:14 +08:00
|
|
|
#define CREATE_TRACE_POINTS
|
|
|
|
#include <trace/events/kvm.h>
|
|
|
|
|
2020-10-01 09:22:22 +08:00
|
|
|
#include <linux/kvm_dirty_ring.h>
|
|
|
|
|
2016-05-18 19:26:23 +08:00
|
|
|
/* Worst case buffer size needed for holding an integer. */
|
|
|
|
#define ITOA_MAX_LEN 12
|
|
|
|
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
MODULE_AUTHOR("Qumranet");
|
|
|
|
MODULE_LICENSE("GPL");
|
|
|
|
|
2015-09-18 18:34:53 +08:00
|
|
|
/* Architectures should define their poll value according to the halt latency */
|
2016-10-14 08:53:19 +08:00
|
|
|
unsigned int halt_poll_ns = KVM_HALT_POLL_NS_DEFAULT;
|
2017-06-27 17:51:18 +08:00
|
|
|
module_param(halt_poll_ns, uint, 0644);
|
2016-10-14 08:53:19 +08:00
|
|
|
EXPORT_SYMBOL_GPL(halt_poll_ns);
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
|
2015-09-03 22:07:38 +08:00
|
|
|
/* Default doubles per-vcpu halt_poll_ns. */
|
2016-10-14 08:53:19 +08:00
|
|
|
unsigned int halt_poll_ns_grow = 2;
|
2017-06-27 17:51:18 +08:00
|
|
|
module_param(halt_poll_ns_grow, uint, 0644);
|
2016-10-14 08:53:19 +08:00
|
|
|
EXPORT_SYMBOL_GPL(halt_poll_ns_grow);
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2019-01-27 18:17:15 +08:00
|
|
|
/* The start value to grow halt_poll_ns from */
|
|
|
|
unsigned int halt_poll_ns_grow_start = 10000; /* 10us */
|
|
|
|
module_param(halt_poll_ns_grow_start, uint, 0644);
|
|
|
|
EXPORT_SYMBOL_GPL(halt_poll_ns_grow_start);
|
|
|
|
|
2015-09-03 22:07:38 +08:00
|
|
|
/* Default resets per-vcpu halt_poll_ns . */
|
2016-10-14 08:53:19 +08:00
|
|
|
unsigned int halt_poll_ns_shrink;
|
2017-06-27 17:51:18 +08:00
|
|
|
module_param(halt_poll_ns_shrink, uint, 0644);
|
2016-10-14 08:53:19 +08:00
|
|
|
EXPORT_SYMBOL_GPL(halt_poll_ns_shrink);
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2009-06-05 02:08:24 +08:00
|
|
|
/*
|
|
|
|
* Ordering of locks:
|
|
|
|
*
|
2015-02-26 14:58:24 +08:00
|
|
|
* kvm->lock --> kvm->slots_lock --> kvm->irq_lock
|
2009-06-05 02:08:24 +08:00
|
|
|
*/
|
|
|
|
|
2019-01-04 09:14:28 +08:00
|
|
|
DEFINE_MUTEX(kvm_lock);
|
2013-09-10 18:58:35 +08:00
|
|
|
static DEFINE_RAW_SPINLOCK(kvm_count_lock);
|
2007-11-14 20:38:21 +08:00
|
|
|
LIST_HEAD(vm_list);
|
2007-02-12 16:54:44 +08:00
|
|
|
|
2008-12-07 18:55:45 +08:00
|
|
|
static cpumask_var_t cpus_hardware_enabled;
|
2015-02-26 14:58:21 +08:00
|
|
|
static int kvm_usage_count;
|
2009-09-15 17:37:46 +08:00
|
|
|
static atomic_t hardware_enable_failed;
|
2007-05-24 18:03:52 +08:00
|
|
|
|
2019-12-19 05:55:16 +08:00
|
|
|
static struct kmem_cache *kvm_vcpu_cache;
|
2007-04-19 22:27:43 +08:00
|
|
|
|
2007-07-11 23:17:21 +08:00
|
|
|
static __read_mostly struct preempt_ops kvm_preempt_ops;
|
2020-01-09 22:57:19 +08:00
|
|
|
static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_running_vcpu);
|
2007-07-11 23:17:21 +08:00
|
|
|
|
2008-04-16 05:05:42 +08:00
|
|
|
struct dentry *kvm_debugfs_dir;
|
2015-03-28 11:21:01 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_debugfs_dir);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static const struct file_operations stat_fops_per_vm;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2007-02-22 00:04:26 +08:00
|
|
|
static long kvm_vcpu_ioctl(struct file *file, unsigned int ioctl,
|
|
|
|
unsigned long arg);
|
2015-02-03 16:35:15 +08:00
|
|
|
#ifdef CONFIG_KVM_COMPAT
|
2011-06-08 08:45:37 +08:00
|
|
|
static long kvm_vcpu_compat_ioctl(struct file *file, unsigned int ioctl,
|
|
|
|
unsigned long arg);
|
2018-06-17 17:16:21 +08:00
|
|
|
#define KVM_COMPAT(c) .compat_ioctl = (c)
|
|
|
|
#else
|
2019-11-14 21:17:39 +08:00
|
|
|
/*
|
|
|
|
* For architectures that don't implement a compat infrastructure,
|
|
|
|
* adopt a double line of defense:
|
|
|
|
* - Prevent a compat task from opening /dev/kvm
|
|
|
|
* - If the open has been done by a 64bit task, and the KVM fd
|
|
|
|
* passed to a compat task, let the ioctls fail.
|
|
|
|
*/
|
2018-06-17 17:16:21 +08:00
|
|
|
static long kvm_no_compat_ioctl(struct file *file, unsigned int ioctl,
|
|
|
|
unsigned long arg) { return -EINVAL; }
|
2019-11-14 00:05:23 +08:00
|
|
|
|
|
|
|
static int kvm_no_compat_open(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
return is_compat_task() ? -ENODEV : 0;
|
|
|
|
}
|
|
|
|
#define KVM_COMPAT(c) .compat_ioctl = kvm_no_compat_ioctl, \
|
|
|
|
.open = kvm_no_compat_open
|
2011-06-08 08:45:37 +08:00
|
|
|
#endif
|
2009-09-15 17:37:46 +08:00
|
|
|
static int hardware_enable_all(void);
|
|
|
|
static void hardware_disable_all(void);
|
2007-02-22 00:04:26 +08:00
|
|
|
|
2009-12-24 00:35:24 +08:00
|
|
|
static void kvm_io_bus_destroy(struct kvm_io_bus *bus);
|
2013-12-30 04:12:29 +08:00
|
|
|
|
2014-02-08 15:51:57 +08:00
|
|
|
__visible bool kvm_rebooting;
|
2010-12-02 23:52:50 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_rebooting);
|
2008-05-13 18:23:38 +08:00
|
|
|
|
2017-07-12 23:56:44 +08:00
|
|
|
#define KVM_EVENT_CREATE_VM 0
|
|
|
|
#define KVM_EVENT_DESTROY_VM 1
|
|
|
|
static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm);
|
|
|
|
static unsigned long long kvm_createvm_count;
|
|
|
|
static unsigned long long kvm_active_vms;
|
|
|
|
|
2020-06-06 12:26:27 +08:00
|
|
|
__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,
|
|
|
|
unsigned long start, unsigned long end)
|
2017-12-01 02:05:45 +08:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2019-11-12 06:12:27 +08:00
|
|
|
bool kvm_is_zone_device_pfn(kvm_pfn_t pfn)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* The metadata used by is_zone_device_page() to determine whether or
|
|
|
|
* not a page is ZONE_DEVICE is guaranteed to be valid if and only if
|
|
|
|
* the device has been pinned, e.g. by get_user_pages(). WARN if the
|
|
|
|
* page_count() is zero to help detect bad usage of this helper.
|
|
|
|
*/
|
|
|
|
if (!pfn_valid(pfn) || WARN_ON_ONCE(!page_count(pfn_to_page(pfn))))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
return is_zone_device_page(pfn_to_page(pfn));
|
|
|
|
}
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
|
2008-07-29 00:26:24 +08:00
|
|
|
{
|
2019-11-12 06:12:27 +08:00
|
|
|
/*
|
|
|
|
* ZONE_DEVICE pages currently set PG_reserved, but from a refcounting
|
|
|
|
* perspective they are "normal" pages, albeit with slightly different
|
|
|
|
* usage rules.
|
|
|
|
*/
|
2013-07-25 09:04:38 +08:00
|
|
|
if (pfn_valid(pfn))
|
2019-11-12 06:12:27 +08:00
|
|
|
return PageReserved(pfn_to_page(pfn)) &&
|
2019-10-12 11:37:31 +08:00
|
|
|
!is_zero_pfn(pfn) &&
|
2019-11-12 06:12:27 +08:00
|
|
|
!kvm_is_zone_device_pfn(pfn);
|
2008-07-29 00:26:24 +08:00
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2020-01-09 04:24:36 +08:00
|
|
|
bool kvm_is_transparent_hugepage(kvm_pfn_t pfn)
|
|
|
|
{
|
|
|
|
struct page *page = pfn_to_page(pfn);
|
|
|
|
|
|
|
|
if (!PageTransCompoundMap(page))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
return is_transparent_hugepage(compound_head(page));
|
|
|
|
}
|
|
|
|
|
2007-02-22 00:04:26 +08:00
|
|
|
/*
|
|
|
|
* Switches to specified vcpu, until a matching vcpu_put()
|
|
|
|
*/
|
2017-12-05 04:35:23 +08:00
|
|
|
void vcpu_load(struct kvm_vcpu *vcpu)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2017-12-05 04:35:23 +08:00
|
|
|
int cpu = get_cpu();
|
2020-01-09 22:57:19 +08:00
|
|
|
|
|
|
|
__this_cpu_write(kvm_running_vcpu, vcpu);
|
2007-07-11 23:17:21 +08:00
|
|
|
preempt_notifier_register(&vcpu->preempt_notifier);
|
KVM: Portability: split kvm_vcpu_ioctl
This patch splits kvm_vcpu_ioctl into archtecture independent parts, and
x86 specific parts which go to kvm_arch_vcpu_ioctl in x86.c.
Common ioctls for all architectures are:
KVM_RUN, KVM_GET/SET_(S-)REGS, KVM_TRANSLATE, KVM_INTERRUPT,
KVM_DEBUG_GUEST, KVM_SET_SIGNAL_MASK, KVM_GET/SET_FPU
Note that some PPC chips don't have an FPU, so we might need an #ifdef
around KVM_GET/SET_FPU one day.
x86 specific ioctls are:
KVM_GET/SET_LAPIC, KVM_SET_CPUID, KVM_GET/SET_MSRS
An interresting aspect is vcpu_load/vcpu_put. We now have a common
vcpu_load/put which does the preemption stuff, and an architecture
specific kvm_arch_vcpu_load/put. In the x86 case, this one calls the
vmx/svm function defined in kvm_x86_ops.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-10-12 01:16:52 +08:00
|
|
|
kvm_arch_vcpu_load(vcpu, cpu);
|
2007-07-11 23:17:21 +08:00
|
|
|
put_cpu();
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
2016-07-09 06:36:06 +08:00
|
|
|
EXPORT_SYMBOL_GPL(vcpu_load);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
KVM: Portability: split kvm_vcpu_ioctl
This patch splits kvm_vcpu_ioctl into archtecture independent parts, and
x86 specific parts which go to kvm_arch_vcpu_ioctl in x86.c.
Common ioctls for all architectures are:
KVM_RUN, KVM_GET/SET_(S-)REGS, KVM_TRANSLATE, KVM_INTERRUPT,
KVM_DEBUG_GUEST, KVM_SET_SIGNAL_MASK, KVM_GET/SET_FPU
Note that some PPC chips don't have an FPU, so we might need an #ifdef
around KVM_GET/SET_FPU one day.
x86 specific ioctls are:
KVM_GET/SET_LAPIC, KVM_SET_CPUID, KVM_GET/SET_MSRS
An interresting aspect is vcpu_load/vcpu_put. We now have a common
vcpu_load/put which does the preemption stuff, and an architecture
specific kvm_arch_vcpu_load/put. In the x86 case, this one calls the
vmx/svm function defined in kvm_x86_ops.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-10-12 01:16:52 +08:00
|
|
|
void vcpu_put(struct kvm_vcpu *vcpu)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2007-07-11 23:17:21 +08:00
|
|
|
preempt_disable();
|
KVM: Portability: split kvm_vcpu_ioctl
This patch splits kvm_vcpu_ioctl into archtecture independent parts, and
x86 specific parts which go to kvm_arch_vcpu_ioctl in x86.c.
Common ioctls for all architectures are:
KVM_RUN, KVM_GET/SET_(S-)REGS, KVM_TRANSLATE, KVM_INTERRUPT,
KVM_DEBUG_GUEST, KVM_SET_SIGNAL_MASK, KVM_GET/SET_FPU
Note that some PPC chips don't have an FPU, so we might need an #ifdef
around KVM_GET/SET_FPU one day.
x86 specific ioctls are:
KVM_GET/SET_LAPIC, KVM_SET_CPUID, KVM_GET/SET_MSRS
An interresting aspect is vcpu_load/vcpu_put. We now have a common
vcpu_load/put which does the preemption stuff, and an architecture
specific kvm_arch_vcpu_load/put. In the x86 case, this one calls the
vmx/svm function defined in kvm_x86_ops.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-10-12 01:16:52 +08:00
|
|
|
kvm_arch_vcpu_put(vcpu);
|
2007-07-11 23:17:21 +08:00
|
|
|
preempt_notifier_unregister(&vcpu->preempt_notifier);
|
2020-01-09 22:57:19 +08:00
|
|
|
__this_cpu_write(kvm_running_vcpu, NULL);
|
2007-07-11 23:17:21 +08:00
|
|
|
preempt_enable();
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
2016-07-09 06:36:06 +08:00
|
|
|
EXPORT_SYMBOL_GPL(vcpu_put);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2017-04-27 20:33:43 +08:00
|
|
|
/* TODO: merge with kvm_arch_vcpu_should_kick */
|
|
|
|
static bool kvm_request_needs_ipi(struct kvm_vcpu *vcpu, unsigned req)
|
|
|
|
{
|
|
|
|
int mode = kvm_vcpu_exiting_guest_mode(vcpu);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We need to wait for the VCPU to reenable interrupts and get out of
|
|
|
|
* READING_SHADOW_PAGE_TABLES mode.
|
|
|
|
*/
|
|
|
|
if (req & KVM_REQUEST_WAIT)
|
|
|
|
return mode != OUTSIDE_GUEST_MODE;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Need to kick a running VCPU, but otherwise there is nothing to do.
|
|
|
|
*/
|
|
|
|
return mode == IN_GUEST_MODE;
|
|
|
|
}
|
|
|
|
|
2007-06-08 00:18:30 +08:00
|
|
|
static void ack_flush(void *_completed)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2017-06-30 19:25:45 +08:00
|
|
|
static inline bool kvm_kick_many_cpus(const struct cpumask *cpus, bool wait)
|
|
|
|
{
|
|
|
|
if (unlikely(!cpus))
|
|
|
|
cpus = cpu_online_mask;
|
|
|
|
|
|
|
|
if (cpumask_empty(cpus))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
smp_call_function_many(cpus, ack_flush, NULL, wait);
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2018-05-16 23:21:28 +08:00
|
|
|
bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,
|
2020-05-06 21:17:53 +08:00
|
|
|
struct kvm_vcpu *except,
|
2018-05-16 23:21:28 +08:00
|
|
|
unsigned long *vcpu_bitmap, cpumask_var_t tmp)
|
2007-06-08 00:18:30 +08:00
|
|
|
{
|
2008-07-20 19:24:22 +08:00
|
|
|
int i, cpu, me;
|
2007-06-08 00:18:30 +08:00
|
|
|
struct kvm_vcpu *vcpu;
|
2018-05-16 23:21:28 +08:00
|
|
|
bool called;
|
2008-12-08 17:58:04 +08:00
|
|
|
|
2011-01-12 15:41:22 +08:00
|
|
|
me = get_cpu();
|
2018-05-16 23:21:28 +08:00
|
|
|
|
2009-06-09 20:56:29 +08:00
|
|
|
kvm_for_each_vcpu(i, vcpu, kvm) {
|
2020-05-06 21:17:53 +08:00
|
|
|
if ((vcpu_bitmap && !test_bit(i, vcpu_bitmap)) ||
|
|
|
|
vcpu == except)
|
2018-05-16 23:21:28 +08:00
|
|
|
continue;
|
|
|
|
|
2011-01-12 15:41:22 +08:00
|
|
|
kvm_make_request(req, vcpu);
|
2007-06-08 00:18:30 +08:00
|
|
|
cpu = vcpu->cpu;
|
2011-01-12 15:40:31 +08:00
|
|
|
|
2017-04-27 04:32:26 +08:00
|
|
|
if (!(req & KVM_REQUEST_NO_WAKEUP) && kvm_vcpu_wake_up(vcpu))
|
|
|
|
continue;
|
2017-04-27 04:32:23 +08:00
|
|
|
|
2018-05-16 23:21:28 +08:00
|
|
|
if (tmp != NULL && cpu != -1 && cpu != me &&
|
2017-04-27 20:33:43 +08:00
|
|
|
kvm_request_needs_ipi(vcpu, req))
|
2018-05-16 23:21:28 +08:00
|
|
|
__cpumask_set_cpu(cpu, tmp);
|
2008-12-08 17:56:24 +08:00
|
|
|
}
|
2018-05-16 23:21:28 +08:00
|
|
|
|
|
|
|
called = kvm_kick_many_cpus(tmp, !!(req & KVM_REQUEST_WAIT));
|
2011-01-12 15:41:22 +08:00
|
|
|
put_cpu();
|
2018-05-16 23:21:28 +08:00
|
|
|
|
|
|
|
return called;
|
|
|
|
}
|
|
|
|
|
2020-05-06 21:17:53 +08:00
|
|
|
bool kvm_make_all_cpus_request_except(struct kvm *kvm, unsigned int req,
|
|
|
|
struct kvm_vcpu *except)
|
2018-05-16 23:21:28 +08:00
|
|
|
{
|
|
|
|
cpumask_var_t cpus;
|
|
|
|
bool called;
|
|
|
|
|
|
|
|
zalloc_cpumask_var(&cpus, GFP_ATOMIC);
|
|
|
|
|
2020-05-06 21:17:53 +08:00
|
|
|
called = kvm_make_vcpus_request_mask(kvm, req, except, NULL, cpus);
|
2018-05-16 23:21:28 +08:00
|
|
|
|
2008-12-08 17:58:04 +08:00
|
|
|
free_cpumask_var(cpus);
|
2008-12-08 17:56:24 +08:00
|
|
|
return called;
|
2007-06-08 00:18:30 +08:00
|
|
|
}
|
|
|
|
|
2020-05-06 21:17:53 +08:00
|
|
|
bool kvm_make_all_cpus_request(struct kvm *kvm, unsigned int req)
|
|
|
|
{
|
|
|
|
return kvm_make_all_cpus_request_except(kvm, req, NULL);
|
|
|
|
}
|
KVM: VMX: update vcpu posted-interrupt descriptor when assigning device
For VMX, when a vcpu enters HLT emulation, pi_post_block will:
1) Add vcpu to per-cpu list of blocked vcpus.
2) Program the posted-interrupt descriptor "notification vector"
to POSTED_INTR_WAKEUP_VECTOR
With interrupt remapping, an interrupt will set the PIR bit for the
vector programmed for the device on the CPU, test-and-set the
ON bit on the posted interrupt descriptor, and if the ON bit is clear
generate an interrupt for the notification vector.
This way, the target CPU wakes upon a device interrupt and wakes up
the target vcpu.
Problem is that pi_post_block only programs the notification vector
if kvm_arch_has_assigned_device() is true. Its possible for the
following to happen:
1) vcpu V HLTs on pcpu P, kvm_arch_has_assigned_device is false,
notification vector is not programmed
2) device is assigned to VM
3) device interrupts vcpu V, sets ON bit
(notification vector not programmed, so pcpu P remains in idle)
4) vcpu 0 IPIs vcpu V (in guest), but since pi descriptor ON bit is set,
kvm_vcpu_kick is skipped
5) vcpu 0 busy spins on vcpu V's response for several seconds, until
RCU watchdog NMIs all vCPUs.
To fix this, use the start_assignment kvm_x86_ops callback to kick
vcpus out of the halt loop, so the notification vector is
properly reprogrammed to the wakeup vector.
Reported-by: Pei Zhang <pezhang@redhat.com>
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Message-Id: <20210526172014.GA29007@fuller.cnet>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 01:20:14 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_make_all_cpus_request);
|
2020-05-06 21:17:53 +08:00
|
|
|
|
2015-01-16 07:58:52 +08:00
|
|
|
#ifndef CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL
|
2008-12-08 17:56:24 +08:00
|
|
|
void kvm_flush_remote_tlbs(struct kvm *kvm)
|
2008-02-21 03:47:24 +08:00
|
|
|
{
|
2016-03-13 11:10:28 +08:00
|
|
|
/*
|
|
|
|
* Read tlbs_dirty before setting KVM_REQ_TLB_FLUSH in
|
|
|
|
* kvm_make_all_cpus_request.
|
|
|
|
*/
|
|
|
|
long dirty_count = smp_load_acquire(&kvm->tlbs_dirty);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We want to publish modifications to the page tables before reading
|
|
|
|
* mode. Pairs with a memory barrier in arch-specific code.
|
|
|
|
* - x86: smp_mb__after_srcu_read_unlock in vcpu_enter_guest
|
|
|
|
* and smp_mb in walk_shadow_page_lockless_begin/end.
|
|
|
|
* - powerpc: smp_mb in kvmppc_prepare_to_enter.
|
|
|
|
*
|
|
|
|
* There is already an smp_mb__after_atomic() before
|
|
|
|
* kvm_make_all_cpus_request() reads vcpu->mode. We reuse that
|
|
|
|
* barrier here.
|
|
|
|
*/
|
2018-07-19 16:40:17 +08:00
|
|
|
if (!kvm_arch_flush_remote_tlb(kvm)
|
|
|
|
|| kvm_make_all_cpus_request(kvm, KVM_REQ_TLB_FLUSH))
|
2021-06-19 06:27:03 +08:00
|
|
|
++kvm->stat.generic.remote_tlb_flush;
|
2014-04-17 17:06:12 +08:00
|
|
|
cmpxchg(&kvm->tlbs_dirty, dirty_count, 0);
|
2008-02-21 03:47:24 +08:00
|
|
|
}
|
2013-10-08 00:47:59 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs);
|
2015-01-16 07:58:52 +08:00
|
|
|
#endif
|
2008-02-21 03:47:24 +08:00
|
|
|
|
2008-12-08 17:56:24 +08:00
|
|
|
void kvm_reload_remote_mmus(struct kvm *kvm)
|
|
|
|
{
|
2014-09-24 15:57:55 +08:00
|
|
|
kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD);
|
2008-12-08 17:56:24 +08:00
|
|
|
}
|
2008-02-21 03:47:24 +08:00
|
|
|
|
2020-07-03 10:35:39 +08:00
|
|
|
#ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE
|
|
|
|
static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache *mc,
|
|
|
|
gfp_t gfp_flags)
|
|
|
|
{
|
|
|
|
gfp_flags |= mc->gfp_zero;
|
|
|
|
|
|
|
|
if (mc->kmem_cache)
|
|
|
|
return kmem_cache_alloc(mc->kmem_cache, gfp_flags);
|
|
|
|
else
|
|
|
|
return (void *)__get_free_page(gfp_flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
int kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int min)
|
|
|
|
{
|
|
|
|
void *obj;
|
|
|
|
|
|
|
|
if (mc->nobjs >= min)
|
|
|
|
return 0;
|
|
|
|
while (mc->nobjs < ARRAY_SIZE(mc->objects)) {
|
|
|
|
obj = mmu_memory_cache_alloc_obj(mc, GFP_KERNEL_ACCOUNT);
|
|
|
|
if (!obj)
|
|
|
|
return mc->nobjs >= min ? 0 : -ENOMEM;
|
|
|
|
mc->objects[mc->nobjs++] = obj;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
int kvm_mmu_memory_cache_nr_free_objects(struct kvm_mmu_memory_cache *mc)
|
|
|
|
{
|
|
|
|
return mc->nobjs;
|
|
|
|
}
|
|
|
|
|
|
|
|
void kvm_mmu_free_memory_cache(struct kvm_mmu_memory_cache *mc)
|
|
|
|
{
|
|
|
|
while (mc->nobjs) {
|
|
|
|
if (mc->kmem_cache)
|
|
|
|
kmem_cache_free(mc->kmem_cache, mc->objects[--mc->nobjs]);
|
|
|
|
else
|
|
|
|
free_page((unsigned long)mc->objects[--mc->nobjs]);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc)
|
|
|
|
{
|
|
|
|
void *p;
|
|
|
|
|
|
|
|
if (WARN_ON(!mc->nobjs))
|
|
|
|
p = mmu_memory_cache_alloc_obj(mc, GFP_ATOMIC | __GFP_ACCOUNT);
|
|
|
|
else
|
|
|
|
p = mc->objects[--mc->nobjs];
|
|
|
|
BUG_ON(!p);
|
|
|
|
return p;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2019-12-19 05:55:30 +08:00
|
|
|
static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
|
2007-07-27 15:16:56 +08:00
|
|
|
{
|
|
|
|
mutex_init(&vcpu->mutex);
|
|
|
|
vcpu->cpu = -1;
|
|
|
|
vcpu->kvm = kvm;
|
|
|
|
vcpu->vcpu_id = id;
|
2011-02-01 22:52:41 +08:00
|
|
|
vcpu->pid = NULL;
|
2020-04-24 13:48:37 +08:00
|
|
|
rcuwait_init(&vcpu->wait);
|
2010-10-14 17:22:46 +08:00
|
|
|
kvm_async_pf_vcpu_init(vcpu);
|
2007-07-27 15:16:56 +08:00
|
|
|
|
2015-09-18 22:29:55 +08:00
|
|
|
vcpu->pre_pcpu = -1;
|
|
|
|
INIT_LIST_HEAD(&vcpu->blocked_vcpu_list);
|
|
|
|
|
2012-07-18 21:37:46 +08:00
|
|
|
kvm_vcpu_set_in_spin_loop(vcpu, false);
|
|
|
|
kvm_vcpu_set_dy_eligible(vcpu, false);
|
2013-03-05 02:02:07 +08:00
|
|
|
vcpu->preempted = false;
|
KVM: Boost vCPUs that are delivering interrupts
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 155%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 179%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-18 19:39:06 +08:00
|
|
|
vcpu->ready = false;
|
2019-12-19 05:55:17 +08:00
|
|
|
preempt_notifier_init(&vcpu->preempt_notifier, &kvm_preempt_ops);
|
2021-08-05 06:28:40 +08:00
|
|
|
vcpu->last_used_slot = 0;
|
2007-07-27 15:16:56 +08:00
|
|
|
}
|
|
|
|
|
2019-12-19 05:55:14 +08:00
|
|
|
void kvm_vcpu_destroy(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
2020-10-01 09:22:22 +08:00
|
|
|
kvm_dirty_ring_free(&vcpu->dirty_ring);
|
2019-12-19 05:55:14 +08:00
|
|
|
kvm_arch_vcpu_destroy(vcpu);
|
2019-12-19 05:55:15 +08:00
|
|
|
|
2019-12-19 05:55:29 +08:00
|
|
|
/*
|
|
|
|
* No need for rcu_read_lock as VCPU_RUN is the only place that changes
|
|
|
|
* the vcpu->pid pointer, and at destruction time all file descriptors
|
|
|
|
* are already gone.
|
|
|
|
*/
|
|
|
|
put_pid(rcu_dereference_protected(vcpu->pid, 1));
|
|
|
|
|
2019-12-19 05:55:30 +08:00
|
|
|
free_page((unsigned long)vcpu->run);
|
2019-12-19 05:55:15 +08:00
|
|
|
kmem_cache_free(kvm_vcpu_cache, vcpu);
|
2019-12-19 05:55:14 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_destroy);
|
|
|
|
|
2008-07-25 22:24:52 +08:00
|
|
|
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
|
|
|
|
static inline struct kvm *mmu_notifier_to_kvm(struct mmu_notifier *mn)
|
|
|
|
{
|
|
|
|
return container_of(mn, struct kvm, mmu_notifier);
|
|
|
|
}
|
|
|
|
|
2020-06-06 12:26:27 +08:00
|
|
|
static void kvm_mmu_notifier_invalidate_range(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm,
|
|
|
|
unsigned long start, unsigned long end)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
int idx;
|
|
|
|
|
|
|
|
idx = srcu_read_lock(&kvm->srcu);
|
|
|
|
kvm_arch_mmu_notifier_invalidate_range(kvm, start, end);
|
|
|
|
srcu_read_unlock(&kvm->srcu, idx);
|
|
|
|
}
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
typedef bool (*hva_handler_t)(struct kvm *kvm, struct kvm_gfn_range *range);
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
typedef void (*on_lock_fn_t)(struct kvm *kvm, unsigned long start,
|
|
|
|
unsigned long end);
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
struct kvm_hva_range {
|
|
|
|
unsigned long start;
|
|
|
|
unsigned long end;
|
|
|
|
pte_t pte;
|
|
|
|
hva_handler_t handler;
|
2021-04-02 08:56:55 +08:00
|
|
|
on_lock_fn_t on_lock;
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
bool flush_on_ret;
|
|
|
|
bool may_block;
|
|
|
|
};
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
/*
|
|
|
|
* Use a dedicated stub instead of NULL to indicate that there is no callback
|
|
|
|
* function/handler. The compiler technically can't guarantee that a real
|
|
|
|
* function will have a non-zero address, and so it will generate code to
|
|
|
|
* check for !NULL, whereas comparing against a stub will be elided at compile
|
|
|
|
* time (unless the compiler is getting long in the tooth, e.g. gcc 4.9).
|
|
|
|
*/
|
|
|
|
static void kvm_null_fn(void)
|
|
|
|
{
|
|
|
|
|
|
|
|
}
|
|
|
|
#define IS_KVM_NULL_FN(fn) ((fn) == (void *)kvm_null_fn)
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,
|
|
|
|
const struct kvm_hva_range *range)
|
|
|
|
{
|
KVM: Take mmu_lock when handling MMU notifier iff the hva hits a memslot
Defer acquiring mmu_lock in the MMU notifier paths until a "hit" has been
detected in the memslots, i.e. don't take the lock for notifications that
don't affect the guest.
For small VMs, spurious locking is a minor annoyance. And for "volatile"
setups where the majority of notifications _are_ relevant, this barely
qualifies as an optimization.
But, for large VMs (hundreds of threads) with static setups, e.g. no
page migration, no swapping, etc..., the vast majority of MMU notifier
callbacks will be unrelated to the guest, e.g. will often be in response
to the userspace VMM adjusting its own virtual address space. In such
large VMs, acquiring mmu_lock can be painful as it blocks vCPUs from
handling page faults. In some scenarios it can even be "fatal" in the
sense that it causes unacceptable brownouts, e.g. when rebuilding huge
pages after live migration, a significant percentage of vCPUs will be
attempting to handle page faults.
x86's TDP MMU implementation is especially susceptible to spurious
locking due it taking mmu_lock for read when handling page faults.
Because rwlock is fair, a single writer will stall future readers, while
the writer is itself stalled waiting for in-progress readers to complete.
This is exacerbated by the MMU notifiers often firing multiple times in
quick succession, e.g. moving a page will (always?) invoke three separate
notifiers: .invalidate_range_start(), invalidate_range_end(), and
.change_pte(). Unnecessarily taking mmu_lock each time means even a
single spurious sequence can be problematic.
Note, this optimizes only the unpaired callbacks. Optimizing the
.invalidate_range_{start,end}() pairs is more complex and will be done in
a future patch.
Suggested-by: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:56 +08:00
|
|
|
bool ret = false, locked = false;
|
2021-04-02 08:56:55 +08:00
|
|
|
struct kvm_gfn_range gfn_range;
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
struct kvm_memory_slot *slot;
|
|
|
|
struct kvm_memslots *slots;
|
|
|
|
int i, idx;
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
/* A null handler is allowed if and only if on_lock() is provided. */
|
|
|
|
if (WARN_ON_ONCE(IS_KVM_NULL_FN(range->on_lock) &&
|
|
|
|
IS_KVM_NULL_FN(range->handler)))
|
|
|
|
return 0;
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
idx = srcu_read_lock(&kvm->srcu);
|
|
|
|
|
|
|
|
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
|
|
|
|
slots = __kvm_memslots(kvm, i);
|
|
|
|
kvm_for_each_memslot(slot, slots) {
|
|
|
|
unsigned long hva_start, hva_end;
|
|
|
|
|
|
|
|
hva_start = max(range->start, slot->userspace_addr);
|
|
|
|
hva_end = min(range->end, slot->userspace_addr +
|
|
|
|
(slot->npages << PAGE_SHIFT));
|
|
|
|
if (hva_start >= hva_end)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* To optimize for the likely case where the address
|
|
|
|
* range is covered by zero or one memslots, don't
|
|
|
|
* bother making these conditional (to avoid writes on
|
|
|
|
* the second or later invocation of the handler).
|
|
|
|
*/
|
|
|
|
gfn_range.pte = range->pte;
|
|
|
|
gfn_range.may_block = range->may_block;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* {gfn(page) | page intersects with [hva_start, hva_end)} =
|
|
|
|
* {gfn_start, gfn_start+1, ..., gfn_end-1}.
|
|
|
|
*/
|
|
|
|
gfn_range.start = hva_to_gfn_memslot(hva_start, slot);
|
|
|
|
gfn_range.end = hva_to_gfn_memslot(hva_end + PAGE_SIZE - 1, slot);
|
|
|
|
gfn_range.slot = slot;
|
|
|
|
|
KVM: Take mmu_lock when handling MMU notifier iff the hva hits a memslot
Defer acquiring mmu_lock in the MMU notifier paths until a "hit" has been
detected in the memslots, i.e. don't take the lock for notifications that
don't affect the guest.
For small VMs, spurious locking is a minor annoyance. And for "volatile"
setups where the majority of notifications _are_ relevant, this barely
qualifies as an optimization.
But, for large VMs (hundreds of threads) with static setups, e.g. no
page migration, no swapping, etc..., the vast majority of MMU notifier
callbacks will be unrelated to the guest, e.g. will often be in response
to the userspace VMM adjusting its own virtual address space. In such
large VMs, acquiring mmu_lock can be painful as it blocks vCPUs from
handling page faults. In some scenarios it can even be "fatal" in the
sense that it causes unacceptable brownouts, e.g. when rebuilding huge
pages after live migration, a significant percentage of vCPUs will be
attempting to handle page faults.
x86's TDP MMU implementation is especially susceptible to spurious
locking due it taking mmu_lock for read when handling page faults.
Because rwlock is fair, a single writer will stall future readers, while
the writer is itself stalled waiting for in-progress readers to complete.
This is exacerbated by the MMU notifiers often firing multiple times in
quick succession, e.g. moving a page will (always?) invoke three separate
notifiers: .invalidate_range_start(), invalidate_range_end(), and
.change_pte(). Unnecessarily taking mmu_lock each time means even a
single spurious sequence can be problematic.
Note, this optimizes only the unpaired callbacks. Optimizing the
.invalidate_range_{start,end}() pairs is more complex and will be done in
a future patch.
Suggested-by: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:56 +08:00
|
|
|
if (!locked) {
|
|
|
|
locked = true;
|
|
|
|
KVM_MMU_LOCK(kvm);
|
2021-08-03 15:45:41 +08:00
|
|
|
if (!IS_KVM_NULL_FN(range->on_lock))
|
|
|
|
range->on_lock(kvm, range->start, range->end);
|
|
|
|
if (IS_KVM_NULL_FN(range->handler))
|
|
|
|
break;
|
KVM: Take mmu_lock when handling MMU notifier iff the hva hits a memslot
Defer acquiring mmu_lock in the MMU notifier paths until a "hit" has been
detected in the memslots, i.e. don't take the lock for notifications that
don't affect the guest.
For small VMs, spurious locking is a minor annoyance. And for "volatile"
setups where the majority of notifications _are_ relevant, this barely
qualifies as an optimization.
But, for large VMs (hundreds of threads) with static setups, e.g. no
page migration, no swapping, etc..., the vast majority of MMU notifier
callbacks will be unrelated to the guest, e.g. will often be in response
to the userspace VMM adjusting its own virtual address space. In such
large VMs, acquiring mmu_lock can be painful as it blocks vCPUs from
handling page faults. In some scenarios it can even be "fatal" in the
sense that it causes unacceptable brownouts, e.g. when rebuilding huge
pages after live migration, a significant percentage of vCPUs will be
attempting to handle page faults.
x86's TDP MMU implementation is especially susceptible to spurious
locking due it taking mmu_lock for read when handling page faults.
Because rwlock is fair, a single writer will stall future readers, while
the writer is itself stalled waiting for in-progress readers to complete.
This is exacerbated by the MMU notifiers often firing multiple times in
quick succession, e.g. moving a page will (always?) invoke three separate
notifiers: .invalidate_range_start(), invalidate_range_end(), and
.change_pte(). Unnecessarily taking mmu_lock each time means even a
single spurious sequence can be problematic.
Note, this optimizes only the unpaired callbacks. Optimizing the
.invalidate_range_{start,end}() pairs is more complex and will be done in
a future patch.
Suggested-by: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:56 +08:00
|
|
|
}
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
ret |= range->handler(kvm, &gfn_range);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (range->flush_on_ret && (ret || kvm->tlbs_dirty))
|
|
|
|
kvm_flush_remote_tlbs(kvm);
|
|
|
|
|
KVM: Take mmu_lock when handling MMU notifier iff the hva hits a memslot
Defer acquiring mmu_lock in the MMU notifier paths until a "hit" has been
detected in the memslots, i.e. don't take the lock for notifications that
don't affect the guest.
For small VMs, spurious locking is a minor annoyance. And for "volatile"
setups where the majority of notifications _are_ relevant, this barely
qualifies as an optimization.
But, for large VMs (hundreds of threads) with static setups, e.g. no
page migration, no swapping, etc..., the vast majority of MMU notifier
callbacks will be unrelated to the guest, e.g. will often be in response
to the userspace VMM adjusting its own virtual address space. In such
large VMs, acquiring mmu_lock can be painful as it blocks vCPUs from
handling page faults. In some scenarios it can even be "fatal" in the
sense that it causes unacceptable brownouts, e.g. when rebuilding huge
pages after live migration, a significant percentage of vCPUs will be
attempting to handle page faults.
x86's TDP MMU implementation is especially susceptible to spurious
locking due it taking mmu_lock for read when handling page faults.
Because rwlock is fair, a single writer will stall future readers, while
the writer is itself stalled waiting for in-progress readers to complete.
This is exacerbated by the MMU notifiers often firing multiple times in
quick succession, e.g. moving a page will (always?) invoke three separate
notifiers: .invalidate_range_start(), invalidate_range_end(), and
.change_pte(). Unnecessarily taking mmu_lock each time means even a
single spurious sequence can be problematic.
Note, this optimizes only the unpaired callbacks. Optimizing the
.invalidate_range_{start,end}() pairs is more complex and will be done in
a future patch.
Suggested-by: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:56 +08:00
|
|
|
if (locked)
|
|
|
|
KVM_MMU_UNLOCK(kvm);
|
2021-04-02 08:56:55 +08:00
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
srcu_read_unlock(&kvm->srcu, idx);
|
|
|
|
|
|
|
|
/* The notifiers are averse to booleans. :-( */
|
|
|
|
return (int)ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn,
|
|
|
|
unsigned long start,
|
|
|
|
unsigned long end,
|
|
|
|
pte_t pte,
|
|
|
|
hva_handler_t handler)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
const struct kvm_hva_range range = {
|
|
|
|
.start = start,
|
|
|
|
.end = end,
|
|
|
|
.pte = pte,
|
|
|
|
.handler = handler,
|
2021-04-02 08:56:55 +08:00
|
|
|
.on_lock = (void *)kvm_null_fn,
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
.flush_on_ret = true,
|
|
|
|
.may_block = false,
|
|
|
|
};
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
return __kvm_handle_hva_range(kvm, &range);
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static __always_inline int kvm_handle_hva_range_no_flush(struct mmu_notifier *mn,
|
|
|
|
unsigned long start,
|
|
|
|
unsigned long end,
|
|
|
|
hva_handler_t handler)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
const struct kvm_hva_range range = {
|
|
|
|
.start = start,
|
|
|
|
.end = end,
|
|
|
|
.pte = __pte(0),
|
|
|
|
.handler = handler,
|
2021-04-02 08:56:55 +08:00
|
|
|
.on_lock = (void *)kvm_null_fn,
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
.flush_on_ret = false,
|
|
|
|
.may_block = false,
|
|
|
|
};
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
return __kvm_handle_hva_range(kvm, &range);
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
}
|
2009-09-24 02:47:18 +08:00
|
|
|
static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm,
|
|
|
|
unsigned long address,
|
|
|
|
pte_t pte)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
|
2021-03-26 10:19:48 +08:00
|
|
|
trace_kvm_set_spte_hva(address);
|
|
|
|
|
KVM: Assert that notifier count is elevated in .change_pte()
In KVM's .change_pte() notification callback, replace the notifier
sequence bump with a WARN_ON assertion that the notifier count is
elevated. An elevated count provides stricter protections than bumping
the sequence, and the sequence is guarnateed to be bumped before the
count hits zero.
When .change_pte() was added by commit 828502d30073 ("ksm: add
mmu_notifier set_pte_at_notify()"), bumping the sequence was necessary
as .change_pte() would be invoked without any surrounding notifications.
However, since commit 6bdb913f0a70 ("mm: wrap calls to set_pte_at_notify
with invalidate_range_start and invalidate_range_end"), all calls to
.change_pte() are guaranteed to be surrounded by start() and end(), and
so are guaranteed to run with an elevated notifier count.
Note, wrapping .change_pte() with .invalidate_range_{start,end}() is a
bug of sorts, as invalidating the secondary MMU's (KVM's) PTE defeats
the purpose of .change_pte(). Every arch's kvm_set_spte_hva() assumes
.change_pte() is called when the relevant SPTE is present in KVM's MMU,
as the original goal was to accelerate Kernel Samepage Merging (KSM) by
updating KVM's SPTEs without requiring a VM-Exit (due to invalidating
the SPTE). I.e. it means that .change_pte() is effectively dead code
on _all_ architectures.
x86 and MIPS are clearcut nops if the old SPTE is not-present, and that
is guaranteed due to the prior invalidation. PPC simply unmaps the SPTE,
which again should be a nop due to the invalidation. arm64 is a bit
murky, but it's also likely a nop because kvm_pgtable_stage2_map() is
called without a cache pointer, which means it will map an entry if and
only if an existing PTE was found.
For now, take advantage of the bug to simplify future consolidation of
KVMs's MMU notifier code. Doing so will not greatly complicate fixing
.change_pte(), assuming it's even worth fixing. .change_pte() has been
broken for 8+ years and no one has complained. Even if there are
KSM+KVM users that care deeply about its performance, the benefits of
avoiding VM-Exits via .change_pte() need to be reevaluated to justify
the added complexity and testing burden. Ripping out .change_pte()
entirely would be a lot easier.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:49 +08:00
|
|
|
/*
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
* .change_pte() must be surrounded by .invalidate_range_{start,end}().
|
2021-08-03 15:45:41 +08:00
|
|
|
* If mmu_notifier_count is zero, then no in-progress invalidations,
|
|
|
|
* including this one, found a relevant memslot at start(); rechecking
|
|
|
|
* memslots here is unnecessary. Note, a false positive (count elevated
|
|
|
|
* by a different invalidation) is sub-optimal but functionally ok.
|
KVM: Assert that notifier count is elevated in .change_pte()
In KVM's .change_pte() notification callback, replace the notifier
sequence bump with a WARN_ON assertion that the notifier count is
elevated. An elevated count provides stricter protections than bumping
the sequence, and the sequence is guarnateed to be bumped before the
count hits zero.
When .change_pte() was added by commit 828502d30073 ("ksm: add
mmu_notifier set_pte_at_notify()"), bumping the sequence was necessary
as .change_pte() would be invoked without any surrounding notifications.
However, since commit 6bdb913f0a70 ("mm: wrap calls to set_pte_at_notify
with invalidate_range_start and invalidate_range_end"), all calls to
.change_pte() are guaranteed to be surrounded by start() and end(), and
so are guaranteed to run with an elevated notifier count.
Note, wrapping .change_pte() with .invalidate_range_{start,end}() is a
bug of sorts, as invalidating the secondary MMU's (KVM's) PTE defeats
the purpose of .change_pte(). Every arch's kvm_set_spte_hva() assumes
.change_pte() is called when the relevant SPTE is present in KVM's MMU,
as the original goal was to accelerate Kernel Samepage Merging (KSM) by
updating KVM's SPTEs without requiring a VM-Exit (due to invalidating
the SPTE). I.e. it means that .change_pte() is effectively dead code
on _all_ architectures.
x86 and MIPS are clearcut nops if the old SPTE is not-present, and that
is guaranteed due to the prior invalidation. PPC simply unmaps the SPTE,
which again should be a nop due to the invalidation. arm64 is a bit
murky, but it's also likely a nop because kvm_pgtable_stage2_map() is
called without a cache pointer, which means it will map an entry if and
only if an existing PTE was found.
For now, take advantage of the bug to simplify future consolidation of
KVMs's MMU notifier code. Doing so will not greatly complicate fixing
.change_pte(), assuming it's even worth fixing. .change_pte() has been
broken for 8+ years and no one has complained. Even if there are
KSM+KVM users that care deeply about its performance, the benefits of
avoiding VM-Exits via .change_pte() need to be reevaluated to justify
the added complexity and testing burden. Ripping out .change_pte()
entirely would be a lot easier.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:49 +08:00
|
|
|
*/
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
WARN_ON_ONCE(!READ_ONCE(kvm->mn_active_invalidate_count));
|
2021-08-03 15:45:41 +08:00
|
|
|
if (!READ_ONCE(kvm->mmu_notifier_count))
|
|
|
|
return;
|
KVM: Assert that notifier count is elevated in .change_pte()
In KVM's .change_pte() notification callback, replace the notifier
sequence bump with a WARN_ON assertion that the notifier count is
elevated. An elevated count provides stricter protections than bumping
the sequence, and the sequence is guarnateed to be bumped before the
count hits zero.
When .change_pte() was added by commit 828502d30073 ("ksm: add
mmu_notifier set_pte_at_notify()"), bumping the sequence was necessary
as .change_pte() would be invoked without any surrounding notifications.
However, since commit 6bdb913f0a70 ("mm: wrap calls to set_pte_at_notify
with invalidate_range_start and invalidate_range_end"), all calls to
.change_pte() are guaranteed to be surrounded by start() and end(), and
so are guaranteed to run with an elevated notifier count.
Note, wrapping .change_pte() with .invalidate_range_{start,end}() is a
bug of sorts, as invalidating the secondary MMU's (KVM's) PTE defeats
the purpose of .change_pte(). Every arch's kvm_set_spte_hva() assumes
.change_pte() is called when the relevant SPTE is present in KVM's MMU,
as the original goal was to accelerate Kernel Samepage Merging (KSM) by
updating KVM's SPTEs without requiring a VM-Exit (due to invalidating
the SPTE). I.e. it means that .change_pte() is effectively dead code
on _all_ architectures.
x86 and MIPS are clearcut nops if the old SPTE is not-present, and that
is guaranteed due to the prior invalidation. PPC simply unmaps the SPTE,
which again should be a nop due to the invalidation. arm64 is a bit
murky, but it's also likely a nop because kvm_pgtable_stage2_map() is
called without a cache pointer, which means it will map an entry if and
only if an existing PTE was found.
For now, take advantage of the bug to simplify future consolidation of
KVMs's MMU notifier code. Doing so will not greatly complicate fixing
.change_pte(), assuming it's even worth fixing. .change_pte() has been
broken for 8+ years and no one has complained. Even if there are
KSM+KVM users that care deeply about its performance, the benefits of
avoiding VM-Exits via .change_pte() need to be reevaluated to justify
the added complexity and testing burden. Ripping out .change_pte()
entirely would be a lot easier.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:49 +08:00
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
kvm_handle_hva_range(mn, address, address + 1, pte, kvm_set_spte_gfn);
|
2009-09-24 02:47:18 +08:00
|
|
|
}
|
|
|
|
|
2021-08-11 04:52:39 +08:00
|
|
|
void kvm_inc_notifier_count(struct kvm *kvm, unsigned long start,
|
2021-04-02 08:56:55 +08:00
|
|
|
unsigned long end)
|
2008-07-25 22:24:52 +08:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* The count increase must become visible at unlock time as no
|
|
|
|
* spte can be established without taking the mmu_lock and
|
|
|
|
* count is also read inside the mmu_lock critical section.
|
|
|
|
*/
|
|
|
|
kvm->mmu_notifier_count++;
|
2021-02-22 10:45:22 +08:00
|
|
|
if (likely(kvm->mmu_notifier_count == 1)) {
|
2021-04-02 08:56:55 +08:00
|
|
|
kvm->mmu_notifier_range_start = start;
|
|
|
|
kvm->mmu_notifier_range_end = end;
|
2021-02-22 10:45:22 +08:00
|
|
|
} else {
|
|
|
|
/*
|
|
|
|
* Fully tracking multiple concurrent ranges has dimishing
|
|
|
|
* returns. Keep things simple and just find the minimal range
|
|
|
|
* which includes the current and new ranges. As there won't be
|
|
|
|
* enough information to subtract a range after its invalidate
|
|
|
|
* completes, any ranges invalidated concurrently will
|
|
|
|
* accumulate and persist until all outstanding invalidates
|
|
|
|
* complete.
|
|
|
|
*/
|
|
|
|
kvm->mmu_notifier_range_start =
|
2021-04-02 08:56:55 +08:00
|
|
|
min(kvm->mmu_notifier_range_start, start);
|
2021-02-22 10:45:22 +08:00
|
|
|
kvm->mmu_notifier_range_end =
|
2021-04-02 08:56:55 +08:00
|
|
|
max(kvm->mmu_notifier_range_end, end);
|
2021-02-22 10:45:22 +08:00
|
|
|
}
|
2021-04-02 08:56:55 +08:00
|
|
|
}
|
2021-08-11 04:52:39 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_inc_notifier_count);
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
|
|
|
|
const struct mmu_notifier_range *range)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
const struct kvm_hva_range hva_range = {
|
|
|
|
.start = range->start,
|
|
|
|
.end = range->end,
|
|
|
|
.pte = __pte(0),
|
|
|
|
.handler = kvm_unmap_gfn_range,
|
|
|
|
.on_lock = kvm_inc_notifier_count,
|
|
|
|
.flush_on_ret = true,
|
|
|
|
.may_block = mmu_notifier_range_blockable(range),
|
|
|
|
};
|
2012-02-10 14:28:31 +08:00
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
trace_kvm_unmap_hva_range(range->start, range->end);
|
|
|
|
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
/*
|
|
|
|
* Prevent memslot modification between range_start() and range_end()
|
|
|
|
* so that conditionally locking provides the same result in both
|
|
|
|
* functions. Without that guarantee, the mmu_notifier_count
|
|
|
|
* adjustments will be imbalanced.
|
|
|
|
*
|
|
|
|
* Pairs with the decrement in range_end().
|
|
|
|
*/
|
|
|
|
spin_lock(&kvm->mn_invalidate_lock);
|
|
|
|
kvm->mn_active_invalidate_count++;
|
|
|
|
spin_unlock(&kvm->mn_invalidate_lock);
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
__kvm_handle_hva_range(kvm, &hva_range);
|
2018-08-22 12:52:33 +08:00
|
|
|
|
2020-06-06 12:26:27 +08:00
|
|
|
return 0;
|
2008-07-25 22:24:52 +08:00
|
|
|
}
|
|
|
|
|
2021-08-11 04:52:39 +08:00
|
|
|
void kvm_dec_notifier_count(struct kvm *kvm, unsigned long start,
|
2021-04-02 08:56:55 +08:00
|
|
|
unsigned long end)
|
2008-07-25 22:24:52 +08:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* This sequence increase will notify the kvm page fault that
|
|
|
|
* the page that is going to be mapped in the spte could have
|
|
|
|
* been freed.
|
|
|
|
*/
|
|
|
|
kvm->mmu_notifier_seq++;
|
2011-12-12 20:37:21 +08:00
|
|
|
smp_wmb();
|
2008-07-25 22:24:52 +08:00
|
|
|
/*
|
|
|
|
* The above sequence increase must be visible before the
|
2011-12-12 20:37:21 +08:00
|
|
|
* below count decrease, which is ensured by the smp_wmb above
|
|
|
|
* in conjunction with the smp_rmb in mmu_notifier_retry().
|
2008-07-25 22:24:52 +08:00
|
|
|
*/
|
|
|
|
kvm->mmu_notifier_count--;
|
2021-04-02 08:56:55 +08:00
|
|
|
}
|
2021-08-11 04:52:39 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_dec_notifier_count);
|
|
|
|
|
2021-04-02 08:56:55 +08:00
|
|
|
|
|
|
|
static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
|
|
|
|
const struct mmu_notifier_range *range)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
|
|
|
const struct kvm_hva_range hva_range = {
|
|
|
|
.start = range->start,
|
|
|
|
.end = range->end,
|
|
|
|
.pte = __pte(0),
|
|
|
|
.handler = (void *)kvm_null_fn,
|
|
|
|
.on_lock = kvm_dec_notifier_count,
|
|
|
|
.flush_on_ret = false,
|
|
|
|
.may_block = mmu_notifier_range_blockable(range),
|
|
|
|
};
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
bool wake;
|
2021-04-02 08:56:55 +08:00
|
|
|
|
|
|
|
__kvm_handle_hva_range(kvm, &hva_range);
|
2008-07-25 22:24:52 +08:00
|
|
|
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
/* Pairs with the increment in range_start(). */
|
|
|
|
spin_lock(&kvm->mn_invalidate_lock);
|
|
|
|
wake = (--kvm->mn_active_invalidate_count == 0);
|
|
|
|
spin_unlock(&kvm->mn_invalidate_lock);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* There can only be one waiter, since the wait happens under
|
|
|
|
* slots_lock.
|
|
|
|
*/
|
|
|
|
if (wake)
|
|
|
|
rcuwait_wake_up(&kvm->mn_memslots_update_rcuwait);
|
|
|
|
|
2008-07-25 22:24:52 +08:00
|
|
|
BUG_ON(kvm->mmu_notifier_count < 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm,
|
2014-09-23 05:54:42 +08:00
|
|
|
unsigned long start,
|
|
|
|
unsigned long end)
|
2008-07-25 22:24:52 +08:00
|
|
|
{
|
2021-03-26 10:19:48 +08:00
|
|
|
trace_kvm_age_hva(start, end);
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
return kvm_handle_hva_range(mn, start, end, __pte(0), kvm_age_gfn);
|
2008-07-25 22:24:52 +08:00
|
|
|
}
|
|
|
|
|
2015-09-10 06:35:41 +08:00
|
|
|
static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm,
|
|
|
|
unsigned long start,
|
|
|
|
unsigned long end)
|
|
|
|
{
|
2021-03-26 10:19:48 +08:00
|
|
|
trace_kvm_age_hva(start, end);
|
|
|
|
|
2015-09-10 06:35:41 +08:00
|
|
|
/*
|
|
|
|
* Even though we do not flush TLB, this will still adversely
|
|
|
|
* affect performance on pre-Haswell Intel EPT, where there is
|
|
|
|
* no EPT Access Bit to clear so that we have to tear down EPT
|
|
|
|
* tables instead. If we find this unacceptable, we can always
|
|
|
|
* add a parameter to kvm_age_hva so that it effectively doesn't
|
|
|
|
* do anything on clear_young.
|
|
|
|
*
|
|
|
|
* Also note that currently we never issue secondary TLB flushes
|
|
|
|
* from clear_young, leaving this job up to the regular system
|
|
|
|
* cadence. If we find this inaccurate, we might come up with a
|
|
|
|
* more sophisticated heuristic later.
|
|
|
|
*/
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
return kvm_handle_hva_range_no_flush(mn, start, end, kvm_age_gfn);
|
2015-09-10 06:35:41 +08:00
|
|
|
}
|
|
|
|
|
2011-01-14 07:47:10 +08:00
|
|
|
static int kvm_mmu_notifier_test_young(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm,
|
|
|
|
unsigned long address)
|
|
|
|
{
|
2021-03-26 10:19:48 +08:00
|
|
|
trace_kvm_test_age_hva(address);
|
|
|
|
|
KVM: Move x86's MMU notifier memslot walkers to generic code
Move the hva->gfn lookup for MMU notifiers into common code. Every arch
does a similar lookup, and some arch code is all but identical across
multiple architectures.
In addition to consolidating code, this will allow introducing
optimizations that will benefit all architectures without incurring
multiple walks of the memslots, e.g. by taking mmu_lock if and only if a
relevant range exists in the memslots.
The use of __always_inline to avoid indirect call retpolines, as done by
x86, may also benefit other architectures.
Consolidating the lookups also fixes a wart in x86, where the legacy MMU
and TDP MMU each do their own memslot walks.
Lastly, future enhancements to the memslot implementation, e.g. to add an
interval tree to track host address, will need to touch far less arch
specific code.
MIPS, PPC, and arm64 will be converted one at a time in future patches.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20210402005658.3024832-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-04-02 08:56:50 +08:00
|
|
|
return kvm_handle_hva_range_no_flush(mn, address, address + 1,
|
|
|
|
kvm_test_age_gfn);
|
2011-01-14 07:47:10 +08:00
|
|
|
}
|
|
|
|
|
2008-12-11 04:23:26 +08:00
|
|
|
static void kvm_mmu_notifier_release(struct mmu_notifier *mn,
|
|
|
|
struct mm_struct *mm)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = mmu_notifier_to_kvm(mn);
|
2010-04-20 14:29:29 +08:00
|
|
|
int idx;
|
|
|
|
|
|
|
|
idx = srcu_read_lock(&kvm->srcu);
|
2012-08-25 02:54:57 +08:00
|
|
|
kvm_arch_flush_shadow_all(kvm);
|
2010-04-20 14:29:29 +08:00
|
|
|
srcu_read_unlock(&kvm->srcu, idx);
|
2008-12-11 04:23:26 +08:00
|
|
|
}
|
|
|
|
|
2008-07-25 22:24:52 +08:00
|
|
|
static const struct mmu_notifier_ops kvm_mmu_notifier_ops = {
|
2020-06-06 12:26:27 +08:00
|
|
|
.invalidate_range = kvm_mmu_notifier_invalidate_range,
|
2008-07-25 22:24:52 +08:00
|
|
|
.invalidate_range_start = kvm_mmu_notifier_invalidate_range_start,
|
|
|
|
.invalidate_range_end = kvm_mmu_notifier_invalidate_range_end,
|
|
|
|
.clear_flush_young = kvm_mmu_notifier_clear_flush_young,
|
2015-09-10 06:35:41 +08:00
|
|
|
.clear_young = kvm_mmu_notifier_clear_young,
|
2011-01-14 07:47:10 +08:00
|
|
|
.test_young = kvm_mmu_notifier_test_young,
|
2009-09-24 02:47:18 +08:00
|
|
|
.change_pte = kvm_mmu_notifier_change_pte,
|
2008-12-11 04:23:26 +08:00
|
|
|
.release = kvm_mmu_notifier_release,
|
2008-07-25 22:24:52 +08:00
|
|
|
};
|
2009-12-20 20:54:04 +08:00
|
|
|
|
|
|
|
static int kvm_init_mmu_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
kvm->mmu_notifier.ops = &kvm_mmu_notifier_ops;
|
|
|
|
return mmu_notifier_register(&kvm->mmu_notifier, current->mm);
|
|
|
|
}
|
|
|
|
|
|
|
|
#else /* !(CONFIG_MMU_NOTIFIER && KVM_ARCH_WANT_MMU_NOTIFIER) */
|
|
|
|
|
|
|
|
static int kvm_init_mmu_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-07-25 22:24:52 +08:00
|
|
|
#endif /* CONFIG_MMU_NOTIFIER && KVM_ARCH_WANT_MMU_NOTIFIER */
|
|
|
|
|
2021-06-06 10:10:44 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
|
|
|
|
static int kvm_pm_notifier_call(struct notifier_block *bl,
|
|
|
|
unsigned long state,
|
|
|
|
void *unused)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = container_of(bl, struct kvm, pm_notifier);
|
|
|
|
|
|
|
|
return kvm_arch_pm_notifier(kvm, state);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_init_pm_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
kvm->pm_notifier.notifier_call = kvm_pm_notifier_call;
|
|
|
|
/* Suspend KVM before we suspend ftrace, RCU, etc. */
|
|
|
|
kvm->pm_notifier.priority = INT_MAX;
|
|
|
|
register_pm_notifier(&kvm->pm_notifier);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_destroy_pm_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
unregister_pm_notifier(&kvm->pm_notifier);
|
|
|
|
}
|
|
|
|
#else /* !CONFIG_HAVE_KVM_PM_NOTIFIER */
|
|
|
|
static void kvm_init_pm_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_destroy_pm_notifier(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */
|
|
|
|
|
2015-05-17 17:41:37 +08:00
|
|
|
static struct kvm_memslots *kvm_alloc_memslots(void)
|
2011-11-24 17:40:57 +08:00
|
|
|
{
|
|
|
|
int i;
|
2015-05-17 17:41:37 +08:00
|
|
|
struct kvm_memslots *slots;
|
2011-11-24 17:40:57 +08:00
|
|
|
|
2019-02-12 03:02:49 +08:00
|
|
|
slots = kvzalloc(sizeof(struct kvm_memslots), GFP_KERNEL_ACCOUNT);
|
2015-05-17 17:41:37 +08:00
|
|
|
if (!slots)
|
|
|
|
return NULL;
|
|
|
|
|
2011-11-24 17:40:57 +08:00
|
|
|
for (i = 0; i < KVM_MEM_SLOTS_NUM; i++)
|
2020-02-19 05:07:32 +08:00
|
|
|
slots->id_to_index[i] = -1;
|
2015-05-17 17:41:37 +08:00
|
|
|
|
|
|
|
return slots;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_destroy_dirty_bitmap(struct kvm_memory_slot *memslot)
|
|
|
|
{
|
|
|
|
if (!memslot->dirty_bitmap)
|
|
|
|
return;
|
|
|
|
|
|
|
|
kvfree(memslot->dirty_bitmap);
|
|
|
|
memslot->dirty_bitmap = NULL;
|
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:27 +08:00
|
|
|
static void kvm_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot)
|
2015-05-17 17:41:37 +08:00
|
|
|
{
|
2020-02-19 05:07:27 +08:00
|
|
|
kvm_destroy_dirty_bitmap(slot);
|
2015-05-17 17:41:37 +08:00
|
|
|
|
2020-02-19 05:07:27 +08:00
|
|
|
kvm_arch_free_memslot(kvm, slot);
|
2015-05-17 17:41:37 +08:00
|
|
|
|
2020-02-19 05:07:27 +08:00
|
|
|
slot->flags = 0;
|
|
|
|
slot->npages = 0;
|
2015-05-17 17:41:37 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_free_memslots(struct kvm *kvm, struct kvm_memslots *slots)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *memslot;
|
|
|
|
|
|
|
|
if (!slots)
|
|
|
|
return;
|
|
|
|
|
|
|
|
kvm_for_each_memslot(memslot, slots)
|
2020-02-19 05:07:27 +08:00
|
|
|
kvm_free_memslot(kvm, memslot);
|
2015-05-17 17:41:37 +08:00
|
|
|
|
|
|
|
kvfree(slots);
|
2011-11-24 17:40:57 +08:00
|
|
|
}
|
|
|
|
|
2021-06-24 05:28:46 +08:00
|
|
|
static umode_t kvm_stats_debugfs_mode(const struct _kvm_stats_desc *pdesc)
|
|
|
|
{
|
|
|
|
switch (pdesc->desc.flags & KVM_STATS_TYPE_MASK) {
|
|
|
|
case KVM_STATS_TYPE_INSTANT:
|
|
|
|
return 0444;
|
|
|
|
case KVM_STATS_TYPE_CUMULATIVE:
|
|
|
|
case KVM_STATS_TYPE_PEAK:
|
|
|
|
default:
|
|
|
|
return 0644;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2016-05-18 19:26:23 +08:00
|
|
|
static void kvm_destroy_vm_debugfs(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
int i;
|
2021-06-24 05:28:46 +08:00
|
|
|
int kvm_debugfs_num_entries = kvm_vm_stats_header.num_desc +
|
|
|
|
kvm_vcpu_stats_header.num_desc;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
|
|
|
if (!kvm->debugfs_dentry)
|
|
|
|
return;
|
|
|
|
|
|
|
|
debugfs_remove_recursive(kvm->debugfs_dentry);
|
|
|
|
|
2016-09-08 02:47:21 +08:00
|
|
|
if (kvm->debugfs_stat_data) {
|
|
|
|
for (i = 0; i < kvm_debugfs_num_entries; i++)
|
|
|
|
kfree(kvm->debugfs_stat_data[i]);
|
|
|
|
kfree(kvm->debugfs_stat_data);
|
|
|
|
}
|
2016-05-18 19:26:23 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_create_vm_debugfs(struct kvm *kvm, int fd)
|
|
|
|
{
|
2021-08-04 17:28:52 +08:00
|
|
|
static DEFINE_MUTEX(kvm_debugfs_lock);
|
|
|
|
struct dentry *dent;
|
2016-05-18 19:26:23 +08:00
|
|
|
char dir_name[ITOA_MAX_LEN * 2];
|
|
|
|
struct kvm_stat_data *stat_data;
|
2021-06-24 05:28:46 +08:00
|
|
|
const struct _kvm_stats_desc *pdesc;
|
2021-07-31 06:04:49 +08:00
|
|
|
int i, ret;
|
2021-06-24 05:28:46 +08:00
|
|
|
int kvm_debugfs_num_entries = kvm_vm_stats_header.num_desc +
|
|
|
|
kvm_vcpu_stats_header.num_desc;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
|
|
|
if (!debugfs_initialized())
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
snprintf(dir_name, sizeof(dir_name), "%d-%d", task_pid_nr(current), fd);
|
2021-08-04 17:28:52 +08:00
|
|
|
mutex_lock(&kvm_debugfs_lock);
|
|
|
|
dent = debugfs_lookup(dir_name, kvm_debugfs_dir);
|
|
|
|
if (dent) {
|
|
|
|
pr_warn_ratelimited("KVM: debugfs: duplicate directory %s\n", dir_name);
|
|
|
|
dput(dent);
|
|
|
|
mutex_unlock(&kvm_debugfs_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
dent = debugfs_create_dir(dir_name, kvm_debugfs_dir);
|
|
|
|
mutex_unlock(&kvm_debugfs_lock);
|
|
|
|
if (IS_ERR(dent))
|
|
|
|
return 0;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2021-08-04 17:28:52 +08:00
|
|
|
kvm->debugfs_dentry = dent;
|
2016-05-18 19:26:23 +08:00
|
|
|
kvm->debugfs_stat_data = kcalloc(kvm_debugfs_num_entries,
|
|
|
|
sizeof(*kvm->debugfs_stat_data),
|
2019-02-12 03:02:49 +08:00
|
|
|
GFP_KERNEL_ACCOUNT);
|
2016-05-18 19:26:23 +08:00
|
|
|
if (!kvm->debugfs_stat_data)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
2021-06-24 05:28:46 +08:00
|
|
|
for (i = 0; i < kvm_vm_stats_header.num_desc; ++i) {
|
|
|
|
pdesc = &kvm_vm_stats_desc[i];
|
2019-02-12 03:02:49 +08:00
|
|
|
stat_data = kzalloc(sizeof(*stat_data), GFP_KERNEL_ACCOUNT);
|
2016-05-18 19:26:23 +08:00
|
|
|
if (!stat_data)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
stat_data->kvm = kvm;
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc = pdesc;
|
|
|
|
stat_data->kind = KVM_STAT_VM;
|
|
|
|
kvm->debugfs_stat_data[i] = stat_data;
|
|
|
|
debugfs_create_file(pdesc->name, kvm_stats_debugfs_mode(pdesc),
|
|
|
|
kvm->debugfs_dentry, stat_data,
|
|
|
|
&stat_fops_per_vm);
|
|
|
|
}
|
|
|
|
|
|
|
|
for (i = 0; i < kvm_vcpu_stats_header.num_desc; ++i) {
|
|
|
|
pdesc = &kvm_vcpu_stats_desc[i];
|
2019-02-12 03:02:49 +08:00
|
|
|
stat_data = kzalloc(sizeof(*stat_data), GFP_KERNEL_ACCOUNT);
|
2016-05-18 19:26:23 +08:00
|
|
|
if (!stat_data)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
stat_data->kvm = kvm;
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc = pdesc;
|
|
|
|
stat_data->kind = KVM_STAT_VCPU;
|
2021-07-02 03:55:00 +08:00
|
|
|
kvm->debugfs_stat_data[i + kvm_vm_stats_header.num_desc] = stat_data;
|
2021-06-24 05:28:46 +08:00
|
|
|
debugfs_create_file(pdesc->name, kvm_stats_debugfs_mode(pdesc),
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm->debugfs_dentry, stat_data,
|
|
|
|
&stat_fops_per_vm);
|
2016-05-18 19:26:23 +08:00
|
|
|
}
|
2021-07-31 06:04:49 +08:00
|
|
|
|
|
|
|
ret = kvm_arch_create_vm_debugfs(kvm);
|
|
|
|
if (ret) {
|
|
|
|
kvm_destroy_vm_debugfs(kvm);
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
|
2016-05-18 19:26:23 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-11-05 03:26:00 +08:00
|
|
|
/*
|
|
|
|
* Called after the VM is otherwise initialized, but just before adding it to
|
|
|
|
* the vm_list.
|
|
|
|
*/
|
|
|
|
int __weak kvm_arch_post_init_vm(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Called just after removing the VM from the vm_list, but before doing any
|
|
|
|
* other destruction.
|
|
|
|
*/
|
|
|
|
void __weak kvm_arch_pre_destroy_vm(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2021-07-31 06:04:49 +08:00
|
|
|
/*
|
|
|
|
* Called after per-vm debugfs created. When called kvm->debugfs_dentry should
|
|
|
|
* be setup already, so we can create arch-specific debugfs entries under it.
|
|
|
|
* Cleanup should be automatic done in kvm_destroy_vm_debugfs() recursively, so
|
|
|
|
* a per-arch destroy interface is not needed.
|
|
|
|
*/
|
|
|
|
int __weak kvm_arch_create_vm_debugfs(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-01-04 17:25:20 +08:00
|
|
|
static struct kvm *kvm_create_vm(unsigned long type)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2010-11-10 00:02:49 +08:00
|
|
|
struct kvm *kvm = kvm_arch_alloc_vm();
|
2019-10-25 07:03:26 +08:00
|
|
|
int r = -ENOMEM;
|
|
|
|
int i;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2010-11-10 00:02:49 +08:00
|
|
|
if (!kvm)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
2021-02-03 02:57:24 +08:00
|
|
|
KVM_MMU_LOCK_INIT(kvm);
|
2017-02-28 06:30:07 +08:00
|
|
|
mmgrab(current->mm);
|
2016-03-21 17:15:25 +08:00
|
|
|
kvm->mm = current->mm;
|
|
|
|
kvm_eventfd_init(kvm);
|
|
|
|
mutex_init(&kvm->lock);
|
|
|
|
mutex_init(&kvm->irq_lock);
|
|
|
|
mutex_init(&kvm->slots_lock);
|
2021-05-19 01:34:11 +08:00
|
|
|
mutex_init(&kvm->slots_arch_lock);
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
spin_lock_init(&kvm->mn_invalidate_lock);
|
|
|
|
rcuwait_init(&kvm->mn_memslots_update_rcuwait);
|
|
|
|
|
2016-03-21 17:15:25 +08:00
|
|
|
INIT_LIST_HEAD(&kvm->devices);
|
|
|
|
|
2012-12-11 01:33:32 +08:00
|
|
|
BUILD_BUG_ON(KVM_MEM_SLOTS_NUM > SHRT_MAX);
|
|
|
|
|
2019-11-04 19:16:49 +08:00
|
|
|
if (init_srcu_struct(&kvm->srcu))
|
|
|
|
goto out_err_no_srcu;
|
|
|
|
if (init_srcu_struct(&kvm->irq_srcu))
|
|
|
|
goto out_err_no_irq_srcu;
|
|
|
|
|
2019-11-04 20:23:53 +08:00
|
|
|
refcount_set(&kvm->users_count, 1);
|
2015-05-17 23:30:37 +08:00
|
|
|
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
|
2017-02-04 12:44:51 +08:00
|
|
|
struct kvm_memslots *slots = kvm_alloc_memslots();
|
2019-10-25 07:03:26 +08:00
|
|
|
|
2017-02-04 12:44:51 +08:00
|
|
|
if (!slots)
|
2019-10-25 19:34:58 +08:00
|
|
|
goto out_err_no_arch_destroy_vm;
|
KVM: Remove the hack to trigger memslot generation wraparound
x86 captures a subset of the memslot generation (19 bits) in its MMIO
sptes so that it can expedite emulated MMIO handling by checking only
the releveant spte, i.e. doesn't need to do a full page fault walk.
Because the MMIO sptes capture only 19 bits (due to limited space in
the sptes), there is a non-zero probability that the MMIO generation
could wrap, e.g. after 500k memslot updates. Since normal usage is
extremely unlikely to result in 500k memslot updates, a hack was added
by commit 69c9ea93eaea ("KVM: MMU: init kvm generation close to mmio
wrap-around value") to offset the MMIO generation in order to trigger
a wraparound, e.g. after 150 memslot updates.
When separate memslot generation sequences were assigned to each
address space, commit 00f034a12fdd ("KVM: do not bias the generation
number in kvm_current_mmio_generation") moved the offset logic into the
initialization of the memslot generation itself so that the per-address
space bit(s) were not dropped/corrupted by the MMIO shenanigans.
Remove the offset hack for three reasons:
- While it does exercise x86's kvm_mmu_invalidate_mmio_sptes(), simply
wrapping the generation doesn't actually test the interesting case
of having stale MMIO sptes with the new generation number, e.g. old
sptes with a generation number of 0.
- Triggering kvm_mmu_invalidate_mmio_sptes() prematurely makes its
performance rather important since the probability of invalidating
MMIO sptes jumps from "effectively never" to "fairly likely". This
limits what can be done in future patches, e.g. to simplify the
invalidation code, as doing so without proper caution could lead to
a noticeable performance regression.
- Forcing the memslots generation, which is a 64-bit number, to wrap
prevents KVM from assuming the memslots generation will never wrap.
This in turn prevents KVM from using an arbitrary bit for the
"update in-progress" flag, e.g. using bit 63 would immediately
collide with using a large value as the starting generation number.
The "update in-progress" flag is effectively forced into bit 0 so
that it's (subtly) taken into account when incrementing the
generation.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-06 05:01:17 +08:00
|
|
|
/* Generations must be different for each address space. */
|
2019-02-06 05:01:18 +08:00
|
|
|
slots->generation = i;
|
2017-02-04 12:44:51 +08:00
|
|
|
rcu_assign_pointer(kvm->memslots[i], slots);
|
2015-05-17 23:30:37 +08:00
|
|
|
}
|
2014-08-20 20:29:21 +08:00
|
|
|
|
2009-12-24 00:35:24 +08:00
|
|
|
for (i = 0; i < KVM_NR_BUSES; i++) {
|
2017-07-07 16:51:38 +08:00
|
|
|
rcu_assign_pointer(kvm->buses[i],
|
2019-02-12 03:02:49 +08:00
|
|
|
kzalloc(sizeof(struct kvm_io_bus), GFP_KERNEL_ACCOUNT));
|
2010-11-09 19:42:12 +08:00
|
|
|
if (!kvm->buses[i])
|
2019-10-25 19:34:58 +08:00
|
|
|
goto out_err_no_arch_destroy_vm;
|
2009-12-24 00:35:24 +08:00
|
|
|
}
|
2008-07-25 22:24:52 +08:00
|
|
|
|
2020-04-18 06:14:46 +08:00
|
|
|
kvm->max_halt_poll_ns = halt_poll_ns;
|
|
|
|
|
2012-01-04 17:25:20 +08:00
|
|
|
r = kvm_arch_init_vm(kvm, type);
|
2010-11-10 00:02:49 +08:00
|
|
|
if (r)
|
2019-10-25 19:34:58 +08:00
|
|
|
goto out_err_no_arch_destroy_vm;
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
r = hardware_enable_all();
|
|
|
|
if (r)
|
2014-01-16 20:44:20 +08:00
|
|
|
goto out_err_no_disable;
|
2009-09-15 17:37:46 +08:00
|
|
|
|
2014-08-06 20:24:45 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_IRQFD
|
2009-08-24 16:54:23 +08:00
|
|
|
INIT_HLIST_HEAD(&kvm->irq_ack_notifier_list);
|
2009-01-04 23:10:50 +08:00
|
|
|
#endif
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2011-06-04 04:04:53 +08:00
|
|
|
r = kvm_init_mmu_notifier(kvm);
|
2019-11-05 03:26:00 +08:00
|
|
|
if (r)
|
|
|
|
goto out_err_no_mmu_notifier;
|
|
|
|
|
|
|
|
r = kvm_arch_post_init_vm(kvm);
|
2011-06-04 04:04:53 +08:00
|
|
|
if (r)
|
|
|
|
goto out_err;
|
|
|
|
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2007-07-23 15:08:21 +08:00
|
|
|
list_add(&kvm->vm_list, &vm_list);
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2010-11-10 00:02:49 +08:00
|
|
|
|
2015-07-04 00:53:58 +08:00
|
|
|
preempt_notifier_inc();
|
2021-06-06 10:10:44 +08:00
|
|
|
kvm_init_pm_notifier(kvm);
|
2015-07-04 00:53:58 +08:00
|
|
|
|
2007-02-22 01:28:04 +08:00
|
|
|
return kvm;
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
out_err:
|
2019-11-05 03:26:00 +08:00
|
|
|
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
|
|
|
|
if (kvm->mmu_notifier.ops)
|
|
|
|
mmu_notifier_unregister(&kvm->mmu_notifier, current->mm);
|
|
|
|
#endif
|
|
|
|
out_err_no_mmu_notifier:
|
2009-09-15 17:37:46 +08:00
|
|
|
hardware_disable_all();
|
2014-01-16 20:44:20 +08:00
|
|
|
out_err_no_disable:
|
2019-10-25 19:34:58 +08:00
|
|
|
kvm_arch_destroy_vm(kvm);
|
|
|
|
out_err_no_arch_destroy_vm:
|
2019-11-04 20:23:53 +08:00
|
|
|
WARN_ON_ONCE(!refcount_dec_and_test(&kvm->users_count));
|
2009-12-24 00:35:24 +08:00
|
|
|
for (i = 0; i < KVM_NR_BUSES; i++)
|
2017-08-02 23:55:54 +08:00
|
|
|
kfree(kvm_get_bus(kvm, i));
|
2015-05-17 23:30:37 +08:00
|
|
|
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
|
2017-08-02 23:55:54 +08:00
|
|
|
kvm_free_memslots(kvm, __kvm_memslots(kvm, i));
|
2019-11-04 19:16:49 +08:00
|
|
|
cleanup_srcu_struct(&kvm->irq_srcu);
|
|
|
|
out_err_no_irq_srcu:
|
|
|
|
cleanup_srcu_struct(&kvm->srcu);
|
|
|
|
out_err_no_srcu:
|
2010-11-10 00:02:49 +08:00
|
|
|
kvm_arch_free_vm(kvm);
|
2016-03-21 17:15:25 +08:00
|
|
|
mmdrop(current->mm);
|
2009-09-15 17:37:46 +08:00
|
|
|
return ERR_PTR(r);
|
2007-02-22 01:28:04 +08:00
|
|
|
}
|
|
|
|
|
2013-04-25 22:11:23 +08:00
|
|
|
static void kvm_destroy_devices(struct kvm *kvm)
|
|
|
|
{
|
2016-01-01 19:47:12 +08:00
|
|
|
struct kvm_device *dev, *tmp;
|
2013-04-25 22:11:23 +08:00
|
|
|
|
2016-08-10 01:13:01 +08:00
|
|
|
/*
|
|
|
|
* We do not need to take the kvm->lock here, because nobody else
|
|
|
|
* has a reference to the struct kvm at this point and therefore
|
|
|
|
* cannot access the devices list anyhow.
|
|
|
|
*/
|
2016-01-01 19:47:12 +08:00
|
|
|
list_for_each_entry_safe(dev, tmp, &kvm->devices, vm_node) {
|
|
|
|
list_del(&dev->vm_node);
|
2013-04-25 22:11:23 +08:00
|
|
|
dev->ops->destroy(dev);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2007-02-22 01:28:04 +08:00
|
|
|
static void kvm_destroy_vm(struct kvm *kvm)
|
|
|
|
{
|
2009-12-24 00:35:24 +08:00
|
|
|
int i;
|
2007-11-21 22:41:05 +08:00
|
|
|
struct mm_struct *mm = kvm->mm;
|
|
|
|
|
2021-06-06 10:10:44 +08:00
|
|
|
kvm_destroy_pm_notifier(kvm);
|
2017-07-12 23:56:44 +08:00
|
|
|
kvm_uevent_notify_change(KVM_EVENT_DESTROY_VM, kvm);
|
2016-05-18 19:26:23 +08:00
|
|
|
kvm_destroy_vm_debugfs(kvm);
|
2009-01-06 10:03:02 +08:00
|
|
|
kvm_arch_sync_events(kvm);
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2007-02-12 16:54:44 +08:00
|
|
|
list_del(&kvm->vm_list);
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2019-11-05 03:26:00 +08:00
|
|
|
kvm_arch_pre_destroy_vm(kvm);
|
|
|
|
|
2008-11-19 19:58:46 +08:00
|
|
|
kvm_free_irq_routing(kvm);
|
2017-03-15 16:01:17 +08:00
|
|
|
for (i = 0; i < KVM_NR_BUSES; i++) {
|
2017-08-02 23:55:54 +08:00
|
|
|
struct kvm_io_bus *bus = kvm_get_bus(kvm, i);
|
2017-07-07 16:51:38 +08:00
|
|
|
|
|
|
|
if (bus)
|
|
|
|
kvm_io_bus_destroy(bus);
|
2017-03-15 16:01:17 +08:00
|
|
|
kvm->buses[i] = NULL;
|
|
|
|
}
|
2009-12-20 21:13:43 +08:00
|
|
|
kvm_coalesced_mmio_free(kvm);
|
2008-07-25 22:24:52 +08:00
|
|
|
#if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
|
|
|
|
mmu_notifier_unregister(&kvm->mmu_notifier, kvm->mm);
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
/*
|
|
|
|
* At this point, pending calls to invalidate_range_start()
|
|
|
|
* have completed but no more MMU notifiers will run, so
|
|
|
|
* mn_active_invalidate_count may remain unbalanced.
|
|
|
|
* No threads can be waiting in install_new_memslots as the
|
|
|
|
* last reference on KVM has been dropped, but freeing
|
|
|
|
* memslots would deadlock without this manual intervention.
|
|
|
|
*/
|
|
|
|
WARN_ON(rcuwait_active(&kvm->mn_memslots_update_rcuwait));
|
|
|
|
kvm->mn_active_invalidate_count = 0;
|
2009-03-19 18:20:36 +08:00
|
|
|
#else
|
2012-08-25 02:54:57 +08:00
|
|
|
kvm_arch_flush_shadow_all(kvm);
|
2008-05-30 22:05:54 +08:00
|
|
|
#endif
|
2007-11-18 18:43:45 +08:00
|
|
|
kvm_arch_destroy_vm(kvm);
|
2013-04-25 22:11:23 +08:00
|
|
|
kvm_destroy_devices(kvm);
|
2015-05-17 23:30:37 +08:00
|
|
|
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
|
2017-08-02 23:55:54 +08:00
|
|
|
kvm_free_memslots(kvm, __kvm_memslots(kvm, i));
|
2014-06-03 19:44:17 +08:00
|
|
|
cleanup_srcu_struct(&kvm->irq_srcu);
|
2010-11-10 00:02:49 +08:00
|
|
|
cleanup_srcu_struct(&kvm->srcu);
|
|
|
|
kvm_arch_free_vm(kvm);
|
2015-07-04 00:53:58 +08:00
|
|
|
preempt_notifier_dec();
|
2009-09-15 17:37:46 +08:00
|
|
|
hardware_disable_all();
|
2007-11-21 22:41:05 +08:00
|
|
|
mmdrop(mm);
|
2007-02-22 01:28:04 +08:00
|
|
|
}
|
|
|
|
|
2008-03-30 21:01:25 +08:00
|
|
|
void kvm_get_kvm(struct kvm *kvm)
|
|
|
|
{
|
2017-02-20 19:06:21 +08:00
|
|
|
refcount_inc(&kvm->users_count);
|
2008-03-30 21:01:25 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_get_kvm);
|
|
|
|
|
2021-06-25 23:32:07 +08:00
|
|
|
/*
|
|
|
|
* Make sure the vm is not during destruction, which is a safe version of
|
|
|
|
* kvm_get_kvm(). Return true if kvm referenced successfully, false otherwise.
|
|
|
|
*/
|
|
|
|
bool kvm_get_kvm_safe(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
return refcount_inc_not_zero(&kvm->users_count);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_get_kvm_safe);
|
|
|
|
|
2008-03-30 21:01:25 +08:00
|
|
|
void kvm_put_kvm(struct kvm *kvm)
|
|
|
|
{
|
2017-02-20 19:06:21 +08:00
|
|
|
if (refcount_dec_and_test(&kvm->users_count))
|
2008-03-30 21:01:25 +08:00
|
|
|
kvm_destroy_vm(kvm);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_put_kvm);
|
|
|
|
|
2019-10-22 06:58:42 +08:00
|
|
|
/*
|
|
|
|
* Used to put a reference that was taken on behalf of an object associated
|
|
|
|
* with a user-visible file descriptor, e.g. a vcpu or device, if installation
|
|
|
|
* of the new file descriptor fails and the reference cannot be transferred to
|
|
|
|
* its final owner. In such cases, the caller is still actively using @kvm and
|
|
|
|
* will fail miserably if the refcount unexpectedly hits zero.
|
|
|
|
*/
|
|
|
|
void kvm_put_kvm_no_destroy(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
WARN_ON(refcount_dec_and_test(&kvm->users_count));
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_put_kvm_no_destroy);
|
2008-03-30 21:01:25 +08:00
|
|
|
|
2007-02-22 01:28:04 +08:00
|
|
|
static int kvm_vm_release(struct inode *inode, struct file *filp)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = filp->private_data;
|
|
|
|
|
2009-05-20 22:30:49 +08:00
|
|
|
kvm_irqfd_release(kvm);
|
|
|
|
|
2008-03-30 21:01:25 +08:00
|
|
|
kvm_put_kvm(kvm);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2010-10-27 17:23:54 +08:00
|
|
|
/*
|
|
|
|
* Allocation size is twice as large as the actual dirty bitmap size.
|
2020-02-19 05:07:29 +08:00
|
|
|
* See kvm_vm_ioctl_get_dirty_log() why this is needed.
|
2010-10-27 17:23:54 +08:00
|
|
|
*/
|
2020-02-27 09:32:27 +08:00
|
|
|
static int kvm_alloc_dirty_bitmap(struct kvm_memory_slot *memslot)
|
2010-10-27 17:22:19 +08:00
|
|
|
{
|
2010-10-27 17:23:54 +08:00
|
|
|
unsigned long dirty_bytes = 2 * kvm_dirty_bitmap_bytes(memslot);
|
2010-10-27 17:22:19 +08:00
|
|
|
|
2019-02-12 03:02:49 +08:00
|
|
|
memslot->dirty_bitmap = kvzalloc(dirty_bytes, GFP_KERNEL_ACCOUNT);
|
2010-10-27 17:22:19 +08:00
|
|
|
if (!memslot->dirty_bitmap)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2011-11-24 17:40:57 +08:00
|
|
|
/*
|
2020-02-19 05:07:31 +08:00
|
|
|
* Delete a memslot by decrementing the number of used slots and shifting all
|
|
|
|
* other entries in the array forward one spot.
|
2011-11-24 17:40:57 +08:00
|
|
|
*/
|
2020-02-19 05:07:31 +08:00
|
|
|
static inline void kvm_memslot_delete(struct kvm_memslots *slots,
|
|
|
|
struct kvm_memory_slot *memslot)
|
2011-11-24 17:40:57 +08:00
|
|
|
{
|
2014-11-14 07:00:13 +08:00
|
|
|
struct kvm_memory_slot *mslots = slots->memslots;
|
2020-02-19 05:07:31 +08:00
|
|
|
int i;
|
2011-11-24 17:41:54 +08:00
|
|
|
|
2020-02-19 05:07:31 +08:00
|
|
|
if (WARN_ON(slots->id_to_index[memslot->id] == -1))
|
|
|
|
return;
|
2014-12-02 01:29:26 +08:00
|
|
|
|
2020-02-19 05:07:31 +08:00
|
|
|
slots->used_slots--;
|
|
|
|
|
2021-08-05 06:28:38 +08:00
|
|
|
if (atomic_read(&slots->last_used_slot) >= slots->used_slots)
|
|
|
|
atomic_set(&slots->last_used_slot, 0);
|
2020-03-21 04:55:40 +08:00
|
|
|
|
2020-02-19 05:07:31 +08:00
|
|
|
for (i = slots->id_to_index[memslot->id]; i < slots->used_slots; i++) {
|
2014-12-02 01:29:24 +08:00
|
|
|
mslots[i] = mslots[i + 1];
|
|
|
|
slots->id_to_index[mslots[i].id] = i;
|
|
|
|
}
|
2020-02-19 05:07:31 +08:00
|
|
|
mslots[i] = *memslot;
|
|
|
|
slots->id_to_index[memslot->id] = -1;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* "Insert" a new memslot by incrementing the number of used slots. Returns
|
|
|
|
* the new slot's initial index into the memslots array.
|
|
|
|
*/
|
|
|
|
static inline int kvm_memslot_insert_back(struct kvm_memslots *slots)
|
|
|
|
{
|
|
|
|
return slots->used_slots++;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Move a changed memslot backwards in the array by shifting existing slots
|
|
|
|
* with a higher GFN toward the front of the array. Note, the changed memslot
|
|
|
|
* itself is not preserved in the array, i.e. not swapped at this time, only
|
|
|
|
* its new index into the array is tracked. Returns the changed memslot's
|
|
|
|
* current index into the memslots array.
|
|
|
|
*/
|
|
|
|
static inline int kvm_memslot_move_backward(struct kvm_memslots *slots,
|
|
|
|
struct kvm_memory_slot *memslot)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *mslots = slots->memslots;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
if (WARN_ON_ONCE(slots->id_to_index[memslot->id] == -1) ||
|
|
|
|
WARN_ON_ONCE(!slots->used_slots))
|
|
|
|
return -1;
|
2014-12-28 01:01:00 +08:00
|
|
|
|
|
|
|
/*
|
2020-02-19 05:07:31 +08:00
|
|
|
* Move the target memslot backward in the array by shifting existing
|
|
|
|
* memslots with a higher GFN (than the target memslot) towards the
|
|
|
|
* front of the array.
|
2014-12-28 01:01:00 +08:00
|
|
|
*/
|
2020-02-19 05:07:31 +08:00
|
|
|
for (i = slots->id_to_index[memslot->id]; i < slots->used_slots - 1; i++) {
|
|
|
|
if (memslot->base_gfn > mslots[i + 1].base_gfn)
|
|
|
|
break;
|
|
|
|
|
|
|
|
WARN_ON_ONCE(memslot->base_gfn == mslots[i + 1].base_gfn);
|
2011-11-24 17:41:54 +08:00
|
|
|
|
2020-02-19 05:07:31 +08:00
|
|
|
/* Shift the next memslot forward one and update its index. */
|
|
|
|
mslots[i] = mslots[i + 1];
|
|
|
|
slots->id_to_index[mslots[i].id] = i;
|
|
|
|
}
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Move a changed memslot forwards in the array by shifting existing slots with
|
|
|
|
* a lower GFN toward the back of the array. Note, the changed memslot itself
|
|
|
|
* is not preserved in the array, i.e. not swapped at this time, only its new
|
|
|
|
* index into the array is tracked. Returns the changed memslot's final index
|
|
|
|
* into the memslots array.
|
|
|
|
*/
|
|
|
|
static inline int kvm_memslot_move_forward(struct kvm_memslots *slots,
|
|
|
|
struct kvm_memory_slot *memslot,
|
|
|
|
int start)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *mslots = slots->memslots;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = start; i > 0; i--) {
|
|
|
|
if (memslot->base_gfn < mslots[i - 1].base_gfn)
|
|
|
|
break;
|
|
|
|
|
|
|
|
WARN_ON_ONCE(memslot->base_gfn == mslots[i - 1].base_gfn);
|
|
|
|
|
|
|
|
/* Shift the next memslot back one and update its index. */
|
|
|
|
mslots[i] = mslots[i - 1];
|
|
|
|
slots->id_to_index[mslots[i].id] = i;
|
|
|
|
}
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Re-sort memslots based on their GFN to account for an added, deleted, or
|
|
|
|
* moved memslot. Sorting memslots by GFN allows using a binary search during
|
|
|
|
* memslot lookup.
|
|
|
|
*
|
|
|
|
* IMPORTANT: Slots are sorted from highest GFN to lowest GFN! I.e. the entry
|
|
|
|
* at memslots[0] has the highest GFN.
|
|
|
|
*
|
|
|
|
* The sorting algorithm takes advantage of having initially sorted memslots
|
|
|
|
* and knowing the position of the changed memslot. Sorting is also optimized
|
|
|
|
* by not swapping the updated memslot and instead only shifting other memslots
|
|
|
|
* and tracking the new index for the update memslot. Only once its final
|
|
|
|
* index is known is the updated memslot copied into its position in the array.
|
|
|
|
*
|
|
|
|
* - When deleting a memslot, the deleted memslot simply needs to be moved to
|
|
|
|
* the end of the array.
|
|
|
|
*
|
|
|
|
* - When creating a memslot, the algorithm "inserts" the new memslot at the
|
|
|
|
* end of the array and then it forward to its correct location.
|
|
|
|
*
|
|
|
|
* - When moving a memslot, the algorithm first moves the updated memslot
|
|
|
|
* backward to handle the scenario where the memslot's GFN was changed to a
|
|
|
|
* lower value. update_memslots() then falls through and runs the same flow
|
|
|
|
* as creating a memslot to move the memslot forward to handle the scenario
|
|
|
|
* where its GFN was changed to a higher value.
|
|
|
|
*
|
|
|
|
* Note, slots are sorted from highest->lowest instead of lowest->highest for
|
|
|
|
* historical reasons. Originally, invalid memslots where denoted by having
|
|
|
|
* GFN=0, thus sorting from highest->lowest naturally sorted invalid memslots
|
|
|
|
* to the end of the array. The current algorithm uses dedicated logic to
|
|
|
|
* delete a memslot and thus does not rely on invalid memslots having GFN=0.
|
|
|
|
*
|
|
|
|
* The other historical motiviation for highest->lowest was to improve the
|
|
|
|
* performance of memslot lookup. KVM originally used a linear search starting
|
|
|
|
* at memslots[0]. On x86, the largest memslot usually has one of the highest,
|
|
|
|
* if not *the* highest, GFN, as the bulk of the guest's RAM is located in a
|
|
|
|
* single memslot above the 4gb boundary. As the largest memslot is also the
|
|
|
|
* most likely to be referenced, sorting it to the front of the array was
|
|
|
|
* advantageous. The current binary search starts from the middle of the array
|
|
|
|
* and uses an LRU pointer to improve performance for all memslots and GFNs.
|
|
|
|
*/
|
|
|
|
static void update_memslots(struct kvm_memslots *slots,
|
|
|
|
struct kvm_memory_slot *memslot,
|
|
|
|
enum kvm_mr_change change)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
if (change == KVM_MR_DELETE) {
|
|
|
|
kvm_memslot_delete(slots, memslot);
|
|
|
|
} else {
|
|
|
|
if (change == KVM_MR_CREATE)
|
|
|
|
i = kvm_memslot_insert_back(slots);
|
|
|
|
else
|
|
|
|
i = kvm_memslot_move_backward(slots, memslot);
|
|
|
|
i = kvm_memslot_move_forward(slots, memslot, i);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Copy the memslot to its new position in memslots and update
|
|
|
|
* its index accordingly.
|
|
|
|
*/
|
|
|
|
slots->memslots[i] = *memslot;
|
|
|
|
slots->id_to_index[memslot->id] = i;
|
|
|
|
}
|
2011-11-24 17:40:57 +08:00
|
|
|
}
|
|
|
|
|
2015-05-18 19:59:39 +08:00
|
|
|
static int check_memory_region_flags(const struct kvm_userspace_memory_region *mem)
|
2012-08-21 10:58:13 +08:00
|
|
|
{
|
2012-08-21 11:02:51 +08:00
|
|
|
u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES;
|
|
|
|
|
2014-08-26 20:00:37 +08:00
|
|
|
#ifdef __KVM_HAVE_READONLY_MEM
|
2012-08-21 11:02:51 +08:00
|
|
|
valid_flags |= KVM_MEM_READONLY;
|
|
|
|
#endif
|
|
|
|
|
|
|
|
if (mem->flags & ~valid_flags)
|
2012-08-21 10:58:13 +08:00
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-12-24 23:49:30 +08:00
|
|
|
static struct kvm_memslots *install_new_memslots(struct kvm *kvm,
|
2015-05-17 23:30:37 +08:00
|
|
|
int as_id, struct kvm_memslots *slots)
|
2012-12-24 23:49:30 +08:00
|
|
|
{
|
2015-05-17 23:30:37 +08:00
|
|
|
struct kvm_memslots *old_memslots = __kvm_memslots(kvm, as_id);
|
KVM: Explicitly define the "memslot update in-progress" bit
KVM uses bit 0 of the memslots generation as an "update in-progress"
flag, which is used by x86 to prevent caching MMIO access while the
memslots are changing. Although the intended behavior is flag-like,
e.g. MMIO sptes intentionally drop the in-progress bit so as to avoid
caching data from in-flux memslots, the implementation oftentimes treats
the bit as part of the generation number itself, e.g. incrementing the
generation increments twice, once to set the flag and once to clear it.
Prior to commit 4bd518f1598d ("KVM: use separate generations for
each address space"), incorporating the "update in-progress" bit into
the generation number largely made sense, e.g. "real" generations are
even, "bogus" generations are odd, most code doesn't need to be aware of
the bit, etc...
Now that unique memslots generation numbers are assigned to each address
space, stealthing the in-progress status into the generation number
results in a wide variety of subtle code, e.g. kvm_create_vm() jumps
over bit 0 when initializing the memslots generation without any hint as
to why.
Explicitly define the flag and convert as much code as possible (which
isn't much) to actually treat it like a flag. This paves the way for
eventually using a different bit for "update in-progress" so that it can
be a flag in truth instead of a awkward extension to the generation
number.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-06 05:01:14 +08:00
|
|
|
u64 gen = old_memslots->generation;
|
2012-12-24 23:49:30 +08:00
|
|
|
|
KVM: Explicitly define the "memslot update in-progress" bit
KVM uses bit 0 of the memslots generation as an "update in-progress"
flag, which is used by x86 to prevent caching MMIO access while the
memslots are changing. Although the intended behavior is flag-like,
e.g. MMIO sptes intentionally drop the in-progress bit so as to avoid
caching data from in-flux memslots, the implementation oftentimes treats
the bit as part of the generation number itself, e.g. incrementing the
generation increments twice, once to set the flag and once to clear it.
Prior to commit 4bd518f1598d ("KVM: use separate generations for
each address space"), incorporating the "update in-progress" bit into
the generation number largely made sense, e.g. "real" generations are
even, "bogus" generations are odd, most code doesn't need to be aware of
the bit, etc...
Now that unique memslots generation numbers are assigned to each address
space, stealthing the in-progress status into the generation number
results in a wide variety of subtle code, e.g. kvm_create_vm() jumps
over bit 0 when initializing the memslots generation without any hint as
to why.
Explicitly define the flag and convert as much code as possible (which
isn't much) to actually treat it like a flag. This paves the way for
eventually using a different bit for "update in-progress" so that it can
be a flag in truth instead of a awkward extension to the generation
number.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-06 05:01:14 +08:00
|
|
|
WARN_ON(gen & KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS);
|
|
|
|
slots->generation = gen | KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS;
|
2014-08-19 06:46:06 +08:00
|
|
|
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
/*
|
|
|
|
* Do not store the new memslots while there are invalidations in
|
2021-08-03 15:45:41 +08:00
|
|
|
* progress, otherwise the locking in invalidate_range_start and
|
|
|
|
* invalidate_range_end will be unbalanced.
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
*/
|
|
|
|
spin_lock(&kvm->mn_invalidate_lock);
|
|
|
|
prepare_to_rcuwait(&kvm->mn_memslots_update_rcuwait);
|
|
|
|
while (kvm->mn_active_invalidate_count) {
|
|
|
|
set_current_state(TASK_UNINTERRUPTIBLE);
|
|
|
|
spin_unlock(&kvm->mn_invalidate_lock);
|
|
|
|
schedule();
|
|
|
|
spin_lock(&kvm->mn_invalidate_lock);
|
|
|
|
}
|
|
|
|
finish_rcuwait(&kvm->mn_memslots_update_rcuwait);
|
2015-05-17 23:30:37 +08:00
|
|
|
rcu_assign_pointer(kvm->memslots[as_id], slots);
|
KVM: Block memslot updates across range_start() and range_end()
We would like to avoid taking mmu_lock for .invalidate_range_{start,end}()
notifications that are unrelated to KVM. Because mmu_notifier_count
must be modified while holding mmu_lock for write, and must always
be paired across start->end to stay balanced, lock elision must
happen in both or none. Therefore, in preparation for this change,
this patch prevents memslot updates across range_start() and range_end().
Note, technically flag-only memslot updates could be allowed in parallel,
but stalling a memslot update for a relatively short amount of time is
not a scalability issue, and this is all more than complex enough.
A long note on the locking: a previous version of the patch used an rwsem
to block the memslot update while the MMU notifier run, but this resulted
in the following deadlock involving the pseudo-lock tagged as
"mmu_notifier_invalidate_range_start".
======================================================
WARNING: possible circular locking dependency detected
5.12.0-rc3+ #6 Tainted: G OE
------------------------------------------------------
qemu-system-x86/3069 is trying to acquire lock:
ffffffff9c775ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: __mmu_notifier_invalidate_range_end+0x5/0x190
but task is already holding lock:
ffffaff7410a9160 (&kvm->mmu_notifier_slots_lock){.+.+}-{3:3}, at: kvm_mmu_notifier_invalidate_range_start+0x36d/0x4f0 [kvm]
which lock already depends on the new lock.
This corresponds to the following MMU notifier logic:
invalidate_range_start
take pseudo lock
down_read() (*)
release pseudo lock
invalidate_range_end
take pseudo lock (**)
up_read()
release pseudo lock
At point (*) we take the mmu_notifiers_slots_lock inside the pseudo lock;
at point (**) we take the pseudo lock inside the mmu_notifiers_slots_lock.
This could cause a deadlock (ignoring for a second that the pseudo lock
is not a lock):
- invalidate_range_start waits on down_read(), because the rwsem is
held by install_new_memslots
- install_new_memslots waits on down_write(), because the rwsem is
held till (another) invalidate_range_end finishes
- invalidate_range_end sits waits on the pseudo lock, held by
invalidate_range_start.
Removing the fairness of the rwsem breaks the cycle (in lockdep terms,
it would change the *shared* rwsem readers into *shared recursive*
readers), so open-code the wait using a readers count and a
spinlock. This also allows handling blockable and non-blockable
critical section in the same way.
Losing the rwsem fairness does theoretically allow MMU notifiers to
block install_new_memslots forever. Note that mm/mmu_notifier.c's own
retry scheme in mmu_interval_read_begin also uses wait/wake_up
and is likewise not fair.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-27 20:09:15 +08:00
|
|
|
spin_unlock(&kvm->mn_invalidate_lock);
|
2021-05-19 01:34:11 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Acquired in kvm_set_memslot. Must be released before synchronize
|
|
|
|
* SRCU below in order to avoid deadlock with another thread
|
|
|
|
* acquiring the slots_arch_lock in an srcu critical section.
|
|
|
|
*/
|
|
|
|
mutex_unlock(&kvm->slots_arch_lock);
|
|
|
|
|
2012-12-24 23:49:30 +08:00
|
|
|
synchronize_srcu_expedited(&kvm->srcu);
|
2013-07-04 12:40:29 +08:00
|
|
|
|
2014-08-19 06:46:06 +08:00
|
|
|
/*
|
KVM: Explicitly define the "memslot update in-progress" bit
KVM uses bit 0 of the memslots generation as an "update in-progress"
flag, which is used by x86 to prevent caching MMIO access while the
memslots are changing. Although the intended behavior is flag-like,
e.g. MMIO sptes intentionally drop the in-progress bit so as to avoid
caching data from in-flux memslots, the implementation oftentimes treats
the bit as part of the generation number itself, e.g. incrementing the
generation increments twice, once to set the flag and once to clear it.
Prior to commit 4bd518f1598d ("KVM: use separate generations for
each address space"), incorporating the "update in-progress" bit into
the generation number largely made sense, e.g. "real" generations are
even, "bogus" generations are odd, most code doesn't need to be aware of
the bit, etc...
Now that unique memslots generation numbers are assigned to each address
space, stealthing the in-progress status into the generation number
results in a wide variety of subtle code, e.g. kvm_create_vm() jumps
over bit 0 when initializing the memslots generation without any hint as
to why.
Explicitly define the flag and convert as much code as possible (which
isn't much) to actually treat it like a flag. This paves the way for
eventually using a different bit for "update in-progress" so that it can
be a flag in truth instead of a awkward extension to the generation
number.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-06 05:01:14 +08:00
|
|
|
* Increment the new memslot generation a second time, dropping the
|
2019-12-11 14:26:23 +08:00
|
|
|
* update in-progress flag and incrementing the generation based on
|
KVM: Explicitly define the "memslot update in-progress" bit
KVM uses bit 0 of the memslots generation as an "update in-progress"
flag, which is used by x86 to prevent caching MMIO access while the
memslots are changing. Although the intended behavior is flag-like,
e.g. MMIO sptes intentionally drop the in-progress bit so as to avoid
caching data from in-flux memslots, the implementation oftentimes treats
the bit as part of the generation number itself, e.g. incrementing the
generation increments twice, once to set the flag and once to clear it.
Prior to commit 4bd518f1598d ("KVM: use separate generations for
each address space"), incorporating the "update in-progress" bit into
the generation number largely made sense, e.g. "real" generations are
even, "bogus" generations are odd, most code doesn't need to be aware of
the bit, etc...
Now that unique memslots generation numbers are assigned to each address
space, stealthing the in-progress status into the generation number
results in a wide variety of subtle code, e.g. kvm_create_vm() jumps
over bit 0 when initializing the memslots generation without any hint as
to why.
Explicitly define the flag and convert as much code as possible (which
isn't much) to actually treat it like a flag. This paves the way for
eventually using a different bit for "update in-progress" so that it can
be a flag in truth instead of a awkward extension to the generation
number.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-06 05:01:14 +08:00
|
|
|
* the number of address spaces. This provides a unique and easily
|
|
|
|
* identifiable generation number while the memslots are in flux.
|
|
|
|
*/
|
|
|
|
gen = slots->generation & ~KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS;
|
|
|
|
|
|
|
|
/*
|
2017-02-04 12:44:51 +08:00
|
|
|
* Generations must be unique even across address spaces. We do not need
|
|
|
|
* a global counter for that, instead the generation space is evenly split
|
|
|
|
* across address spaces. For example, with two address spaces, address
|
2019-02-06 05:01:18 +08:00
|
|
|
* space 0 will use generations 0, 2, 4, ... while address space 1 will
|
|
|
|
* use generations 1, 3, 5, ...
|
2014-08-19 06:46:06 +08:00
|
|
|
*/
|
2019-02-06 05:01:18 +08:00
|
|
|
gen += KVM_ADDRESS_SPACE_NUM;
|
2014-08-19 06:46:06 +08:00
|
|
|
|
2019-02-06 04:54:17 +08:00
|
|
|
kvm_arch_memslots_updated(kvm, gen);
|
2014-08-19 06:46:06 +08:00
|
|
|
|
2019-02-06 04:54:17 +08:00
|
|
|
slots->generation = gen;
|
2013-07-04 12:40:29 +08:00
|
|
|
|
|
|
|
return old_memslots;
|
2012-12-24 23:49:30 +08:00
|
|
|
}
|
|
|
|
|
2021-05-19 01:34:10 +08:00
|
|
|
static size_t kvm_memslots_size(int slots)
|
|
|
|
{
|
|
|
|
return sizeof(struct kvm_memslots) +
|
|
|
|
(sizeof(struct kvm_memory_slot) * slots);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_copy_memslots(struct kvm_memslots *to,
|
|
|
|
struct kvm_memslots *from)
|
|
|
|
{
|
|
|
|
memcpy(to, from, kvm_memslots_size(from->used_slots));
|
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:32 +08:00
|
|
|
/*
|
|
|
|
* Note, at a minimum, the current number of used slots must be allocated, even
|
|
|
|
* when deleting a memslot, as we need a complete duplicate of the memslots for
|
|
|
|
* use when invalidating a memslot prior to deleting/moving the memslot.
|
|
|
|
*/
|
|
|
|
static struct kvm_memslots *kvm_dup_memslots(struct kvm_memslots *old,
|
|
|
|
enum kvm_mr_change change)
|
|
|
|
{
|
|
|
|
struct kvm_memslots *slots;
|
2021-05-19 01:34:10 +08:00
|
|
|
size_t new_size;
|
2020-02-19 05:07:32 +08:00
|
|
|
|
|
|
|
if (change == KVM_MR_CREATE)
|
2021-05-19 01:34:10 +08:00
|
|
|
new_size = kvm_memslots_size(old->used_slots + 1);
|
2020-02-19 05:07:32 +08:00
|
|
|
else
|
2021-05-19 01:34:10 +08:00
|
|
|
new_size = kvm_memslots_size(old->used_slots);
|
2020-02-19 05:07:32 +08:00
|
|
|
|
|
|
|
slots = kvzalloc(new_size, GFP_KERNEL_ACCOUNT);
|
|
|
|
if (likely(slots))
|
2021-05-19 01:34:10 +08:00
|
|
|
kvm_copy_memslots(slots, old);
|
2020-02-19 05:07:32 +08:00
|
|
|
|
|
|
|
return slots;
|
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:23 +08:00
|
|
|
static int kvm_set_memslot(struct kvm *kvm,
|
|
|
|
const struct kvm_userspace_memory_region *mem,
|
2020-02-19 05:07:24 +08:00
|
|
|
struct kvm_memory_slot *old,
|
2020-02-19 05:07:23 +08:00
|
|
|
struct kvm_memory_slot *new, int as_id,
|
|
|
|
enum kvm_mr_change change)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot;
|
|
|
|
struct kvm_memslots *slots;
|
|
|
|
int r;
|
|
|
|
|
2021-05-19 01:34:11 +08:00
|
|
|
/*
|
|
|
|
* Released in install_new_memslots.
|
|
|
|
*
|
|
|
|
* Must be held from before the current memslots are copied until
|
|
|
|
* after the new memslots are installed with rcu_assign_pointer,
|
|
|
|
* then released before the synchronize srcu in install_new_memslots.
|
|
|
|
*
|
|
|
|
* When modifying memslots outside of the slots_lock, must be held
|
|
|
|
* before reading the pointer to the current memslots until after all
|
|
|
|
* changes to those memslots are complete.
|
|
|
|
*
|
|
|
|
* These rules ensure that installing new memslots does not lose
|
|
|
|
* changes made to the previous memslots.
|
|
|
|
*/
|
|
|
|
mutex_lock(&kvm->slots_arch_lock);
|
|
|
|
|
2020-02-19 05:07:32 +08:00
|
|
|
slots = kvm_dup_memslots(__kvm_memslots(kvm, as_id), change);
|
2021-05-19 01:34:11 +08:00
|
|
|
if (!slots) {
|
|
|
|
mutex_unlock(&kvm->slots_arch_lock);
|
2020-02-19 05:07:23 +08:00
|
|
|
return -ENOMEM;
|
2021-05-19 01:34:11 +08:00
|
|
|
}
|
2020-02-19 05:07:23 +08:00
|
|
|
|
|
|
|
if (change == KVM_MR_DELETE || change == KVM_MR_MOVE) {
|
|
|
|
/*
|
|
|
|
* Note, the INVALID flag needs to be in the appropriate entry
|
|
|
|
* in the freshly allocated memslots, not in @old or @new.
|
|
|
|
*/
|
|
|
|
slot = id_to_memslot(slots, old->id);
|
|
|
|
slot->flags |= KVM_MEMSLOT_INVALID;
|
|
|
|
|
|
|
|
/*
|
2021-05-19 01:34:11 +08:00
|
|
|
* We can re-use the memory from the old memslots.
|
|
|
|
* It will be overwritten with a copy of the new memslots
|
|
|
|
* after reacquiring the slots_arch_lock below.
|
2020-02-19 05:07:23 +08:00
|
|
|
*/
|
|
|
|
slots = install_new_memslots(kvm, as_id, slots);
|
|
|
|
|
|
|
|
/* From this point no new shadow pages pointing to a deleted,
|
|
|
|
* or moved, memslot will be created.
|
|
|
|
*
|
|
|
|
* validation of sp->gfn happens in:
|
|
|
|
* - gfn_to_hva (kvm_read_guest, gfn_to_pfn)
|
|
|
|
* - kvm_is_visible_gfn (mmu_check_root)
|
|
|
|
*/
|
|
|
|
kvm_arch_flush_shadow_memslot(kvm, slot);
|
2021-05-19 01:34:11 +08:00
|
|
|
|
|
|
|
/* Released in install_new_memslots. */
|
|
|
|
mutex_lock(&kvm->slots_arch_lock);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* The arch-specific fields of the memslots could have changed
|
|
|
|
* between releasing the slots_arch_lock in
|
|
|
|
* install_new_memslots and here, so get a fresh copy of the
|
|
|
|
* slots.
|
|
|
|
*/
|
|
|
|
kvm_copy_memslots(slots, __kvm_memslots(kvm, as_id));
|
2020-02-19 05:07:23 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
r = kvm_arch_prepare_memory_region(kvm, new, mem, change);
|
|
|
|
if (r)
|
|
|
|
goto out_slots;
|
|
|
|
|
|
|
|
update_memslots(slots, new, change);
|
|
|
|
slots = install_new_memslots(kvm, as_id, slots);
|
|
|
|
|
|
|
|
kvm_arch_commit_memory_region(kvm, mem, old, new, change);
|
|
|
|
|
|
|
|
kvfree(slots);
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
out_slots:
|
2021-05-19 01:34:11 +08:00
|
|
|
if (change == KVM_MR_DELETE || change == KVM_MR_MOVE) {
|
|
|
|
slot = id_to_memslot(slots, old->id);
|
|
|
|
slot->flags &= ~KVM_MEMSLOT_INVALID;
|
2020-02-19 05:07:23 +08:00
|
|
|
slots = install_new_memslots(kvm, as_id, slots);
|
2021-05-19 01:34:11 +08:00
|
|
|
} else {
|
|
|
|
mutex_unlock(&kvm->slots_arch_lock);
|
|
|
|
}
|
2020-02-19 05:07:23 +08:00
|
|
|
kvfree(slots);
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:26 +08:00
|
|
|
static int kvm_delete_memslot(struct kvm *kvm,
|
|
|
|
const struct kvm_userspace_memory_region *mem,
|
|
|
|
struct kvm_memory_slot *old, int as_id)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot new;
|
|
|
|
int r;
|
|
|
|
|
|
|
|
if (!old->npages)
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
memset(&new, 0, sizeof(new));
|
|
|
|
new.id = old->id;
|
2020-10-15 02:26:46 +08:00
|
|
|
/*
|
|
|
|
* This is only for debugging purpose; it should never be referenced
|
|
|
|
* for a removed memslot.
|
|
|
|
*/
|
|
|
|
new.as_id = as_id;
|
2020-02-19 05:07:26 +08:00
|
|
|
|
|
|
|
r = kvm_set_memslot(kvm, mem, old, &new, as_id, KVM_MR_DELETE);
|
|
|
|
if (r)
|
|
|
|
return r;
|
|
|
|
|
2020-02-19 05:07:27 +08:00
|
|
|
kvm_free_memslot(kvm, old);
|
2020-02-19 05:07:26 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
/*
|
|
|
|
* Allocate some memory and give it an address in the guest physical address
|
|
|
|
* space.
|
|
|
|
*
|
|
|
|
* Discontiguous memory is allowed, mostly for framebuffers.
|
2007-10-29 09:40:42 +08:00
|
|
|
*
|
2014-10-27 23:22:56 +08:00
|
|
|
* Must be called holding kvm->slots_lock for write.
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
*/
|
2007-10-29 09:40:42 +08:00
|
|
|
int __kvm_set_memory_region(struct kvm *kvm,
|
2015-05-18 19:59:39 +08:00
|
|
|
const struct kvm_userspace_memory_region *mem)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
|
|
|
struct kvm_memory_slot old, new;
|
2020-02-19 05:07:28 +08:00
|
|
|
struct kvm_memory_slot *tmp;
|
2013-01-29 10:00:07 +08:00
|
|
|
enum kvm_mr_change change;
|
2020-02-19 05:07:28 +08:00
|
|
|
int as_id, id;
|
|
|
|
int r;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2012-08-21 10:58:13 +08:00
|
|
|
r = check_memory_region_flags(mem);
|
|
|
|
if (r)
|
2020-02-19 05:07:22 +08:00
|
|
|
return r;
|
2012-08-21 10:58:13 +08:00
|
|
|
|
2015-05-17 23:30:37 +08:00
|
|
|
as_id = mem->slot >> 16;
|
|
|
|
id = (u16)mem->slot;
|
|
|
|
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
/* General sanity checks */
|
|
|
|
if (mem->memory_size & (PAGE_SIZE - 1))
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
if (mem->guest_phys_addr & (PAGE_SIZE - 1))
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
2011-05-07 15:35:38 +08:00
|
|
|
/* We can read the guest memory with __xxx_user() later on. */
|
2020-06-01 16:17:45 +08:00
|
|
|
if ((mem->userspace_addr & (PAGE_SIZE - 1)) ||
|
2021-01-21 20:08:15 +08:00
|
|
|
(mem->userspace_addr != untagged_addr(mem->userspace_addr)) ||
|
Remove 'type' argument from access_ok() function
Nobody has actually used the type (VERIFY_READ vs VERIFY_WRITE) argument
of the user address range verification function since we got rid of the
old racy i386-only code to walk page tables by hand.
It existed because the original 80386 would not honor the write protect
bit when in kernel mode, so you had to do COW by hand before doing any
user access. But we haven't supported that in a long time, and these
days the 'type' argument is a purely historical artifact.
A discussion about extending 'user_access_begin()' to do the range
checking resulted this patch, because there is no way we're going to
move the old VERIFY_xyz interface to that model. And it's best done at
the end of the merge window when I've done most of my merges, so let's
just get this done once and for all.
This patch was mostly done with a sed-script, with manual fix-ups for
the cases that weren't of the trivial 'access_ok(VERIFY_xyz' form.
There were a couple of notable cases:
- csky still had the old "verify_area()" name as an alias.
- the iter_iov code had magical hardcoded knowledge of the actual
values of VERIFY_{READ,WRITE} (not that they mattered, since nothing
really used it)
- microblaze used the type argument for a debug printout
but other than those oddities this should be a total no-op patch.
I tried to fix up all architectures, did fairly extensive grepping for
access_ok() uses, and the changes are trivial, but I may have missed
something. Any missed conversion should be trivially fixable, though.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-01-04 10:57:57 +08:00
|
|
|
!access_ok((void __user *)(unsigned long)mem->userspace_addr,
|
2020-06-01 16:17:45 +08:00
|
|
|
mem->memory_size))
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
2015-05-17 23:30:37 +08:00
|
|
|
if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_MEM_SLOTS_NUM)
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2020-02-19 05:07:26 +08:00
|
|
|
/*
|
|
|
|
* Make a full copy of the old memslot, the pointer will become stale
|
|
|
|
* when the memslots are re-sorted by update_memslots(), and the old
|
|
|
|
* memslot needs to be referenced after calling update_memslots(), e.g.
|
2020-02-19 05:07:29 +08:00
|
|
|
* to free its resources and for arch specific behavior.
|
2020-02-19 05:07:26 +08:00
|
|
|
*/
|
2020-02-19 05:07:31 +08:00
|
|
|
tmp = id_to_memslot(__kvm_memslots(kvm, as_id), id);
|
|
|
|
if (tmp) {
|
|
|
|
old = *tmp;
|
|
|
|
tmp = NULL;
|
|
|
|
} else {
|
|
|
|
memset(&old, 0, sizeof(old));
|
|
|
|
old.id = id;
|
|
|
|
}
|
2020-02-19 05:07:28 +08:00
|
|
|
|
2020-02-19 05:07:26 +08:00
|
|
|
if (!mem->memory_size)
|
|
|
|
return kvm_delete_memslot(kvm, mem, &old, as_id);
|
|
|
|
|
2020-10-15 02:26:46 +08:00
|
|
|
new.as_id = as_id;
|
2015-05-17 23:30:37 +08:00
|
|
|
new.id = id;
|
2020-02-19 05:07:28 +08:00
|
|
|
new.base_gfn = mem->guest_phys_addr >> PAGE_SHIFT;
|
|
|
|
new.npages = mem->memory_size >> PAGE_SHIFT;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
new.flags = mem->flags;
|
2020-02-19 05:07:20 +08:00
|
|
|
new.userspace_addr = mem->userspace_addr;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2020-02-19 05:07:28 +08:00
|
|
|
if (new.npages > KVM_MEM_MAX_NR_PAGES)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2020-02-19 05:07:26 +08:00
|
|
|
if (!old.npages) {
|
|
|
|
change = KVM_MR_CREATE;
|
2020-02-19 05:07:28 +08:00
|
|
|
new.dirty_bitmap = NULL;
|
|
|
|
memset(&new.arch, 0, sizeof(new.arch));
|
2020-02-19 05:07:26 +08:00
|
|
|
} else { /* Modify an existing slot. */
|
|
|
|
if ((new.userspace_addr != old.userspace_addr) ||
|
2020-02-19 05:07:28 +08:00
|
|
|
(new.npages != old.npages) ||
|
2020-02-19 05:07:26 +08:00
|
|
|
((new.flags ^ old.flags) & KVM_MEM_READONLY))
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EINVAL;
|
2015-05-18 19:59:39 +08:00
|
|
|
|
2020-02-19 05:07:28 +08:00
|
|
|
if (new.base_gfn != old.base_gfn)
|
2020-02-19 05:07:26 +08:00
|
|
|
change = KVM_MR_MOVE;
|
|
|
|
else if (new.flags != old.flags)
|
|
|
|
change = KVM_MR_FLAGS_ONLY;
|
|
|
|
else /* Nothing to change. */
|
|
|
|
return 0;
|
2020-02-19 05:07:28 +08:00
|
|
|
|
|
|
|
/* Copy dirty_bitmap and arch from the current memslot. */
|
|
|
|
new.dirty_bitmap = old.dirty_bitmap;
|
|
|
|
memcpy(&new.arch, &old.arch, sizeof(new.arch));
|
2015-05-18 19:59:39 +08:00
|
|
|
}
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2013-01-29 10:00:07 +08:00
|
|
|
if ((change == KVM_MR_CREATE) || (change == KVM_MR_MOVE)) {
|
2013-01-11 17:26:55 +08:00
|
|
|
/* Check for overlaps */
|
2020-02-19 05:07:28 +08:00
|
|
|
kvm_for_each_memslot(tmp, __kvm_memslots(kvm, as_id)) {
|
|
|
|
if (tmp->id == id)
|
2013-01-11 17:26:55 +08:00
|
|
|
continue;
|
2020-02-19 05:07:28 +08:00
|
|
|
if (!((new.base_gfn + new.npages <= tmp->base_gfn) ||
|
|
|
|
(new.base_gfn >= tmp->base_gfn + tmp->npages)))
|
2020-02-19 05:07:22 +08:00
|
|
|
return -EEXIST;
|
2013-01-11 17:26:55 +08:00
|
|
|
}
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:20 +08:00
|
|
|
/* Allocate/free page dirty bitmap as needed */
|
|
|
|
if (!(new.flags & KVM_MEM_LOG_DIRTY_PAGES))
|
|
|
|
new.dirty_bitmap = NULL;
|
2020-10-01 09:22:26 +08:00
|
|
|
else if (!new.dirty_bitmap && !kvm->dirty_ring_size) {
|
2020-02-27 09:32:27 +08:00
|
|
|
r = kvm_alloc_dirty_bitmap(&new);
|
2020-02-19 05:07:22 +08:00
|
|
|
if (r)
|
|
|
|
return r;
|
2020-02-27 09:32:27 +08:00
|
|
|
|
|
|
|
if (kvm_dirty_log_manual_protect_and_init_set(kvm))
|
|
|
|
bitmap_set(new.dirty_bitmap, 0, new.npages);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:23 +08:00
|
|
|
r = kvm_set_memslot(kvm, mem, &old, &new, as_id, change);
|
|
|
|
if (r)
|
|
|
|
goto out_bitmap;
|
2007-10-03 00:52:55 +08:00
|
|
|
|
2020-02-19 05:07:26 +08:00
|
|
|
if (old.dirty_bitmap && !new.dirty_bitmap)
|
|
|
|
kvm_destroy_dirty_bitmap(&old);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
return 0;
|
|
|
|
|
2020-02-19 05:07:21 +08:00
|
|
|
out_bitmap:
|
|
|
|
if (new.dirty_bitmap && !old.dirty_bitmap)
|
|
|
|
kvm_destroy_dirty_bitmap(&new);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
return r;
|
2007-10-25 05:52:57 +08:00
|
|
|
}
|
2007-10-29 09:40:42 +08:00
|
|
|
EXPORT_SYMBOL_GPL(__kvm_set_memory_region);
|
|
|
|
|
|
|
|
int kvm_set_memory_region(struct kvm *kvm,
|
2015-05-18 19:59:39 +08:00
|
|
|
const struct kvm_userspace_memory_region *mem)
|
2007-10-29 09:40:42 +08:00
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
2009-12-24 00:35:26 +08:00
|
|
|
mutex_lock(&kvm->slots_lock);
|
2013-02-27 18:43:00 +08:00
|
|
|
r = __kvm_set_memory_region(kvm, mem);
|
2009-12-24 00:35:26 +08:00
|
|
|
mutex_unlock(&kvm->slots_lock);
|
2007-10-29 09:40:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
2007-10-25 05:52:57 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_set_memory_region);
|
|
|
|
|
2013-12-30 04:12:29 +08:00
|
|
|
static int kvm_vm_ioctl_set_memory_region(struct kvm *kvm,
|
|
|
|
struct kvm_userspace_memory_region *mem)
|
2007-10-25 05:52:57 +08:00
|
|
|
{
|
2015-05-17 23:30:37 +08:00
|
|
|
if ((u16)mem->slot >= KVM_USER_MEM_SLOTS)
|
2007-10-25 05:57:46 +08:00
|
|
|
return -EINVAL;
|
2015-05-18 19:59:39 +08:00
|
|
|
|
2013-02-27 18:43:00 +08:00
|
|
|
return kvm_set_memory_region(kvm, mem);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
#ifndef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
/**
|
|
|
|
* kvm_get_dirty_log - get a snapshot of dirty pages
|
|
|
|
* @kvm: pointer to kvm instance
|
|
|
|
* @log: slot id and address to which we copy the log
|
|
|
|
* @is_dirty: set to '1' if any dirty pages were found
|
|
|
|
* @memslot: set to the associated memslot, always valid on success
|
|
|
|
*/
|
|
|
|
int kvm_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log,
|
|
|
|
int *is_dirty, struct kvm_memory_slot **memslot)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2015-05-17 22:20:07 +08:00
|
|
|
struct kvm_memslots *slots;
|
2017-01-23 00:41:07 +08:00
|
|
|
int i, as_id, id;
|
2010-04-12 18:35:35 +08:00
|
|
|
unsigned long n;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
unsigned long any = 0;
|
|
|
|
|
2020-10-01 09:22:24 +08:00
|
|
|
/* Dirty ring tracking is exclusive to dirty log tracking */
|
|
|
|
if (kvm->dirty_ring_size)
|
|
|
|
return -ENXIO;
|
|
|
|
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
*memslot = NULL;
|
|
|
|
*is_dirty = 0;
|
|
|
|
|
2015-05-17 23:30:37 +08:00
|
|
|
as_id = log->slot >> 16;
|
|
|
|
id = (u16)log->slot;
|
|
|
|
if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS)
|
2017-01-23 00:41:07 +08:00
|
|
|
return -EINVAL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2015-05-17 23:30:37 +08:00
|
|
|
slots = __kvm_memslots(kvm, as_id);
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
*memslot = id_to_memslot(slots, id);
|
2020-02-19 05:07:31 +08:00
|
|
|
if (!(*memslot) || !(*memslot)->dirty_bitmap)
|
2017-01-23 00:41:07 +08:00
|
|
|
return -ENOENT;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
kvm_arch_sync_dirty_log(kvm, *memslot);
|
|
|
|
|
|
|
|
n = kvm_dirty_bitmap_bytes(*memslot);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-02-22 22:43:09 +08:00
|
|
|
for (i = 0; !any && i < n/sizeof(long); ++i)
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
any = (*memslot)->dirty_bitmap[i];
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
Rework kvm_get_dirty_log() so that it "returns" the associated memslot
on success. A future patch will rework memslot handling such that
id_to_memslot() can return NULL, returning the memslot makes it more
obvious that the validity of the memslot has been verified, i.e.
precludes the need to add validity checks in the arch code that are
technically unnecessary.
To maintain ordering in s390, move the call to kvm_arch_sync_dirty_log()
from s390's kvm_vm_ioctl_get_dirty_log() to the new kvm_get_dirty_log().
This is a nop for PPC, the only other arch that doesn't select
KVM_GENERIC_DIRTYLOG_READ_PROTECT, as its sync_dirty_log() is empty.
Ideally, moving the sync_dirty_log() call would be done in a separate
patch, but it can't be done in a follow-on patch because that would
temporarily break s390's ordering. Making the move in a preparatory
patch would be functionally correct, but would create an odd scenario
where the moved sync_dirty_log() would operate on a "different" memslot
due to consuming the result of a different id_to_memslot(). The
memslot couldn't actually be different as slots_lock is held, but the
code is confusing enough as it is, i.e. moving sync_dirty_log() in this
patch is the lesser of all evils.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-19 05:07:30 +08:00
|
|
|
if (copy_to_user(log->dirty_bitmap, (*memslot)->dirty_bitmap, n))
|
2017-01-23 00:41:07 +08:00
|
|
|
return -EFAULT;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-11-18 20:29:43 +08:00
|
|
|
if (any)
|
|
|
|
*is_dirty = 1;
|
2017-01-23 00:41:07 +08:00
|
|
|
return 0;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
2013-10-08 00:47:59 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_get_dirty_log);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
#else /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */
|
2015-01-16 07:58:53 +08:00
|
|
|
/**
|
2019-04-23 19:40:30 +08:00
|
|
|
* kvm_get_dirty_log_protect - get a snapshot of dirty pages
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
* and reenable dirty page tracking for the corresponding pages.
|
2015-01-16 07:58:53 +08:00
|
|
|
* @kvm: pointer to kvm instance
|
|
|
|
* @log: slot id and address to which we copy the log
|
|
|
|
*
|
|
|
|
* We need to keep it in mind that VCPU threads can write to the bitmap
|
|
|
|
* concurrently. So, to avoid losing track of dirty pages we keep the
|
|
|
|
* following order:
|
|
|
|
*
|
|
|
|
* 1. Take a snapshot of the bit and clear it if needed.
|
|
|
|
* 2. Write protect the corresponding page.
|
|
|
|
* 3. Copy the snapshot to the userspace.
|
|
|
|
* 4. Upon return caller flushes TLB's if needed.
|
|
|
|
*
|
|
|
|
* Between 2 and 4, the guest may write to the page using the remaining TLB
|
|
|
|
* entry. This is not a problem because the page is reported dirty using
|
|
|
|
* the snapshot taken before and step 4 ensures that writes done after
|
|
|
|
* exiting to userspace will be logged for the next call.
|
|
|
|
*
|
|
|
|
*/
|
2020-02-19 05:07:29 +08:00
|
|
|
static int kvm_get_dirty_log_protect(struct kvm *kvm, struct kvm_dirty_log *log)
|
2015-01-16 07:58:53 +08:00
|
|
|
{
|
2015-05-17 22:20:07 +08:00
|
|
|
struct kvm_memslots *slots;
|
2015-01-16 07:58:53 +08:00
|
|
|
struct kvm_memory_slot *memslot;
|
2017-01-23 00:30:16 +08:00
|
|
|
int i, as_id, id;
|
2015-01-16 07:58:53 +08:00
|
|
|
unsigned long n;
|
|
|
|
unsigned long *dirty_bitmap;
|
|
|
|
unsigned long *dirty_bitmap_buffer;
|
2020-02-19 05:07:29 +08:00
|
|
|
bool flush;
|
2015-01-16 07:58:53 +08:00
|
|
|
|
2020-10-01 09:22:24 +08:00
|
|
|
/* Dirty ring tracking is exclusive to dirty log tracking */
|
|
|
|
if (kvm->dirty_ring_size)
|
|
|
|
return -ENXIO;
|
|
|
|
|
2015-05-17 23:30:37 +08:00
|
|
|
as_id = log->slot >> 16;
|
|
|
|
id = (u16)log->slot;
|
|
|
|
if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS)
|
2017-01-23 00:30:16 +08:00
|
|
|
return -EINVAL;
|
2015-01-16 07:58:53 +08:00
|
|
|
|
2015-05-17 23:30:37 +08:00
|
|
|
slots = __kvm_memslots(kvm, as_id);
|
|
|
|
memslot = id_to_memslot(slots, id);
|
2020-02-19 05:07:31 +08:00
|
|
|
if (!memslot || !memslot->dirty_bitmap)
|
|
|
|
return -ENOENT;
|
2015-01-16 07:58:53 +08:00
|
|
|
|
|
|
|
dirty_bitmap = memslot->dirty_bitmap;
|
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
kvm_arch_sync_dirty_log(kvm, memslot);
|
|
|
|
|
2015-01-16 07:58:53 +08:00
|
|
|
n = kvm_dirty_bitmap_bytes(memslot);
|
2020-02-19 05:07:29 +08:00
|
|
|
flush = false;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
if (kvm->manual_dirty_log_protect) {
|
|
|
|
/*
|
|
|
|
* Unlike kvm_get_dirty_log, we always return false in *flush,
|
|
|
|
* because no flush is needed until KVM_CLEAR_DIRTY_LOG. There
|
|
|
|
* is some code duplication between this function and
|
|
|
|
* kvm_get_dirty_log, but hopefully all architecture
|
|
|
|
* transition to kvm_get_dirty_log_protect and kvm_get_dirty_log
|
|
|
|
* can be eliminated.
|
|
|
|
*/
|
|
|
|
dirty_bitmap_buffer = dirty_bitmap;
|
|
|
|
} else {
|
|
|
|
dirty_bitmap_buffer = kvm_second_dirty_bitmap(memslot);
|
|
|
|
memset(dirty_bitmap_buffer, 0, n);
|
2015-01-16 07:58:53 +08:00
|
|
|
|
2021-02-03 02:57:24 +08:00
|
|
|
KVM_MMU_LOCK(kvm);
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
for (i = 0; i < n / sizeof(long); i++) {
|
|
|
|
unsigned long mask;
|
|
|
|
gfn_t offset;
|
2015-01-16 07:58:53 +08:00
|
|
|
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
if (!dirty_bitmap[i])
|
|
|
|
continue;
|
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
flush = true;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
mask = xchg(&dirty_bitmap[i], 0);
|
|
|
|
dirty_bitmap_buffer[i] = mask;
|
|
|
|
|
2019-02-02 17:20:27 +08:00
|
|
|
offset = i * BITS_PER_LONG;
|
|
|
|
kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot,
|
|
|
|
offset, mask);
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
}
|
2021-02-03 02:57:24 +08:00
|
|
|
KVM_MMU_UNLOCK(kvm);
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
}
|
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
if (flush)
|
|
|
|
kvm_arch_flush_remote_tlbs_memslot(kvm, memslot);
|
|
|
|
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
if (copy_to_user(log->dirty_bitmap, dirty_bitmap_buffer, n))
|
|
|
|
return -EFAULT;
|
|
|
|
return 0;
|
|
|
|
}
|
2020-02-19 05:07:29 +08:00
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
* kvm_vm_ioctl_get_dirty_log - get and clear the log of dirty pages in a slot
|
|
|
|
* @kvm: kvm instance
|
|
|
|
* @log: slot id and address to which we copy the log
|
|
|
|
*
|
|
|
|
* Steps 1-4 below provide general overview of dirty page logging. See
|
|
|
|
* kvm_get_dirty_log_protect() function description for additional details.
|
|
|
|
*
|
|
|
|
* We call kvm_get_dirty_log_protect() to handle steps 1-3, upon return we
|
|
|
|
* always flush the TLB (step 4) even if previous step failed and the dirty
|
|
|
|
* bitmap may be corrupt. Regardless of previous outcome the KVM logging API
|
|
|
|
* does not preclude user space subsequent dirty log read. Flushing TLB ensures
|
|
|
|
* writes will be marked dirty for next log read.
|
|
|
|
*
|
|
|
|
* 1. Take a snapshot of the bit and clear it if needed.
|
|
|
|
* 2. Write protect the corresponding page.
|
|
|
|
* 3. Copy the snapshot to the userspace.
|
|
|
|
* 4. Flush TLB's if needed.
|
|
|
|
*/
|
|
|
|
static int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,
|
|
|
|
struct kvm_dirty_log *log)
|
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
|
|
|
mutex_lock(&kvm->slots_lock);
|
|
|
|
|
|
|
|
r = kvm_get_dirty_log_protect(kvm, log);
|
|
|
|
|
|
|
|
mutex_unlock(&kvm->slots_lock);
|
|
|
|
return r;
|
|
|
|
}
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* kvm_clear_dirty_log_protect - clear dirty bits in the bitmap
|
|
|
|
* and reenable dirty page tracking for the corresponding pages.
|
|
|
|
* @kvm: pointer to kvm instance
|
|
|
|
* @log: slot id and address from which to fetch the bitmap of dirty pages
|
|
|
|
*/
|
2020-02-19 05:07:29 +08:00
|
|
|
static int kvm_clear_dirty_log_protect(struct kvm *kvm,
|
|
|
|
struct kvm_clear_dirty_log *log)
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
{
|
|
|
|
struct kvm_memslots *slots;
|
|
|
|
struct kvm_memory_slot *memslot;
|
2019-01-03 01:29:37 +08:00
|
|
|
int as_id, id;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
gfn_t offset;
|
2019-01-03 01:29:37 +08:00
|
|
|
unsigned long i, n;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
unsigned long *dirty_bitmap;
|
|
|
|
unsigned long *dirty_bitmap_buffer;
|
2020-02-19 05:07:29 +08:00
|
|
|
bool flush;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
|
2020-10-01 09:22:24 +08:00
|
|
|
/* Dirty ring tracking is exclusive to dirty log tracking */
|
|
|
|
if (kvm->dirty_ring_size)
|
|
|
|
return -ENXIO;
|
|
|
|
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
as_id = log->slot >> 16;
|
|
|
|
id = (u16)log->slot;
|
|
|
|
if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2019-04-17 21:28:44 +08:00
|
|
|
if (log->first_page & 63)
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
slots = __kvm_memslots(kvm, as_id);
|
|
|
|
memslot = id_to_memslot(slots, id);
|
2020-02-19 05:07:31 +08:00
|
|
|
if (!memslot || !memslot->dirty_bitmap)
|
|
|
|
return -ENOENT;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
|
|
|
|
dirty_bitmap = memslot->dirty_bitmap;
|
|
|
|
|
2019-05-08 17:15:45 +08:00
|
|
|
n = ALIGN(log->num_pages, BITS_PER_LONG) / 8;
|
2019-01-03 01:29:37 +08:00
|
|
|
|
|
|
|
if (log->first_page > memslot->npages ||
|
2019-04-17 21:28:44 +08:00
|
|
|
log->num_pages > memslot->npages - log->first_page ||
|
|
|
|
(log->num_pages < memslot->npages - log->first_page && (log->num_pages & 63)))
|
|
|
|
return -EINVAL;
|
2019-01-03 01:29:37 +08:00
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
kvm_arch_sync_dirty_log(kvm, memslot);
|
|
|
|
|
|
|
|
flush = false;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
dirty_bitmap_buffer = kvm_second_dirty_bitmap(memslot);
|
|
|
|
if (copy_from_user(dirty_bitmap_buffer, log->dirty_bitmap, n))
|
|
|
|
return -EFAULT;
|
2015-01-16 07:58:53 +08:00
|
|
|
|
2021-02-03 02:57:24 +08:00
|
|
|
KVM_MMU_LOCK(kvm);
|
2019-05-08 17:15:46 +08:00
|
|
|
for (offset = log->first_page, i = offset / BITS_PER_LONG,
|
|
|
|
n = DIV_ROUND_UP(log->num_pages, BITS_PER_LONG); n--;
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
i++, offset += BITS_PER_LONG) {
|
|
|
|
unsigned long mask = *dirty_bitmap_buffer++;
|
|
|
|
atomic_long_t *p = (atomic_long_t *) &dirty_bitmap[i];
|
|
|
|
if (!mask)
|
2015-01-16 07:58:53 +08:00
|
|
|
continue;
|
|
|
|
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
mask &= atomic_long_fetch_andnot(mask, p);
|
2015-01-16 07:58:53 +08:00
|
|
|
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
/*
|
|
|
|
* mask contains the bits that really have been cleared. This
|
|
|
|
* never includes any bits beyond the length of the memslot (if
|
|
|
|
* the length is not aligned to 64 pages), therefore it is not
|
|
|
|
* a problem if userspace sets them in log->dirty_bitmap.
|
|
|
|
*/
|
2015-03-17 15:19:58 +08:00
|
|
|
if (mask) {
|
2020-02-19 05:07:29 +08:00
|
|
|
flush = true;
|
2015-03-17 15:19:58 +08:00
|
|
|
kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot,
|
|
|
|
offset, mask);
|
|
|
|
}
|
2015-01-16 07:58:53 +08:00
|
|
|
}
|
2021-02-03 02:57:24 +08:00
|
|
|
KVM_MMU_UNLOCK(kvm);
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
|
2020-02-19 05:07:29 +08:00
|
|
|
if (flush)
|
|
|
|
kvm_arch_flush_remote_tlbs_memslot(kvm, memslot);
|
|
|
|
|
2017-01-23 00:30:16 +08:00
|
|
|
return 0;
|
2015-01-16 07:58:53 +08:00
|
|
|
}
|
2020-02-19 05:07:29 +08:00
|
|
|
|
|
|
|
static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm,
|
|
|
|
struct kvm_clear_dirty_log *log)
|
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
|
|
|
mutex_lock(&kvm->slots_lock);
|
|
|
|
|
|
|
|
r = kvm_clear_dirty_log_protect(kvm, log);
|
|
|
|
|
|
|
|
mutex_unlock(&kvm->slots_lock);
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
#endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */
|
2015-01-16 07:58:53 +08:00
|
|
|
|
2010-10-18 21:22:23 +08:00
|
|
|
struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn)
|
|
|
|
{
|
|
|
|
return __gfn_to_memslot(kvm_memslots(kvm), gfn);
|
|
|
|
}
|
2010-06-21 16:44:20 +08:00
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_memslot);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn)
|
|
|
|
{
|
2021-08-05 06:28:40 +08:00
|
|
|
struct kvm_memslots *slots = kvm_vcpu_memslots(vcpu);
|
|
|
|
struct kvm_memory_slot *slot;
|
|
|
|
int slot_index;
|
|
|
|
|
|
|
|
slot = try_get_memslot(slots, vcpu->last_used_slot, gfn);
|
|
|
|
if (slot)
|
|
|
|
return slot;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Fall back to searching all memslots. We purposely use
|
|
|
|
* search_memslots() instead of __gfn_to_memslot() to avoid
|
|
|
|
* thrashing the VM-wide last_used_index in kvm_memslots.
|
|
|
|
*/
|
|
|
|
slot = search_memslots(slots, gfn, &slot_index);
|
|
|
|
if (slot) {
|
|
|
|
vcpu->last_used_slot = slot_index;
|
|
|
|
return slot;
|
|
|
|
}
|
|
|
|
|
|
|
|
return NULL;
|
2015-05-17 19:58:53 +08:00
|
|
|
}
|
2020-04-18 00:21:06 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_memslot);
|
2015-05-17 19:58:53 +08:00
|
|
|
|
2015-11-14 11:21:06 +08:00
|
|
|
bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn)
|
2007-10-25 05:57:46 +08:00
|
|
|
{
|
2011-11-24 17:40:57 +08:00
|
|
|
struct kvm_memory_slot *memslot = gfn_to_memslot(kvm, gfn);
|
2007-10-25 05:57:46 +08:00
|
|
|
|
2020-04-16 21:48:07 +08:00
|
|
|
return kvm_is_visible_memslot(memslot);
|
2007-10-25 05:57:46 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_is_visible_gfn);
|
|
|
|
|
2020-07-08 22:00:23 +08:00
|
|
|
bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
|
|
|
|
|
|
|
return kvm_is_visible_memslot(memslot);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_is_visible_gfn);
|
|
|
|
|
2020-01-09 04:24:37 +08:00
|
|
|
unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn)
|
2010-01-28 19:37:56 +08:00
|
|
|
{
|
|
|
|
struct vm_area_struct *vma;
|
|
|
|
unsigned long addr, size;
|
|
|
|
|
|
|
|
size = PAGE_SIZE;
|
|
|
|
|
2020-01-09 04:24:38 +08:00
|
|
|
addr = kvm_vcpu_gfn_to_hva_prot(vcpu, gfn, NULL);
|
2010-01-28 19:37:56 +08:00
|
|
|
if (kvm_is_error_hva(addr))
|
|
|
|
return PAGE_SIZE;
|
|
|
|
|
2020-06-09 12:33:25 +08:00
|
|
|
mmap_read_lock(current->mm);
|
2010-01-28 19:37:56 +08:00
|
|
|
vma = find_vma(current->mm, addr);
|
|
|
|
if (!vma)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
size = vma_kernel_pagesize(vma);
|
|
|
|
|
|
|
|
out:
|
2020-06-09 12:33:25 +08:00
|
|
|
mmap_read_unlock(current->mm);
|
2010-01-28 19:37:56 +08:00
|
|
|
|
|
|
|
return size;
|
|
|
|
}
|
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
static bool memslot_is_readonly(struct kvm_memory_slot *slot)
|
|
|
|
{
|
|
|
|
return slot->flags & KVM_MEM_READONLY;
|
|
|
|
}
|
|
|
|
|
|
|
|
static unsigned long __gfn_to_hva_many(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
gfn_t *nr_pages, bool write)
|
2007-11-12 04:05:04 +08:00
|
|
|
{
|
2009-12-24 00:35:21 +08:00
|
|
|
if (!slot || slot->flags & KVM_MEMSLOT_INVALID)
|
2012-08-21 11:01:50 +08:00
|
|
|
return KVM_HVA_ERR_BAD;
|
2010-08-22 19:11:43 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
if (memslot_is_readonly(slot) && write)
|
|
|
|
return KVM_HVA_ERR_RO_BAD;
|
2010-08-22 19:11:43 +08:00
|
|
|
|
|
|
|
if (nr_pages)
|
|
|
|
*nr_pages = slot->npages - (gfn - slot->base_gfn);
|
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
return __gfn_to_hva_memslot(slot, gfn);
|
2007-11-12 04:05:04 +08:00
|
|
|
}
|
2010-08-22 19:11:43 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
static unsigned long gfn_to_hva_many(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
gfn_t *nr_pages)
|
|
|
|
{
|
|
|
|
return __gfn_to_hva_many(slot, gfn, nr_pages, true);
|
2007-11-12 04:05:04 +08:00
|
|
|
}
|
2010-08-22 19:11:43 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot,
|
2013-12-30 04:12:29 +08:00
|
|
|
gfn_t gfn)
|
2012-08-21 11:02:51 +08:00
|
|
|
{
|
|
|
|
return gfn_to_hva_many(slot, gfn, NULL);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_hva_memslot);
|
|
|
|
|
2010-08-22 19:11:43 +08:00
|
|
|
unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn)
|
|
|
|
{
|
2010-10-18 21:22:23 +08:00
|
|
|
return gfn_to_hva_many(gfn_to_memslot(kvm, gfn), gfn, NULL);
|
2010-08-22 19:11:43 +08:00
|
|
|
}
|
2008-04-25 21:44:50 +08:00
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_hva);
|
2007-11-12 04:05:04 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn)
|
|
|
|
{
|
|
|
|
return gfn_to_hva_many(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, NULL);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_hva);
|
|
|
|
|
2012-08-21 10:59:53 +08:00
|
|
|
/*
|
2018-10-09 10:41:15 +08:00
|
|
|
* Return the hva of a @gfn and the R/W attribute if possible.
|
|
|
|
*
|
|
|
|
* @slot: the kvm_memory_slot which contains @gfn
|
|
|
|
* @gfn: the gfn to be translated
|
|
|
|
* @writable: used to return the read/write attribute of the @slot if the hva
|
|
|
|
* is valid and @writable is not NULL
|
2012-08-21 10:59:53 +08:00
|
|
|
*/
|
2014-08-19 18:15:00 +08:00
|
|
|
unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot,
|
|
|
|
gfn_t gfn, bool *writable)
|
2012-08-21 10:59:53 +08:00
|
|
|
{
|
2013-10-02 00:58:36 +08:00
|
|
|
unsigned long hva = __gfn_to_hva_many(slot, gfn, NULL, false);
|
|
|
|
|
|
|
|
if (!kvm_is_error_hva(hva) && writable)
|
2013-09-09 19:52:33 +08:00
|
|
|
*writable = !memslot_is_readonly(slot);
|
|
|
|
|
2013-10-02 00:58:36 +08:00
|
|
|
return hva;
|
2012-08-21 10:59:53 +08:00
|
|
|
}
|
|
|
|
|
2014-08-19 18:15:00 +08:00
|
|
|
unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
|
|
|
|
|
|
|
|
return gfn_to_hva_memslot_prot(slot, gfn, writable);
|
|
|
|
}
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *writable)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
|
|
|
|
|
|
|
return gfn_to_hva_memslot_prot(slot, gfn, writable);
|
|
|
|
}
|
|
|
|
|
2011-01-30 11:15:49 +08:00
|
|
|
static inline int check_user_page_hwpoison(unsigned long addr)
|
|
|
|
{
|
mm: unexport __get_user_pages()
This patch unexports the low-level __get_user_pages() function.
Recent refactoring of the get_user_pages* functions allow flags to be
passed through get_user_pages() which eliminates the need for access to
this function from its one user, kvm.
We can see that the two calls to get_user_pages() which replace
__get_user_pages() in kvm_main.c are equivalent by examining their call
stacks:
get_user_page_nowait():
get_user_pages(start, 1, flags, page, NULL)
__get_user_pages_locked(current, current->mm, start, 1, page, NULL, NULL,
false, flags | FOLL_TOUCH)
__get_user_pages(current, current->mm, start, 1,
flags | FOLL_TOUCH | FOLL_GET, page, NULL, NULL)
check_user_page_hwpoison():
get_user_pages(addr, 1, flags, NULL, NULL)
__get_user_pages_locked(current, current->mm, addr, 1, NULL, NULL, NULL,
false, flags | FOLL_TOUCH)
__get_user_pages(current, current->mm, addr, 1, flags | FOLL_TOUCH, NULL,
NULL, NULL)
Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-24 17:57:25 +08:00
|
|
|
int rc, flags = FOLL_HWPOISON | FOLL_WRITE;
|
2011-01-30 11:15:49 +08:00
|
|
|
|
mm: unexport __get_user_pages()
This patch unexports the low-level __get_user_pages() function.
Recent refactoring of the get_user_pages* functions allow flags to be
passed through get_user_pages() which eliminates the need for access to
this function from its one user, kvm.
We can see that the two calls to get_user_pages() which replace
__get_user_pages() in kvm_main.c are equivalent by examining their call
stacks:
get_user_page_nowait():
get_user_pages(start, 1, flags, page, NULL)
__get_user_pages_locked(current, current->mm, start, 1, page, NULL, NULL,
false, flags | FOLL_TOUCH)
__get_user_pages(current, current->mm, start, 1,
flags | FOLL_TOUCH | FOLL_GET, page, NULL, NULL)
check_user_page_hwpoison():
get_user_pages(addr, 1, flags, NULL, NULL)
__get_user_pages_locked(current, current->mm, addr, 1, NULL, NULL, NULL,
false, flags | FOLL_TOUCH)
__get_user_pages(current, current->mm, addr, 1, flags | FOLL_TOUCH, NULL,
NULL, NULL)
Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-24 17:57:25 +08:00
|
|
|
rc = get_user_pages(addr, 1, flags, NULL, NULL);
|
2011-01-30 11:15:49 +08:00
|
|
|
return rc == -EHWPOISON;
|
|
|
|
}
|
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
/*
|
2018-07-27 23:44:41 +08:00
|
|
|
* The fast path to get the writable pfn which will be stored in @pfn,
|
|
|
|
* true indicates success, otherwise false is returned. It's also the
|
2019-12-11 14:26:25 +08:00
|
|
|
* only part that runs if we can in atomic context.
|
2012-08-21 11:00:22 +08:00
|
|
|
*/
|
2018-07-27 23:44:41 +08:00
|
|
|
static bool hva_to_pfn_fast(unsigned long addr, bool write_fault,
|
|
|
|
bool *writable, kvm_pfn_t *pfn)
|
2007-03-30 19:02:32 +08:00
|
|
|
{
|
2007-10-18 22:59:34 +08:00
|
|
|
struct page *page[1];
|
2007-03-30 19:02:32 +08:00
|
|
|
|
2012-08-21 11:00:49 +08:00
|
|
|
/*
|
|
|
|
* Fast pin a writable pfn only if it is a write fault request
|
|
|
|
* or the caller allows to map a writable pfn for a read fault
|
|
|
|
* request.
|
|
|
|
*/
|
|
|
|
if (!(write_fault || writable))
|
|
|
|
return false;
|
2010-10-23 00:18:18 +08:00
|
|
|
|
2020-06-08 12:40:55 +08:00
|
|
|
if (get_user_page_fast_only(addr, FOLL_WRITE, page)) {
|
2012-08-21 11:00:22 +08:00
|
|
|
*pfn = page_to_pfn(page[0]);
|
2010-10-23 00:18:18 +08:00
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
if (writable)
|
|
|
|
*writable = true;
|
|
|
|
return true;
|
|
|
|
}
|
2010-10-14 17:22:46 +08:00
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
return false;
|
|
|
|
}
|
2010-10-23 00:18:18 +08:00
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
/*
|
|
|
|
* The slow path to get the pfn of the specified host virtual address,
|
|
|
|
* 1 indicates success, -errno is returned if error is detected.
|
|
|
|
*/
|
|
|
|
static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault,
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
bool *writable, kvm_pfn_t *pfn)
|
2012-08-21 11:00:22 +08:00
|
|
|
{
|
2017-11-20 06:47:33 +08:00
|
|
|
unsigned int flags = FOLL_HWPOISON;
|
|
|
|
struct page *page;
|
2012-08-21 11:00:22 +08:00
|
|
|
int npages = 0;
|
2010-10-23 00:18:18 +08:00
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
might_sleep();
|
|
|
|
|
|
|
|
if (writable)
|
|
|
|
*writable = write_fault;
|
|
|
|
|
2017-11-20 06:47:33 +08:00
|
|
|
if (write_fault)
|
|
|
|
flags |= FOLL_WRITE;
|
|
|
|
if (async)
|
|
|
|
flags |= FOLL_NOWAIT;
|
2016-10-13 08:20:12 +08:00
|
|
|
|
2017-11-20 06:47:33 +08:00
|
|
|
npages = get_user_pages_unlocked(addr, 1, &page, flags);
|
2012-08-21 11:00:22 +08:00
|
|
|
if (npages != 1)
|
|
|
|
return npages;
|
|
|
|
|
|
|
|
/* map read fault as writable if possible */
|
2012-08-21 11:00:49 +08:00
|
|
|
if (unlikely(!write_fault) && writable) {
|
2017-11-20 06:47:33 +08:00
|
|
|
struct page *wpage;
|
2012-08-21 11:00:22 +08:00
|
|
|
|
2020-06-08 12:40:55 +08:00
|
|
|
if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) {
|
2012-08-21 11:00:22 +08:00
|
|
|
*writable = true;
|
2017-11-20 06:47:33 +08:00
|
|
|
put_page(page);
|
|
|
|
page = wpage;
|
2010-10-23 00:18:18 +08:00
|
|
|
}
|
2010-08-22 19:10:28 +08:00
|
|
|
}
|
2017-11-20 06:47:33 +08:00
|
|
|
*pfn = page_to_pfn(page);
|
2012-08-21 11:00:22 +08:00
|
|
|
return npages;
|
|
|
|
}
|
2007-11-12 04:05:04 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault)
|
|
|
|
{
|
|
|
|
if (unlikely(!(vma->vm_flags & VM_READ)))
|
|
|
|
return false;
|
2008-05-01 04:37:07 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
if (write_fault && (unlikely(!(vma->vm_flags & VM_WRITE))))
|
|
|
|
return false;
|
2010-08-22 19:10:28 +08:00
|
|
|
|
2012-08-21 11:02:51 +08:00
|
|
|
return true;
|
|
|
|
}
|
2010-05-31 14:28:19 +08:00
|
|
|
|
2021-06-24 20:29:04 +08:00
|
|
|
static int kvm_try_get_pfn(kvm_pfn_t pfn)
|
|
|
|
{
|
|
|
|
if (kvm_is_reserved_pfn(pfn))
|
|
|
|
return 1;
|
|
|
|
return get_page_unless_zero(pfn_to_page(pfn));
|
|
|
|
}
|
|
|
|
|
2016-06-07 22:22:47 +08:00
|
|
|
static int hva_to_pfn_remapped(struct vm_area_struct *vma,
|
|
|
|
unsigned long addr, bool *async,
|
2018-01-18 02:18:56 +08:00
|
|
|
bool write_fault, bool *writable,
|
|
|
|
kvm_pfn_t *p_pfn)
|
2016-06-07 22:22:47 +08:00
|
|
|
{
|
2021-02-09 04:19:40 +08:00
|
|
|
kvm_pfn_t pfn;
|
2021-02-01 18:12:11 +08:00
|
|
|
pte_t *ptep;
|
|
|
|
spinlock_t *ptl;
|
2016-06-07 23:51:18 +08:00
|
|
|
int r;
|
|
|
|
|
2021-02-05 18:07:11 +08:00
|
|
|
r = follow_pte(vma->vm_mm, addr, &ptep, &ptl);
|
2016-06-07 23:51:18 +08:00
|
|
|
if (r) {
|
|
|
|
/*
|
|
|
|
* get_user_pages fails for VM_IO and VM_PFNMAP vmas and does
|
|
|
|
* not call the fault handler, so do it here.
|
|
|
|
*/
|
|
|
|
bool unlocked = false;
|
2020-08-12 09:39:01 +08:00
|
|
|
r = fixup_user_fault(current->mm, addr,
|
2016-06-07 23:51:18 +08:00
|
|
|
(write_fault ? FAULT_FLAG_WRITE : 0),
|
|
|
|
&unlocked);
|
2020-05-29 17:42:55 +08:00
|
|
|
if (unlocked)
|
|
|
|
return -EAGAIN;
|
2016-06-07 23:51:18 +08:00
|
|
|
if (r)
|
|
|
|
return r;
|
|
|
|
|
2021-02-05 18:07:11 +08:00
|
|
|
r = follow_pte(vma->vm_mm, addr, &ptep, &ptl);
|
2016-06-07 23:51:18 +08:00
|
|
|
if (r)
|
|
|
|
return r;
|
2021-02-01 18:12:11 +08:00
|
|
|
}
|
2016-06-07 23:51:18 +08:00
|
|
|
|
2021-02-01 18:12:11 +08:00
|
|
|
if (write_fault && !pte_write(*ptep)) {
|
|
|
|
pfn = KVM_PFN_ERR_RO_FAULT;
|
|
|
|
goto out;
|
2016-06-07 23:51:18 +08:00
|
|
|
}
|
|
|
|
|
2018-01-18 02:18:56 +08:00
|
|
|
if (writable)
|
2021-02-01 18:12:11 +08:00
|
|
|
*writable = pte_write(*ptep);
|
|
|
|
pfn = pte_pfn(*ptep);
|
2016-06-07 23:51:18 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Get a reference here because callers of *hva_to_pfn* and
|
|
|
|
* *gfn_to_pfn* ultimately call kvm_release_pfn_clean on the
|
|
|
|
* returned pfn. This is only needed if the VMA has VM_MIXEDMAP
|
|
|
|
* set, but the kvm_get_pfn/kvm_release_pfn_clean pair will
|
|
|
|
* simply do nothing for reserved pfns.
|
|
|
|
*
|
|
|
|
* Whoever called remap_pfn_range is also going to call e.g.
|
|
|
|
* unmap_mapping_range before the underlying pages are freed,
|
|
|
|
* causing a call to our MMU notifier.
|
2021-06-24 20:29:04 +08:00
|
|
|
*
|
|
|
|
* Certain IO or PFNMAP mappings can be backed with valid
|
|
|
|
* struct pages, but be allocated without refcounting e.g.,
|
|
|
|
* tail pages of non-compound higher order allocations, which
|
|
|
|
* would then underflow the refcount when the caller does the
|
|
|
|
* required put_page. Don't allow those pages here.
|
2016-06-07 23:51:18 +08:00
|
|
|
*/
|
2021-06-24 20:29:04 +08:00
|
|
|
if (!kvm_try_get_pfn(pfn))
|
|
|
|
r = -EFAULT;
|
2016-06-07 23:51:18 +08:00
|
|
|
|
2021-02-01 18:12:11 +08:00
|
|
|
out:
|
|
|
|
pte_unmap_unlock(ptep, ptl);
|
2016-06-07 23:51:18 +08:00
|
|
|
*p_pfn = pfn;
|
2021-06-24 20:29:04 +08:00
|
|
|
|
|
|
|
return r;
|
2016-06-07 22:22:47 +08:00
|
|
|
}
|
|
|
|
|
2012-08-21 11:00:49 +08:00
|
|
|
/*
|
|
|
|
* Pin guest page in memory and return its pfn.
|
|
|
|
* @addr: host virtual address which maps memory to the guest
|
|
|
|
* @atomic: whether this function can sleep
|
|
|
|
* @async: whether this function need to wait IO complete if the
|
|
|
|
* host page is not in the memory
|
|
|
|
* @write_fault: whether we should get a writable host page
|
|
|
|
* @writable: whether it allows to map a writable host page for !@write_fault
|
|
|
|
*
|
|
|
|
* The function will map a writable host page for these two cases:
|
|
|
|
* 1): @write_fault = true
|
|
|
|
* 2): @write_fault = false && @writable, @writable will tell the caller
|
|
|
|
* whether the mapping is writable.
|
|
|
|
*/
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
static kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool *async,
|
2012-08-21 11:00:22 +08:00
|
|
|
bool write_fault, bool *writable)
|
|
|
|
{
|
|
|
|
struct vm_area_struct *vma;
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t pfn = 0;
|
2016-06-07 22:22:47 +08:00
|
|
|
int npages, r;
|
2008-05-01 04:37:07 +08:00
|
|
|
|
2012-08-21 11:00:22 +08:00
|
|
|
/* we can do it either atomically or asynchronously, not both */
|
|
|
|
BUG_ON(atomic && async);
|
2007-10-18 22:59:34 +08:00
|
|
|
|
2018-07-27 23:44:41 +08:00
|
|
|
if (hva_to_pfn_fast(addr, write_fault, writable, &pfn))
|
2012-08-21 11:00:22 +08:00
|
|
|
return pfn;
|
|
|
|
|
|
|
|
if (atomic)
|
|
|
|
return KVM_PFN_ERR_FAULT;
|
|
|
|
|
|
|
|
npages = hva_to_pfn_slow(addr, async, write_fault, writable, &pfn);
|
|
|
|
if (npages == 1)
|
|
|
|
return pfn;
|
2007-10-18 22:59:34 +08:00
|
|
|
|
2020-06-09 12:33:25 +08:00
|
|
|
mmap_read_lock(current->mm);
|
2012-08-21 11:00:22 +08:00
|
|
|
if (npages == -EHWPOISON ||
|
|
|
|
(!async && check_user_page_hwpoison(addr))) {
|
|
|
|
pfn = KVM_PFN_ERR_HWPOISON;
|
|
|
|
goto exit;
|
|
|
|
}
|
|
|
|
|
2020-05-29 17:42:55 +08:00
|
|
|
retry:
|
2021-06-29 10:39:17 +08:00
|
|
|
vma = vma_lookup(current->mm, addr);
|
2012-08-21 11:00:22 +08:00
|
|
|
|
|
|
|
if (vma == NULL)
|
|
|
|
pfn = KVM_PFN_ERR_FAULT;
|
2016-06-07 22:22:47 +08:00
|
|
|
else if (vma->vm_flags & (VM_IO | VM_PFNMAP)) {
|
2018-01-18 02:18:56 +08:00
|
|
|
r = hva_to_pfn_remapped(vma, addr, async, write_fault, writable, &pfn);
|
2020-05-29 17:42:55 +08:00
|
|
|
if (r == -EAGAIN)
|
|
|
|
goto retry;
|
2016-06-07 22:22:47 +08:00
|
|
|
if (r < 0)
|
|
|
|
pfn = KVM_PFN_ERR_FAULT;
|
2012-08-21 11:00:22 +08:00
|
|
|
} else {
|
2012-08-21 11:02:51 +08:00
|
|
|
if (async && vma_is_valid(vma, write_fault))
|
2012-08-21 11:00:22 +08:00
|
|
|
*async = true;
|
|
|
|
pfn = KVM_PFN_ERR_FAULT;
|
|
|
|
}
|
|
|
|
exit:
|
2020-06-09 12:33:25 +08:00
|
|
|
mmap_read_unlock(current->mm);
|
2008-05-01 04:37:07 +08:00
|
|
|
return pfn;
|
2008-04-03 03:46:56 +08:00
|
|
|
}
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t __gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
bool atomic, bool *async, bool write_fault,
|
2021-02-22 10:45:22 +08:00
|
|
|
bool *writable, hva_t *hva)
|
2010-08-22 19:10:28 +08:00
|
|
|
{
|
2012-08-21 11:02:51 +08:00
|
|
|
unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault);
|
|
|
|
|
2021-02-22 10:45:22 +08:00
|
|
|
if (hva)
|
|
|
|
*hva = addr;
|
|
|
|
|
2016-02-23 22:36:01 +08:00
|
|
|
if (addr == KVM_HVA_ERR_RO_BAD) {
|
|
|
|
if (writable)
|
|
|
|
*writable = false;
|
2012-08-21 11:02:51 +08:00
|
|
|
return KVM_PFN_ERR_RO_FAULT;
|
2016-02-23 22:36:01 +08:00
|
|
|
}
|
2012-08-21 11:02:51 +08:00
|
|
|
|
2016-02-23 22:36:01 +08:00
|
|
|
if (kvm_is_error_hva(addr)) {
|
|
|
|
if (writable)
|
|
|
|
*writable = false;
|
2012-10-16 20:10:59 +08:00
|
|
|
return KVM_PFN_NOSLOT;
|
2016-02-23 22:36:01 +08:00
|
|
|
}
|
2012-08-21 11:02:51 +08:00
|
|
|
|
|
|
|
/* Do not map writable pfn in the readonly memslot. */
|
|
|
|
if (writable && memslot_is_readonly(slot)) {
|
|
|
|
*writable = false;
|
|
|
|
writable = NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
return hva_to_pfn(addr, atomic, async, write_fault,
|
|
|
|
writable);
|
2010-08-22 19:10:28 +08:00
|
|
|
}
|
2015-04-02 17:20:48 +08:00
|
|
|
EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot);
|
2010-08-22 19:10:28 +08:00
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault,
|
2010-10-23 00:18:18 +08:00
|
|
|
bool *writable)
|
|
|
|
{
|
2015-05-19 22:09:04 +08:00
|
|
|
return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL,
|
2021-02-22 10:45:22 +08:00
|
|
|
write_fault, writable, NULL);
|
2010-10-23 00:18:18 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_pfn_prot);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn)
|
2009-12-24 00:35:19 +08:00
|
|
|
{
|
2021-02-22 10:45:22 +08:00
|
|
|
return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL);
|
2009-12-24 00:35:19 +08:00
|
|
|
}
|
2015-05-19 22:09:04 +08:00
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot);
|
2009-12-24 00:35:19 +08:00
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t gfn_to_pfn_memslot_atomic(struct kvm_memory_slot *slot, gfn_t gfn)
|
2009-12-24 00:35:19 +08:00
|
|
|
{
|
2021-02-22 10:45:22 +08:00
|
|
|
return __gfn_to_pfn_memslot(slot, gfn, true, NULL, true, NULL, NULL);
|
2009-12-24 00:35:19 +08:00
|
|
|
}
|
2012-08-21 10:59:12 +08:00
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic);
|
2009-12-24 00:35:19 +08:00
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn)
|
2015-05-17 19:58:53 +08:00
|
|
|
{
|
|
|
|
return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn)
|
2015-05-19 22:09:04 +08:00
|
|
|
{
|
|
|
|
return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_pfn);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn)
|
2015-05-17 19:58:53 +08:00
|
|
|
{
|
|
|
|
return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn);
|
|
|
|
|
2015-05-19 22:01:50 +08:00
|
|
|
int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
struct page **pages, int nr_pages)
|
2010-08-22 19:11:43 +08:00
|
|
|
{
|
|
|
|
unsigned long addr;
|
2017-08-10 20:14:39 +08:00
|
|
|
gfn_t entry = 0;
|
2010-08-22 19:11:43 +08:00
|
|
|
|
2015-05-19 22:01:50 +08:00
|
|
|
addr = gfn_to_hva_many(slot, gfn, &entry);
|
2010-08-22 19:11:43 +08:00
|
|
|
if (kvm_is_error_hva(addr))
|
|
|
|
return -1;
|
|
|
|
|
|
|
|
if (entry < nr_pages)
|
|
|
|
return 0;
|
|
|
|
|
2020-06-08 12:40:55 +08:00
|
|
|
return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages);
|
2010-08-22 19:11:43 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
static struct page *kvm_pfn_to_page(kvm_pfn_t pfn)
|
2012-07-26 11:58:59 +08:00
|
|
|
{
|
2012-10-16 20:10:59 +08:00
|
|
|
if (is_error_noslot_pfn(pfn))
|
2012-08-03 15:42:10 +08:00
|
|
|
return KVM_ERR_PTR_BAD_PAGE;
|
2012-07-26 11:58:59 +08:00
|
|
|
|
2014-11-10 16:33:56 +08:00
|
|
|
if (kvm_is_reserved_pfn(pfn)) {
|
2012-08-03 15:42:10 +08:00
|
|
|
WARN_ON(1);
|
2012-08-03 15:41:22 +08:00
|
|
|
return KVM_ERR_PTR_BAD_PAGE;
|
2012-08-03 15:42:10 +08:00
|
|
|
}
|
2012-07-26 11:58:59 +08:00
|
|
|
|
|
|
|
return pfn_to_page(pfn);
|
|
|
|
}
|
|
|
|
|
2008-04-03 03:46:56 +08:00
|
|
|
struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn)
|
|
|
|
{
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t pfn;
|
2008-05-01 04:37:07 +08:00
|
|
|
|
|
|
|
pfn = gfn_to_pfn(kvm, gfn);
|
|
|
|
|
2012-07-26 11:58:59 +08:00
|
|
|
return kvm_pfn_to_page(pfn);
|
2007-03-30 19:02:32 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(gfn_to_page);
|
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
void kvm_release_pfn(kvm_pfn_t pfn, bool dirty, struct gfn_to_pfn_cache *cache)
|
|
|
|
{
|
|
|
|
if (pfn == 0)
|
|
|
|
return;
|
|
|
|
|
|
|
|
if (cache)
|
|
|
|
cache->pfn = cache->gfn = 0;
|
|
|
|
|
|
|
|
if (dirty)
|
|
|
|
kvm_release_pfn_dirty(pfn);
|
|
|
|
else
|
|
|
|
kvm_release_pfn_clean(pfn);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_cache_gfn_to_pfn(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
struct gfn_to_pfn_cache *cache, u64 gen)
|
|
|
|
{
|
|
|
|
kvm_release_pfn(cache->pfn, cache->dirty, cache);
|
|
|
|
|
|
|
|
cache->pfn = gfn_to_pfn_memslot(slot, gfn);
|
|
|
|
cache->gfn = gfn;
|
|
|
|
cache->dirty = false;
|
|
|
|
cache->generation = gen;
|
|
|
|
}
|
|
|
|
|
2019-11-13 00:35:06 +08:00
|
|
|
static int __kvm_map_gfn(struct kvm_memslots *slots, gfn_t gfn,
|
2019-12-05 09:30:51 +08:00
|
|
|
struct kvm_host_map *map,
|
|
|
|
struct gfn_to_pfn_cache *cache,
|
|
|
|
bool atomic)
|
2019-02-01 04:24:34 +08:00
|
|
|
{
|
|
|
|
kvm_pfn_t pfn;
|
|
|
|
void *hva = NULL;
|
|
|
|
struct page *page = KVM_UNMAPPED_PAGE;
|
2019-11-13 00:35:06 +08:00
|
|
|
struct kvm_memory_slot *slot = __gfn_to_memslot(slots, gfn);
|
2019-12-05 09:30:51 +08:00
|
|
|
u64 gen = slots->generation;
|
2019-02-01 04:24:34 +08:00
|
|
|
|
|
|
|
if (!map)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
if (cache) {
|
|
|
|
if (!cache->pfn || cache->gfn != gfn ||
|
|
|
|
cache->generation != gen) {
|
|
|
|
if (atomic)
|
|
|
|
return -EAGAIN;
|
|
|
|
kvm_cache_gfn_to_pfn(slot, gfn, cache, gen);
|
|
|
|
}
|
|
|
|
pfn = cache->pfn;
|
|
|
|
} else {
|
|
|
|
if (atomic)
|
|
|
|
return -EAGAIN;
|
|
|
|
pfn = gfn_to_pfn_memslot(slot, gfn);
|
|
|
|
}
|
2019-02-01 04:24:34 +08:00
|
|
|
if (is_error_noslot_pfn(pfn))
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
if (pfn_valid(pfn)) {
|
|
|
|
page = pfn_to_page(pfn);
|
2019-12-05 09:30:51 +08:00
|
|
|
if (atomic)
|
|
|
|
hva = kmap_atomic(page);
|
|
|
|
else
|
|
|
|
hva = kmap(page);
|
2019-05-20 18:06:36 +08:00
|
|
|
#ifdef CONFIG_HAS_IOMEM
|
2019-12-05 09:30:51 +08:00
|
|
|
} else if (!atomic) {
|
2019-02-01 04:24:34 +08:00
|
|
|
hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB);
|
2019-12-05 09:30:51 +08:00
|
|
|
} else {
|
|
|
|
return -EINVAL;
|
2019-05-20 18:06:36 +08:00
|
|
|
#endif
|
2019-02-01 04:24:34 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!hva)
|
|
|
|
return -EFAULT;
|
|
|
|
|
|
|
|
map->page = page;
|
|
|
|
map->hva = hva;
|
|
|
|
map->pfn = pfn;
|
|
|
|
map->gfn = gfn;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
int kvm_map_gfn(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map,
|
|
|
|
struct gfn_to_pfn_cache *cache, bool atomic)
|
2019-11-13 00:35:06 +08:00
|
|
|
{
|
2019-12-05 09:30:51 +08:00
|
|
|
return __kvm_map_gfn(kvm_memslots(vcpu->kvm), gfn, map,
|
|
|
|
cache, atomic);
|
2019-11-13 00:35:06 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_map_gfn);
|
|
|
|
|
2019-02-01 04:24:34 +08:00
|
|
|
int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map)
|
|
|
|
{
|
2019-12-05 09:30:51 +08:00
|
|
|
return __kvm_map_gfn(kvm_vcpu_memslots(vcpu), gfn, map,
|
|
|
|
NULL, false);
|
2019-02-01 04:24:34 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_map);
|
|
|
|
|
2020-10-01 09:20:34 +08:00
|
|
|
static void __kvm_unmap_gfn(struct kvm *kvm,
|
|
|
|
struct kvm_memory_slot *memslot,
|
2019-12-05 09:30:51 +08:00
|
|
|
struct kvm_host_map *map,
|
|
|
|
struct gfn_to_pfn_cache *cache,
|
|
|
|
bool dirty, bool atomic)
|
2019-02-01 04:24:34 +08:00
|
|
|
{
|
|
|
|
if (!map)
|
|
|
|
return;
|
|
|
|
|
|
|
|
if (!map->hva)
|
|
|
|
return;
|
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
if (map->page != KVM_UNMAPPED_PAGE) {
|
|
|
|
if (atomic)
|
|
|
|
kunmap_atomic(map->hva);
|
|
|
|
else
|
|
|
|
kunmap(map->page);
|
|
|
|
}
|
2019-05-27 16:28:25 +08:00
|
|
|
#ifdef CONFIG_HAS_IOMEM
|
2019-12-05 09:30:51 +08:00
|
|
|
else if (!atomic)
|
2019-02-01 04:24:34 +08:00
|
|
|
memunmap(map->hva);
|
2019-12-05 09:30:51 +08:00
|
|
|
else
|
|
|
|
WARN_ONCE(1, "Unexpected unmapping in atomic context");
|
2019-05-27 16:28:25 +08:00
|
|
|
#endif
|
2019-02-01 04:24:34 +08:00
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
if (dirty)
|
2020-10-01 09:20:34 +08:00
|
|
|
mark_page_dirty_in_slot(kvm, memslot, map->gfn);
|
2019-12-05 09:30:51 +08:00
|
|
|
|
|
|
|
if (cache)
|
|
|
|
cache->dirty |= dirty;
|
|
|
|
else
|
|
|
|
kvm_release_pfn(map->pfn, dirty, NULL);
|
2019-02-01 04:24:34 +08:00
|
|
|
|
|
|
|
map->hva = NULL;
|
|
|
|
map->page = NULL;
|
|
|
|
}
|
2019-11-13 00:35:06 +08:00
|
|
|
|
2019-12-05 09:30:51 +08:00
|
|
|
int kvm_unmap_gfn(struct kvm_vcpu *vcpu, struct kvm_host_map *map,
|
|
|
|
struct gfn_to_pfn_cache *cache, bool dirty, bool atomic)
|
2019-11-13 00:35:06 +08:00
|
|
|
{
|
2020-10-01 09:20:34 +08:00
|
|
|
__kvm_unmap_gfn(vcpu->kvm, gfn_to_memslot(vcpu->kvm, map->gfn), map,
|
2019-12-05 09:30:51 +08:00
|
|
|
cache, dirty, atomic);
|
2019-11-13 00:35:06 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_unmap_gfn);
|
|
|
|
|
|
|
|
void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty)
|
|
|
|
{
|
2020-10-01 09:20:34 +08:00
|
|
|
__kvm_unmap_gfn(vcpu->kvm, kvm_vcpu_gfn_to_memslot(vcpu, map->gfn),
|
|
|
|
map, NULL, dirty, false);
|
2019-11-13 00:35:06 +08:00
|
|
|
}
|
2019-02-01 04:24:34 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_unmap);
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
struct page *kvm_vcpu_gfn_to_page(struct kvm_vcpu *vcpu, gfn_t gfn)
|
|
|
|
{
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
kvm_pfn_t pfn;
|
2015-05-17 19:58:53 +08:00
|
|
|
|
|
|
|
pfn = kvm_vcpu_gfn_to_pfn(vcpu, gfn);
|
|
|
|
|
|
|
|
return kvm_pfn_to_page(pfn);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_page);
|
|
|
|
|
2007-11-20 17:49:33 +08:00
|
|
|
void kvm_release_page_clean(struct page *page)
|
|
|
|
{
|
2012-08-03 15:42:52 +08:00
|
|
|
WARN_ON(is_error_page(page));
|
|
|
|
|
2008-04-03 03:46:56 +08:00
|
|
|
kvm_release_pfn_clean(page_to_pfn(page));
|
2007-11-20 17:49:33 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_release_page_clean);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
void kvm_release_pfn_clean(kvm_pfn_t pfn)
|
2008-04-03 03:46:56 +08:00
|
|
|
{
|
2014-11-10 16:33:56 +08:00
|
|
|
if (!is_error_noslot_pfn(pfn) && !kvm_is_reserved_pfn(pfn))
|
2008-05-01 04:37:07 +08:00
|
|
|
put_page(pfn_to_page(pfn));
|
2008-04-03 03:46:56 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_release_pfn_clean);
|
|
|
|
|
2007-11-20 17:49:33 +08:00
|
|
|
void kvm_release_page_dirty(struct page *page)
|
2007-10-18 17:09:33 +08:00
|
|
|
{
|
2012-07-26 11:58:59 +08:00
|
|
|
WARN_ON(is_error_page(page));
|
|
|
|
|
2008-04-03 03:46:56 +08:00
|
|
|
kvm_release_pfn_dirty(page_to_pfn(page));
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_release_page_dirty);
|
|
|
|
|
2017-09-01 23:11:43 +08:00
|
|
|
void kvm_release_pfn_dirty(kvm_pfn_t pfn)
|
2008-04-03 03:46:56 +08:00
|
|
|
{
|
|
|
|
kvm_set_pfn_dirty(pfn);
|
|
|
|
kvm_release_pfn_clean(pfn);
|
|
|
|
}
|
2017-09-01 23:11:43 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty);
|
2008-04-03 03:46:56 +08:00
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
void kvm_set_pfn_dirty(kvm_pfn_t pfn)
|
2008-04-03 03:46:56 +08:00
|
|
|
{
|
2019-12-05 11:05:05 +08:00
|
|
|
if (!kvm_is_reserved_pfn(pfn) && !kvm_is_zone_device_pfn(pfn))
|
|
|
|
SetPageDirty(pfn_to_page(pfn));
|
2007-10-18 17:09:33 +08:00
|
|
|
}
|
2008-04-03 03:46:56 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
void kvm_set_pfn_accessed(kvm_pfn_t pfn)
|
2008-04-03 03:46:56 +08:00
|
|
|
{
|
2019-11-12 06:12:27 +08:00
|
|
|
if (!kvm_is_reserved_pfn(pfn) && !kvm_is_zone_device_pfn(pfn))
|
2008-05-01 04:37:07 +08:00
|
|
|
mark_page_accessed(pfn_to_page(pfn));
|
2008-04-03 03:46:56 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed);
|
|
|
|
|
kvm: rename pfn_t to kvm_pfn_t
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace). This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o. It allows userspace to coordinate
DMA/RDMA from/to persistent memory.
The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver. The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.
The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag. Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.
Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array. Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory. The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.
This patch (of 18):
The core has developed a need for a "pfn_t" type [1]. Move the existing
pfn_t in KVM to kvm_pfn_t [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002199.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-September/002218.html
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-16 08:56:11 +08:00
|
|
|
void kvm_get_pfn(kvm_pfn_t pfn)
|
2008-04-03 03:46:56 +08:00
|
|
|
{
|
2014-11-10 16:33:56 +08:00
|
|
|
if (!kvm_is_reserved_pfn(pfn))
|
2008-05-01 04:37:07 +08:00
|
|
|
get_page(pfn_to_page(pfn));
|
2008-04-03 03:46:56 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_get_pfn);
|
2007-10-18 17:09:33 +08:00
|
|
|
|
2007-10-02 04:14:18 +08:00
|
|
|
static int next_segment(unsigned long len, int offset)
|
|
|
|
{
|
|
|
|
if (len > PAGE_SIZE - offset)
|
|
|
|
return PAGE_SIZE - offset;
|
|
|
|
else
|
|
|
|
return len;
|
|
|
|
}
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
void *data, int offset, int len)
|
2007-10-02 04:14:18 +08:00
|
|
|
{
|
2007-11-12 04:10:22 +08:00
|
|
|
int r;
|
|
|
|
unsigned long addr;
|
2007-10-02 04:14:18 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
addr = gfn_to_hva_memslot_prot(slot, gfn, NULL);
|
2007-11-12 04:10:22 +08:00
|
|
|
if (kvm_is_error_hva(addr))
|
|
|
|
return -EFAULT;
|
2015-04-02 20:08:20 +08:00
|
|
|
r = __copy_from_user(data, (void __user *)addr + offset, len);
|
2007-11-12 04:10:22 +08:00
|
|
|
if (r)
|
2007-10-02 04:14:18 +08:00
|
|
|
return -EFAULT;
|
|
|
|
return 0;
|
|
|
|
}
|
2015-05-17 19:58:53 +08:00
|
|
|
|
|
|
|
int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset,
|
|
|
|
int len)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
|
|
|
|
|
|
|
|
return __kvm_read_guest_page(slot, gfn, data, offset, len);
|
|
|
|
}
|
2007-10-02 04:14:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_read_guest_page);
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data,
|
|
|
|
int offset, int len)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
|
|
|
|
|
|
|
return __kvm_read_guest_page(slot, gfn, data, offset, len);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page);
|
|
|
|
|
2007-10-02 04:14:18 +08:00
|
|
|
int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len)
|
|
|
|
{
|
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
|
|
|
int seg;
|
|
|
|
int offset = offset_in_page(gpa);
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
while ((seg = next_segment(len, offset)) != 0) {
|
|
|
|
ret = kvm_read_guest_page(kvm, gfn, data, offset, seg);
|
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
offset = 0;
|
|
|
|
len -= seg;
|
|
|
|
data += seg;
|
|
|
|
++gfn;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_read_guest);
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
int kvm_vcpu_read_guest(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned long len)
|
2007-12-21 08:18:23 +08:00
|
|
|
{
|
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
2015-05-17 19:58:53 +08:00
|
|
|
int seg;
|
2007-12-21 08:18:23 +08:00
|
|
|
int offset = offset_in_page(gpa);
|
2015-05-17 19:58:53 +08:00
|
|
|
int ret;
|
|
|
|
|
|
|
|
while ((seg = next_segment(len, offset)) != 0) {
|
|
|
|
ret = kvm_vcpu_read_guest_page(vcpu, gfn, data, offset, seg);
|
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
offset = 0;
|
|
|
|
len -= seg;
|
|
|
|
data += seg;
|
|
|
|
++gfn;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest);
|
2007-12-21 08:18:23 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn,
|
|
|
|
void *data, int offset, unsigned long len)
|
|
|
|
{
|
|
|
|
int r;
|
|
|
|
unsigned long addr;
|
|
|
|
|
|
|
|
addr = gfn_to_hva_memslot_prot(slot, gfn, NULL);
|
2007-12-21 08:18:23 +08:00
|
|
|
if (kvm_is_error_hva(addr))
|
|
|
|
return -EFAULT;
|
2008-01-31 02:57:35 +08:00
|
|
|
pagefault_disable();
|
2015-04-02 20:08:20 +08:00
|
|
|
r = __copy_from_user_inatomic(data, (void __user *)addr + offset, len);
|
2008-01-31 02:57:35 +08:00
|
|
|
pagefault_enable();
|
2007-12-21 08:18:23 +08:00
|
|
|
if (r)
|
|
|
|
return -EFAULT;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa,
|
|
|
|
void *data, unsigned long len)
|
|
|
|
{
|
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
|
|
|
struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
|
|
|
int offset = offset_in_page(gpa);
|
|
|
|
|
|
|
|
return __kvm_read_guest_atomic(slot, gfn, data, offset, len);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic);
|
|
|
|
|
2020-10-01 09:20:34 +08:00
|
|
|
static int __kvm_write_guest_page(struct kvm *kvm,
|
|
|
|
struct kvm_memory_slot *memslot, gfn_t gfn,
|
2015-05-17 19:58:53 +08:00
|
|
|
const void *data, int offset, int len)
|
2007-10-02 04:14:18 +08:00
|
|
|
{
|
2007-11-12 04:10:22 +08:00
|
|
|
int r;
|
|
|
|
unsigned long addr;
|
2007-10-02 04:14:18 +08:00
|
|
|
|
2015-04-11 03:47:27 +08:00
|
|
|
addr = gfn_to_hva_memslot(memslot, gfn);
|
2007-11-12 04:10:22 +08:00
|
|
|
if (kvm_is_error_hva(addr))
|
|
|
|
return -EFAULT;
|
2011-05-15 23:22:04 +08:00
|
|
|
r = __copy_to_user((void __user *)addr + offset, data, len);
|
2007-11-12 04:10:22 +08:00
|
|
|
if (r)
|
2007-10-02 04:14:18 +08:00
|
|
|
return -EFAULT;
|
2020-10-01 09:20:34 +08:00
|
|
|
mark_page_dirty_in_slot(kvm, memslot, gfn);
|
2007-10-02 04:14:18 +08:00
|
|
|
return 0;
|
|
|
|
}
|
2015-05-17 19:58:53 +08:00
|
|
|
|
|
|
|
int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn,
|
|
|
|
const void *data, int offset, int len)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
|
|
|
|
|
2020-10-01 09:20:34 +08:00
|
|
|
return __kvm_write_guest_page(kvm, slot, gfn, data, offset, len);
|
2015-05-17 19:58:53 +08:00
|
|
|
}
|
2007-10-02 04:14:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_write_guest_page);
|
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn,
|
|
|
|
const void *data, int offset, int len)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
|
|
|
|
2020-10-01 09:20:34 +08:00
|
|
|
return __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len);
|
2015-05-17 19:58:53 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page);
|
|
|
|
|
2007-10-02 04:14:18 +08:00
|
|
|
int kvm_write_guest(struct kvm *kvm, gpa_t gpa, const void *data,
|
|
|
|
unsigned long len)
|
|
|
|
{
|
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
|
|
|
int seg;
|
|
|
|
int offset = offset_in_page(gpa);
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
while ((seg = next_segment(len, offset)) != 0) {
|
|
|
|
ret = kvm_write_guest_page(kvm, gfn, data, offset, seg);
|
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
offset = 0;
|
|
|
|
len -= seg;
|
|
|
|
data += seg;
|
|
|
|
++gfn;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
2014-12-11 13:52:58 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_write_guest);
|
2007-10-02 04:14:18 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
int kvm_vcpu_write_guest(struct kvm_vcpu *vcpu, gpa_t gpa, const void *data,
|
|
|
|
unsigned long len)
|
|
|
|
{
|
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
|
|
|
int seg;
|
|
|
|
int offset = offset_in_page(gpa);
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
while ((seg = next_segment(len, offset)) != 0) {
|
|
|
|
ret = kvm_vcpu_write_guest_page(vcpu, gfn, data, offset, seg);
|
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
offset = 0;
|
|
|
|
len -= seg;
|
|
|
|
data += seg;
|
|
|
|
++gfn;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest);
|
|
|
|
|
2017-02-04 12:32:28 +08:00
|
|
|
static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots,
|
|
|
|
struct gfn_to_hva_cache *ghc,
|
|
|
|
gpa_t gpa, unsigned long len)
|
2010-10-18 21:22:23 +08:00
|
|
|
{
|
|
|
|
int offset = offset_in_page(gpa);
|
2013-03-30 00:35:21 +08:00
|
|
|
gfn_t start_gfn = gpa >> PAGE_SHIFT;
|
|
|
|
gfn_t end_gfn = (gpa + len - 1) >> PAGE_SHIFT;
|
|
|
|
gfn_t nr_pages_needed = end_gfn - start_gfn + 1;
|
|
|
|
gfn_t nr_pages_avail;
|
2010-10-18 21:22:23 +08:00
|
|
|
|
2020-01-10 03:58:55 +08:00
|
|
|
/* Update ghc->generation before performing any error checks. */
|
2010-10-18 21:22:23 +08:00
|
|
|
ghc->generation = slots->generation;
|
2020-01-10 03:58:55 +08:00
|
|
|
|
|
|
|
if (start_gfn > end_gfn) {
|
|
|
|
ghc->hva = KVM_HVA_ERR_BAD;
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
2018-12-18 05:53:33 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If the requested region crosses two memslots, we still
|
|
|
|
* verify that the entire region is valid here.
|
|
|
|
*/
|
2020-01-10 03:58:55 +08:00
|
|
|
for ( ; start_gfn <= end_gfn; start_gfn += nr_pages_avail) {
|
2018-12-18 05:53:33 +08:00
|
|
|
ghc->memslot = __gfn_to_memslot(slots, start_gfn);
|
|
|
|
ghc->hva = gfn_to_hva_many(ghc->memslot, start_gfn,
|
|
|
|
&nr_pages_avail);
|
|
|
|
if (kvm_is_error_hva(ghc->hva))
|
2020-01-10 03:58:55 +08:00
|
|
|
return -EFAULT;
|
2018-12-18 05:53:33 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Use the slow path for cross page reads and writes. */
|
2020-01-10 03:58:55 +08:00
|
|
|
if (nr_pages_needed == 1)
|
2010-10-18 21:22:23 +08:00
|
|
|
ghc->hva += offset;
|
2018-12-18 05:53:33 +08:00
|
|
|
else
|
2013-03-30 00:35:21 +08:00
|
|
|
ghc->memslot = NULL;
|
2018-12-18 05:53:33 +08:00
|
|
|
|
2020-01-10 03:58:55 +08:00
|
|
|
ghc->gpa = gpa;
|
|
|
|
ghc->len = len;
|
|
|
|
return 0;
|
2010-10-18 21:22:23 +08:00
|
|
|
}
|
2017-02-04 12:32:28 +08:00
|
|
|
|
2017-05-02 22:20:18 +08:00
|
|
|
int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
|
2017-02-04 12:32:28 +08:00
|
|
|
gpa_t gpa, unsigned long len)
|
|
|
|
{
|
2017-05-02 22:20:18 +08:00
|
|
|
struct kvm_memslots *slots = kvm_memslots(kvm);
|
2017-02-04 12:32:28 +08:00
|
|
|
return __kvm_gfn_to_hva_cache_init(slots, ghc, gpa, len);
|
|
|
|
}
|
2017-05-02 22:20:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_gfn_to_hva_cache_init);
|
2010-10-18 21:22:23 +08:00
|
|
|
|
2017-05-02 22:20:18 +08:00
|
|
|
int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
|
2018-12-15 06:34:43 +08:00
|
|
|
void *data, unsigned int offset,
|
|
|
|
unsigned long len)
|
2010-10-18 21:22:23 +08:00
|
|
|
{
|
2017-05-02 22:20:18 +08:00
|
|
|
struct kvm_memslots *slots = kvm_memslots(kvm);
|
2010-10-18 21:22:23 +08:00
|
|
|
int r;
|
2016-11-02 17:08:34 +08:00
|
|
|
gpa_t gpa = ghc->gpa + offset;
|
2010-10-18 21:22:23 +08:00
|
|
|
|
2016-11-02 17:08:34 +08:00
|
|
|
BUG_ON(len + offset > ghc->len);
|
2013-03-30 00:35:21 +08:00
|
|
|
|
2020-01-10 07:56:20 +08:00
|
|
|
if (slots->generation != ghc->generation) {
|
|
|
|
if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len))
|
|
|
|
return -EFAULT;
|
|
|
|
}
|
2013-03-30 00:35:21 +08:00
|
|
|
|
2010-10-18 21:22:23 +08:00
|
|
|
if (kvm_is_error_hva(ghc->hva))
|
|
|
|
return -EFAULT;
|
|
|
|
|
KVM: Check for a bad hva before dropping into the ghc slow path
When reading/writing using the guest/host cache, check for a bad hva
before checking for a NULL memslot, which triggers the slow path for
handing cross-page accesses. Because the memslot is nullified on error
by __kvm_gfn_to_hva_cache_init(), if the bad hva is encountered after
crossing into a new page, then the kvm_{read,write}_guest() slow path
could potentially write/access the first chunk prior to detecting the
bad hva.
Arguably, performing a partial access is semantically correct from an
architectural perspective, but that behavior is certainly not intended.
In the original implementation, memslot was not explicitly nullified
and therefore the partial access behavior varied based on whether the
memslot itself was null, or if the hva was simply bad. The current
behavior was introduced as a seemingly unintentional side effect in
commit f1b9dd5eb86c ("kvm: Disallow wraparound in
kvm_gfn_to_hva_cache_init"), which justified the change with "since some
callers don't check the return code from this function, it sit seems
prudent to clear ghc->memslot in the event of an error".
Regardless of intent, the partial access is dependent on _not_ checking
the result of the cache initialization, which is arguably a bug in its
own right, at best simply weird.
Fixes: 8f964525a121 ("KVM: Allow cross page reads and writes from cached translations.")
Cc: Jim Mattson <jmattson@google.com>
Cc: Andrew Honig <ahonig@google.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-10 07:56:18 +08:00
|
|
|
if (unlikely(!ghc->memslot))
|
|
|
|
return kvm_write_guest(kvm, gpa, data, len);
|
|
|
|
|
2016-11-02 17:08:34 +08:00
|
|
|
r = __copy_to_user((void __user *)ghc->hva + offset, data, len);
|
2010-10-18 21:22:23 +08:00
|
|
|
if (r)
|
|
|
|
return -EFAULT;
|
2020-10-01 09:20:34 +08:00
|
|
|
mark_page_dirty_in_slot(kvm, ghc->memslot, gpa >> PAGE_SHIFT);
|
2010-10-18 21:22:23 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
2017-05-02 22:20:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_write_guest_offset_cached);
|
2016-11-02 17:08:34 +08:00
|
|
|
|
2017-05-02 22:20:18 +08:00
|
|
|
int kvm_write_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
|
|
|
|
void *data, unsigned long len)
|
2016-11-02 17:08:34 +08:00
|
|
|
{
|
2017-05-02 22:20:18 +08:00
|
|
|
return kvm_write_guest_offset_cached(kvm, ghc, data, 0, len);
|
2016-11-02 17:08:34 +08:00
|
|
|
}
|
2017-05-02 22:20:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_write_guest_cached);
|
2010-10-18 21:22:23 +08:00
|
|
|
|
2020-05-25 22:41:19 +08:00
|
|
|
int kvm_read_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
|
|
|
|
void *data, unsigned int offset,
|
|
|
|
unsigned long len)
|
2011-07-12 03:28:11 +08:00
|
|
|
{
|
2017-05-02 22:20:18 +08:00
|
|
|
struct kvm_memslots *slots = kvm_memslots(kvm);
|
2011-07-12 03:28:11 +08:00
|
|
|
int r;
|
2020-05-25 22:41:19 +08:00
|
|
|
gpa_t gpa = ghc->gpa + offset;
|
2011-07-12 03:28:11 +08:00
|
|
|
|
2020-05-25 22:41:19 +08:00
|
|
|
BUG_ON(len + offset > ghc->len);
|
2013-03-30 00:35:21 +08:00
|
|
|
|
2020-01-10 07:56:20 +08:00
|
|
|
if (slots->generation != ghc->generation) {
|
|
|
|
if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len))
|
|
|
|
return -EFAULT;
|
|
|
|
}
|
2013-03-30 00:35:21 +08:00
|
|
|
|
2011-07-12 03:28:11 +08:00
|
|
|
if (kvm_is_error_hva(ghc->hva))
|
|
|
|
return -EFAULT;
|
|
|
|
|
KVM: Check for a bad hva before dropping into the ghc slow path
When reading/writing using the guest/host cache, check for a bad hva
before checking for a NULL memslot, which triggers the slow path for
handing cross-page accesses. Because the memslot is nullified on error
by __kvm_gfn_to_hva_cache_init(), if the bad hva is encountered after
crossing into a new page, then the kvm_{read,write}_guest() slow path
could potentially write/access the first chunk prior to detecting the
bad hva.
Arguably, performing a partial access is semantically correct from an
architectural perspective, but that behavior is certainly not intended.
In the original implementation, memslot was not explicitly nullified
and therefore the partial access behavior varied based on whether the
memslot itself was null, or if the hva was simply bad. The current
behavior was introduced as a seemingly unintentional side effect in
commit f1b9dd5eb86c ("kvm: Disallow wraparound in
kvm_gfn_to_hva_cache_init"), which justified the change with "since some
callers don't check the return code from this function, it sit seems
prudent to clear ghc->memslot in the event of an error".
Regardless of intent, the partial access is dependent on _not_ checking
the result of the cache initialization, which is arguably a bug in its
own right, at best simply weird.
Fixes: 8f964525a121 ("KVM: Allow cross page reads and writes from cached translations.")
Cc: Jim Mattson <jmattson@google.com>
Cc: Andrew Honig <ahonig@google.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-10 07:56:18 +08:00
|
|
|
if (unlikely(!ghc->memslot))
|
2020-05-25 22:41:19 +08:00
|
|
|
return kvm_read_guest(kvm, gpa, data, len);
|
KVM: Check for a bad hva before dropping into the ghc slow path
When reading/writing using the guest/host cache, check for a bad hva
before checking for a NULL memslot, which triggers the slow path for
handing cross-page accesses. Because the memslot is nullified on error
by __kvm_gfn_to_hva_cache_init(), if the bad hva is encountered after
crossing into a new page, then the kvm_{read,write}_guest() slow path
could potentially write/access the first chunk prior to detecting the
bad hva.
Arguably, performing a partial access is semantically correct from an
architectural perspective, but that behavior is certainly not intended.
In the original implementation, memslot was not explicitly nullified
and therefore the partial access behavior varied based on whether the
memslot itself was null, or if the hva was simply bad. The current
behavior was introduced as a seemingly unintentional side effect in
commit f1b9dd5eb86c ("kvm: Disallow wraparound in
kvm_gfn_to_hva_cache_init"), which justified the change with "since some
callers don't check the return code from this function, it sit seems
prudent to clear ghc->memslot in the event of an error".
Regardless of intent, the partial access is dependent on _not_ checking
the result of the cache initialization, which is arguably a bug in its
own right, at best simply weird.
Fixes: 8f964525a121 ("KVM: Allow cross page reads and writes from cached translations.")
Cc: Jim Mattson <jmattson@google.com>
Cc: Andrew Honig <ahonig@google.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-10 07:56:18 +08:00
|
|
|
|
2020-05-25 22:41:19 +08:00
|
|
|
r = __copy_from_user(data, (void __user *)ghc->hva + offset, len);
|
2011-07-12 03:28:11 +08:00
|
|
|
if (r)
|
|
|
|
return -EFAULT;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
2020-05-25 22:41:19 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_read_guest_offset_cached);
|
|
|
|
|
|
|
|
int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
|
|
|
|
void *data, unsigned long len)
|
|
|
|
{
|
|
|
|
return kvm_read_guest_offset_cached(kvm, ghc, data, 0, len);
|
|
|
|
}
|
2017-05-02 22:20:18 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_read_guest_cached);
|
2011-07-12 03:28:11 +08:00
|
|
|
|
2007-10-02 04:14:18 +08:00
|
|
|
int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len)
|
|
|
|
{
|
2020-11-06 18:25:09 +08:00
|
|
|
const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0)));
|
2007-10-02 04:14:18 +08:00
|
|
|
gfn_t gfn = gpa >> PAGE_SHIFT;
|
|
|
|
int seg;
|
|
|
|
int offset = offset_in_page(gpa);
|
|
|
|
int ret;
|
|
|
|
|
2015-02-20 21:21:36 +08:00
|
|
|
while ((seg = next_segment(len, offset)) != 0) {
|
2020-11-06 18:25:09 +08:00
|
|
|
ret = kvm_write_guest_page(kvm, gfn, zero_page, offset, len);
|
2007-10-02 04:14:18 +08:00
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
offset = 0;
|
|
|
|
len -= seg;
|
|
|
|
++gfn;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_clear_guest);
|
|
|
|
|
2020-10-01 09:20:34 +08:00
|
|
|
void mark_page_dirty_in_slot(struct kvm *kvm,
|
|
|
|
struct kvm_memory_slot *memslot,
|
|
|
|
gfn_t gfn)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2020-10-01 09:22:26 +08:00
|
|
|
if (memslot && kvm_slot_dirty_track_enabled(memslot)) {
|
2007-07-31 18:41:14 +08:00
|
|
|
unsigned long rel_gfn = gfn - memslot->base_gfn;
|
2020-10-01 09:22:22 +08:00
|
|
|
u32 slot = (memslot->as_id << 16) | memslot->id;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2020-10-01 09:22:22 +08:00
|
|
|
if (kvm->dirty_ring_size)
|
|
|
|
kvm_dirty_ring_push(kvm_dirty_ring_get(kvm),
|
|
|
|
slot, rel_gfn);
|
|
|
|
else
|
|
|
|
set_bit_le(rel_gfn, memslot->dirty_bitmap);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
}
|
2020-10-15 02:26:55 +08:00
|
|
|
EXPORT_SYMBOL_GPL(mark_page_dirty_in_slot);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2010-10-18 21:22:23 +08:00
|
|
|
void mark_page_dirty(struct kvm *kvm, gfn_t gfn)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *memslot;
|
|
|
|
|
|
|
|
memslot = gfn_to_memslot(kvm, gfn);
|
2020-10-01 09:20:34 +08:00
|
|
|
mark_page_dirty_in_slot(kvm, memslot, gfn);
|
2010-10-18 21:22:23 +08:00
|
|
|
}
|
2013-10-08 00:47:59 +08:00
|
|
|
EXPORT_SYMBOL_GPL(mark_page_dirty);
|
2010-10-18 21:22:23 +08:00
|
|
|
|
2015-05-17 19:58:53 +08:00
|
|
|
void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn)
|
|
|
|
{
|
|
|
|
struct kvm_memory_slot *memslot;
|
|
|
|
|
|
|
|
memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
|
2020-10-01 09:20:34 +08:00
|
|
|
mark_page_dirty_in_slot(vcpu->kvm, memslot, gfn);
|
2015-05-17 19:58:53 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_mark_page_dirty);
|
|
|
|
|
2017-11-25 05:39:01 +08:00
|
|
|
void kvm_sigset_activate(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
if (!vcpu->sigset_active)
|
|
|
|
return;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This does a lockless modification of ->real_blocked, which is fine
|
|
|
|
* because, only current can change ->real_blocked and all readers of
|
|
|
|
* ->real_blocked don't care as long ->real_blocked is always a subset
|
|
|
|
* of ->blocked.
|
|
|
|
*/
|
|
|
|
sigprocmask(SIG_SETMASK, &vcpu->sigset, ¤t->real_blocked);
|
|
|
|
}
|
|
|
|
|
|
|
|
void kvm_sigset_deactivate(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
if (!vcpu->sigset_active)
|
|
|
|
return;
|
|
|
|
|
|
|
|
sigprocmask(SIG_SETMASK, ¤t->real_blocked, NULL);
|
|
|
|
sigemptyset(¤t->real_blocked);
|
|
|
|
}
|
|
|
|
|
2015-09-03 22:07:38 +08:00
|
|
|
static void grow_halt_poll_ns(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
2019-01-27 18:17:16 +08:00
|
|
|
unsigned int old, val, grow, grow_start;
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2015-09-03 22:07:39 +08:00
|
|
|
old = val = vcpu->halt_poll_ns;
|
2019-01-27 18:17:16 +08:00
|
|
|
grow_start = READ_ONCE(halt_poll_ns_grow_start);
|
2016-02-09 20:47:55 +08:00
|
|
|
grow = READ_ONCE(halt_poll_ns_grow);
|
2019-01-27 18:17:14 +08:00
|
|
|
if (!grow)
|
|
|
|
goto out;
|
|
|
|
|
2019-01-27 18:17:16 +08:00
|
|
|
val *= grow;
|
|
|
|
if (val < grow_start)
|
|
|
|
val = grow_start;
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2021-05-06 23:24:43 +08:00
|
|
|
if (val > vcpu->kvm->max_halt_poll_ns)
|
|
|
|
val = vcpu->kvm->max_halt_poll_ns;
|
2016-03-09 08:19:44 +08:00
|
|
|
|
2015-09-03 22:07:38 +08:00
|
|
|
vcpu->halt_poll_ns = val;
|
2019-01-27 18:17:14 +08:00
|
|
|
out:
|
2015-09-03 22:07:39 +08:00
|
|
|
trace_kvm_halt_poll_ns_grow(vcpu->vcpu_id, val, old);
|
2015-09-03 22:07:38 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static void shrink_halt_poll_ns(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
2016-02-09 20:47:55 +08:00
|
|
|
unsigned int old, val, shrink;
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2015-09-03 22:07:39 +08:00
|
|
|
old = val = vcpu->halt_poll_ns;
|
2016-02-09 20:47:55 +08:00
|
|
|
shrink = READ_ONCE(halt_poll_ns_shrink);
|
|
|
|
if (shrink == 0)
|
2015-09-03 22:07:38 +08:00
|
|
|
val = 0;
|
|
|
|
else
|
2016-02-09 20:47:55 +08:00
|
|
|
val /= shrink;
|
2015-09-03 22:07:38 +08:00
|
|
|
|
|
|
|
vcpu->halt_poll_ns = val;
|
2015-09-03 22:07:39 +08:00
|
|
|
trace_kvm_halt_poll_ns_shrink(vcpu->vcpu_id, val, old);
|
2015-09-03 22:07:38 +08:00
|
|
|
}
|
|
|
|
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
static int kvm_vcpu_check_block(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
2018-06-28 05:59:11 +08:00
|
|
|
int ret = -EINTR;
|
|
|
|
int idx = srcu_read_lock(&vcpu->kvm->srcu);
|
|
|
|
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
if (kvm_arch_vcpu_runnable(vcpu)) {
|
|
|
|
kvm_make_request(KVM_REQ_UNHALT, vcpu);
|
2018-06-28 05:59:11 +08:00
|
|
|
goto out;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
}
|
|
|
|
if (kvm_cpu_has_pending_timer(vcpu))
|
2018-06-28 05:59:11 +08:00
|
|
|
goto out;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
if (signal_pending(current))
|
2018-06-28 05:59:11 +08:00
|
|
|
goto out;
|
2021-05-25 21:41:17 +08:00
|
|
|
if (kvm_check_request(KVM_REQ_UNBLOCK, vcpu))
|
|
|
|
goto out;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
|
2018-06-28 05:59:11 +08:00
|
|
|
ret = 0;
|
|
|
|
out:
|
|
|
|
srcu_read_unlock(&vcpu->kvm->srcu, idx);
|
|
|
|
return ret;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
}
|
|
|
|
|
2020-05-09 02:22:40 +08:00
|
|
|
static inline void
|
|
|
|
update_halt_poll_stats(struct kvm_vcpu *vcpu, u64 poll_ns, bool waited)
|
|
|
|
{
|
|
|
|
if (waited)
|
2021-06-19 06:27:03 +08:00
|
|
|
vcpu->stat.generic.halt_poll_fail_ns += poll_ns;
|
2020-05-09 02:22:40 +08:00
|
|
|
else
|
2021-06-19 06:27:03 +08:00
|
|
|
vcpu->stat.generic.halt_poll_success_ns += poll_ns;
|
2020-05-09 02:22:40 +08:00
|
|
|
}
|
|
|
|
|
2007-07-18 17:15:21 +08:00
|
|
|
/*
|
|
|
|
* The vCPU has executed a HLT instruction with in-kernel mode enabled.
|
|
|
|
*/
|
2007-11-01 06:24:24 +08:00
|
|
|
void kvm_vcpu_block(struct kvm_vcpu *vcpu)
|
2007-06-05 20:53:05 +08:00
|
|
|
{
|
2020-05-09 02:22:40 +08:00
|
|
|
ktime_t start, cur, poll_end;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
bool waited = false;
|
2015-09-03 22:07:38 +08:00
|
|
|
u64 block_ns;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
|
2019-08-02 18:37:09 +08:00
|
|
|
kvm_arch_vcpu_blocking(vcpu);
|
|
|
|
|
2020-05-09 02:22:40 +08:00
|
|
|
start = cur = poll_end = ktime_get();
|
2019-03-05 18:30:01 +08:00
|
|
|
if (vcpu->halt_poll_ns && !kvm_arch_no_poll(vcpu)) {
|
2015-09-03 22:07:37 +08:00
|
|
|
ktime_t stop = ktime_add_ns(ktime_get(), vcpu->halt_poll_ns);
|
2015-02-26 14:58:23 +08:00
|
|
|
|
2021-06-19 06:27:03 +08:00
|
|
|
++vcpu->stat.generic.halt_attempted_poll;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
do {
|
|
|
|
/*
|
|
|
|
* This sets KVM_REQ_UNHALT if an interrupt
|
|
|
|
* arrives.
|
|
|
|
*/
|
|
|
|
if (kvm_vcpu_check_block(vcpu) < 0) {
|
2021-06-19 06:27:03 +08:00
|
|
|
++vcpu->stat.generic.halt_successful_poll;
|
2016-05-13 18:16:35 +08:00
|
|
|
if (!vcpu_valid_wakeup(vcpu))
|
2021-06-19 06:27:03 +08:00
|
|
|
++vcpu->stat.generic.halt_poll_invalid;
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
goto out;
|
|
|
|
}
|
2021-07-27 19:12:47 +08:00
|
|
|
cpu_relax();
|
2020-05-09 02:22:40 +08:00
|
|
|
poll_end = cur = ktime_get();
|
2021-05-18 20:00:31 +08:00
|
|
|
} while (kvm_vcpu_can_poll(cur, stop));
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
}
|
2008-05-09 06:47:01 +08:00
|
|
|
|
2020-04-24 13:48:37 +08:00
|
|
|
prepare_to_rcuwait(&vcpu->wait);
|
2008-05-09 06:47:01 +08:00
|
|
|
for (;;) {
|
2020-04-24 13:48:37 +08:00
|
|
|
set_current_state(TASK_INTERRUPTIBLE);
|
2008-05-09 06:47:01 +08:00
|
|
|
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
if (kvm_vcpu_check_block(vcpu) < 0)
|
2008-05-09 06:47:01 +08:00
|
|
|
break;
|
|
|
|
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
waited = true;
|
2007-07-18 17:15:21 +08:00
|
|
|
schedule();
|
|
|
|
}
|
2020-04-24 13:48:37 +08:00
|
|
|
finish_rcuwait(&vcpu->wait);
|
kvm: add halt_poll_ns module parameter
This patch introduces a new module parameter for the KVM module; when it
is present, KVM attempts a bit of polling on every HLT before scheduling
itself out via kvm_vcpu_block.
This parameter helps a lot for latency-bound workloads---in particular
I tested it with O_DSYNC writes with a battery-backed disk in the host.
In this case, writes are fast (because the data doesn't have to go all
the way to the platters) but they cannot be merged by either the host or
the guest. KVM's performance here is usually around 30% of bare metal,
or 50% if you use cache=directsync or cache=writethrough (these
parameters avoid that the guest sends pointless flush requests, and
at the same time they are not slow because of the battery-backed cache).
The bad performance happens because on every halt the host CPU decides
to halt itself too. When the interrupt comes, the vCPU thread is then
migrated to a new physical CPU, and in general the latency is horrible
because the vCPU thread has to be scheduled back in.
With this patch performance reaches 60-65% of bare metal and, more
important, 99% of what you get if you use idle=poll in the guest. This
means that the tunable gets rid of this particular bottleneck, and more
work can be done to improve performance in the kernel or QEMU.
Of course there is some price to pay; every time an otherwise idle vCPUs
is interrupted by an interrupt, it will poll unnecessarily and thus
impose a little load on the host. The above results were obtained with
a mostly random value of the parameter (500000), and the load was around
1.5-2.5% CPU usage on one of the host's core for each idle guest vCPU.
The patch also adds a new stat, /sys/kernel/debug/kvm/halt_successful_poll,
that can be used to tune the parameter. It counts how many HLT
instructions received an interrupt during the polling period; each
successful poll avoids that Linux schedules the VCPU thread out and back
in, and may also avoid a likely trip to C1 and back for the physical CPU.
While the VM is idle, a Linux 4 VCPU VM halts around 10 times per second.
Of these halts, almost all are failed polls. During the benchmark,
instead, basically all halts end within the polling period, except a more
or less constant stream of 50 per second coming from vCPUs that are not
running the benchmark. The wasted time is thus very low. Things may
be slightly different for Windows VMs, which have a ~10 ms timer tick.
The effect is also visible on Marcelo's recently-introduced latency
test for the TSC deadline timer. Though of course a non-RT kernel has
awful latency bounds, the latency of the timer is around 8000-10000 clock
cycles compared to 20000-120000 without setting halt_poll_ns. For the TSC
deadline timer, thus, the effect is both a smaller average latency and
a smaller variance.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-02-05 01:20:58 +08:00
|
|
|
cur = ktime_get();
|
|
|
|
out:
|
2019-08-02 18:37:09 +08:00
|
|
|
kvm_arch_vcpu_unblocking(vcpu);
|
2015-09-03 22:07:38 +08:00
|
|
|
block_ns = ktime_to_ns(cur) - ktime_to_ns(start);
|
|
|
|
|
2020-05-09 02:22:40 +08:00
|
|
|
update_halt_poll_stats(
|
|
|
|
vcpu, ktime_to_ns(ktime_sub(poll_end, start)), waited);
|
|
|
|
|
2019-09-29 09:06:56 +08:00
|
|
|
if (!kvm_arch_no_poll(vcpu)) {
|
|
|
|
if (!vcpu_valid_wakeup(vcpu)) {
|
2015-09-03 22:07:38 +08:00
|
|
|
shrink_halt_poll_ns(vcpu);
|
2020-04-18 06:14:46 +08:00
|
|
|
} else if (vcpu->kvm->max_halt_poll_ns) {
|
2019-09-29 09:06:56 +08:00
|
|
|
if (block_ns <= vcpu->halt_poll_ns)
|
|
|
|
;
|
|
|
|
/* we had a long block, shrink polling */
|
2020-04-18 06:14:46 +08:00
|
|
|
else if (vcpu->halt_poll_ns &&
|
|
|
|
block_ns > vcpu->kvm->max_halt_poll_ns)
|
2019-09-29 09:06:56 +08:00
|
|
|
shrink_halt_poll_ns(vcpu);
|
|
|
|
/* we had a short halt and our poll time is too small */
|
2020-04-18 06:14:46 +08:00
|
|
|
else if (vcpu->halt_poll_ns < vcpu->kvm->max_halt_poll_ns &&
|
|
|
|
block_ns < vcpu->kvm->max_halt_poll_ns)
|
2019-09-29 09:06:56 +08:00
|
|
|
grow_halt_poll_ns(vcpu);
|
|
|
|
} else {
|
|
|
|
vcpu->halt_poll_ns = 0;
|
|
|
|
}
|
|
|
|
}
|
2015-09-03 22:07:38 +08:00
|
|
|
|
2016-05-13 18:16:35 +08:00
|
|
|
trace_kvm_vcpu_wakeup(block_ns, waited, vcpu_valid_wakeup(vcpu));
|
|
|
|
kvm_arch_vcpu_block_finish(vcpu);
|
2007-07-18 17:15:21 +08:00
|
|
|
}
|
2013-10-08 00:47:59 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_block);
|
2007-07-18 17:15:21 +08:00
|
|
|
|
2017-04-27 04:32:26 +08:00
|
|
|
bool kvm_vcpu_wake_up(struct kvm_vcpu *vcpu)
|
2012-03-09 05:44:24 +08:00
|
|
|
{
|
2020-04-24 13:48:37 +08:00
|
|
|
struct rcuwait *waitp;
|
2012-03-09 05:44:24 +08:00
|
|
|
|
2020-04-24 13:48:37 +08:00
|
|
|
waitp = kvm_arch_vcpu_get_wait(vcpu);
|
|
|
|
if (rcuwait_wake_up(waitp)) {
|
KVM: Boost vCPUs that are delivering interrupts
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 155%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 179%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-18 19:39:06 +08:00
|
|
|
WRITE_ONCE(vcpu->ready, true);
|
2021-06-19 06:27:03 +08:00
|
|
|
++vcpu->stat.generic.halt_wakeup;
|
2017-04-27 04:32:26 +08:00
|
|
|
return true;
|
2012-03-09 05:44:24 +08:00
|
|
|
}
|
|
|
|
|
2017-04-27 04:32:26 +08:00
|
|
|
return false;
|
2016-05-05 03:09:44 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_wake_up);
|
|
|
|
|
2017-05-04 21:14:13 +08:00
|
|
|
#ifndef CONFIG_S390
|
2016-05-05 03:09:44 +08:00
|
|
|
/*
|
|
|
|
* Kick a sleeping VCPU, or a guest VCPU in guest mode, into host kernel mode.
|
|
|
|
*/
|
|
|
|
void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
int me;
|
|
|
|
int cpu = vcpu->cpu;
|
|
|
|
|
2017-04-27 04:32:26 +08:00
|
|
|
if (kvm_vcpu_wake_up(vcpu))
|
|
|
|
return;
|
|
|
|
|
2012-03-09 05:44:24 +08:00
|
|
|
me = get_cpu();
|
|
|
|
if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
|
|
|
|
if (kvm_arch_vcpu_should_kick(vcpu))
|
|
|
|
smp_send_reschedule(cpu);
|
|
|
|
put_cpu();
|
|
|
|
}
|
2013-04-11 19:25:15 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_kick);
|
2017-05-04 21:14:13 +08:00
|
|
|
#endif /* !CONFIG_S390 */
|
2012-03-09 05:44:24 +08:00
|
|
|
|
2014-05-23 18:20:42 +08:00
|
|
|
int kvm_vcpu_yield_to(struct kvm_vcpu *target)
|
2012-04-25 21:30:38 +08:00
|
|
|
{
|
|
|
|
struct pid *pid;
|
|
|
|
struct task_struct *task = NULL;
|
2014-05-23 18:20:42 +08:00
|
|
|
int ret = 0;
|
2012-04-25 21:30:38 +08:00
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
pid = rcu_dereference(target->pid);
|
|
|
|
if (pid)
|
2014-09-19 07:40:41 +08:00
|
|
|
task = get_pid_task(pid, PIDTYPE_PID);
|
2012-04-25 21:30:38 +08:00
|
|
|
rcu_read_unlock();
|
|
|
|
if (!task)
|
2013-01-22 15:39:24 +08:00
|
|
|
return ret;
|
|
|
|
ret = yield_to(task, 1);
|
2012-04-25 21:30:38 +08:00
|
|
|
put_task_struct(task);
|
2013-01-22 15:39:24 +08:00
|
|
|
|
|
|
|
return ret;
|
2012-04-25 21:30:38 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_yield_to);
|
|
|
|
|
2012-07-19 17:47:52 +08:00
|
|
|
/*
|
|
|
|
* Helper that checks whether a VCPU is eligible for directed yield.
|
|
|
|
* Most eligible candidate to yield is decided by following heuristics:
|
|
|
|
*
|
|
|
|
* (a) VCPU which has not done pl-exit or cpu relax intercepted recently
|
|
|
|
* (preempted lock holder), indicated by @in_spin_loop.
|
2020-04-01 22:03:10 +08:00
|
|
|
* Set at the beginning and cleared at the end of interception/PLE handler.
|
2012-07-19 17:47:52 +08:00
|
|
|
*
|
|
|
|
* (b) VCPU which has done pl-exit/ cpu relax intercepted but did not get
|
|
|
|
* chance last time (mostly it has become eligible now since we have probably
|
|
|
|
* yielded to lockholder in last iteration. This is done by toggling
|
|
|
|
* @dy_eligible each time a VCPU checked for eligibility.)
|
|
|
|
*
|
|
|
|
* Yielding to a recently pl-exited/cpu relax intercepted VCPU before yielding
|
|
|
|
* to preempted lock-holder could result in wrong VCPU selection and CPU
|
|
|
|
* burning. Giving priority for a potential lock-holder increases lock
|
|
|
|
* progress.
|
|
|
|
*
|
|
|
|
* Since algorithm is based on heuristics, accessing another VCPU data without
|
|
|
|
* locking does not harm. It may result in trying to yield to same VCPU, fail
|
|
|
|
* and continue with next VCPU and so on.
|
|
|
|
*/
|
2013-12-30 04:12:29 +08:00
|
|
|
static bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu)
|
2012-07-19 17:47:52 +08:00
|
|
|
{
|
2014-01-10 08:43:16 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
|
2012-07-19 17:47:52 +08:00
|
|
|
bool eligible;
|
|
|
|
|
|
|
|
eligible = !vcpu->spin_loop.in_spin_loop ||
|
2014-09-05 03:13:31 +08:00
|
|
|
vcpu->spin_loop.dy_eligible;
|
2012-07-19 17:47:52 +08:00
|
|
|
|
|
|
|
if (vcpu->spin_loop.in_spin_loop)
|
|
|
|
kvm_vcpu_set_dy_eligible(vcpu, !vcpu->spin_loop.dy_eligible);
|
|
|
|
|
|
|
|
return eligible;
|
2014-01-10 08:43:16 +08:00
|
|
|
#else
|
|
|
|
return true;
|
2012-07-19 17:47:52 +08:00
|
|
|
#endif
|
2014-01-10 08:43:16 +08:00
|
|
|
}
|
2013-01-22 15:39:24 +08:00
|
|
|
|
2019-08-05 10:03:19 +08:00
|
|
|
/*
|
|
|
|
* Unlike kvm_arch_vcpu_runnable, this function is called outside
|
|
|
|
* a vcpu_load/vcpu_put pair. However, for most architectures
|
|
|
|
* kvm_arch_vcpu_runnable does not require vcpu_load.
|
|
|
|
*/
|
|
|
|
bool __weak kvm_arch_dy_runnable(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
return kvm_arch_vcpu_runnable(vcpu);
|
|
|
|
}
|
|
|
|
|
|
|
|
static bool vcpu_dy_runnable(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
if (kvm_arch_dy_runnable(vcpu))
|
|
|
|
return true;
|
|
|
|
|
|
|
|
#ifdef CONFIG_KVM_ASYNC_PF
|
|
|
|
if (!list_empty_careful(&vcpu->async_pf.done))
|
|
|
|
return true;
|
|
|
|
#endif
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2021-04-16 11:08:10 +08:00
|
|
|
bool __weak kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2017-08-08 12:05:32 +08:00
|
|
|
void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode)
|
2009-10-09 18:03:20 +08:00
|
|
|
{
|
2011-02-01 22:53:28 +08:00
|
|
|
struct kvm *kvm = me->kvm;
|
|
|
|
struct kvm_vcpu *vcpu;
|
|
|
|
int last_boosted_vcpu = me->kvm->last_boosted_vcpu;
|
|
|
|
int yielded = 0;
|
2013-01-22 15:39:24 +08:00
|
|
|
int try = 3;
|
2011-02-01 22:53:28 +08:00
|
|
|
int pass;
|
|
|
|
int i;
|
2009-10-09 18:03:20 +08:00
|
|
|
|
2012-07-18 21:37:46 +08:00
|
|
|
kvm_vcpu_set_in_spin_loop(me, true);
|
2011-02-01 22:53:28 +08:00
|
|
|
/*
|
|
|
|
* We boost the priority of a VCPU that is runnable but not
|
|
|
|
* currently running, because it got preempted by something
|
|
|
|
* else and called schedule in __vcpu_run. Hopefully that
|
|
|
|
* VCPU is holding the lock that we need and will release it.
|
|
|
|
* We approximate round-robin by starting at the last boosted VCPU.
|
|
|
|
*/
|
2013-01-22 15:39:24 +08:00
|
|
|
for (pass = 0; pass < 2 && !yielded && try; pass++) {
|
2011-02-01 22:53:28 +08:00
|
|
|
kvm_for_each_vcpu(i, vcpu, kvm) {
|
2012-06-20 04:51:04 +08:00
|
|
|
if (!pass && i <= last_boosted_vcpu) {
|
2011-02-01 22:53:28 +08:00
|
|
|
i = last_boosted_vcpu;
|
|
|
|
continue;
|
|
|
|
} else if (pass && i > last_boosted_vcpu)
|
|
|
|
break;
|
KVM: Boost vCPUs that are delivering interrupts
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 155%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 179%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-18 19:39:06 +08:00
|
|
|
if (!READ_ONCE(vcpu->ready))
|
2013-03-05 02:02:27 +08:00
|
|
|
continue;
|
2011-02-01 22:53:28 +08:00
|
|
|
if (vcpu == me)
|
|
|
|
continue;
|
2020-04-24 13:48:37 +08:00
|
|
|
if (rcuwait_active(&vcpu->wait) &&
|
|
|
|
!vcpu_dy_runnable(vcpu))
|
2011-02-01 22:53:28 +08:00
|
|
|
continue;
|
2019-08-01 11:30:14 +08:00
|
|
|
if (READ_ONCE(vcpu->preempted) && yield_to_kernel_mode &&
|
2021-04-16 11:08:10 +08:00
|
|
|
!kvm_arch_dy_has_pending_interrupt(vcpu) &&
|
|
|
|
!kvm_arch_vcpu_in_kernel(vcpu))
|
2017-08-08 12:05:32 +08:00
|
|
|
continue;
|
2012-07-19 17:47:52 +08:00
|
|
|
if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
|
|
|
|
continue;
|
2013-01-22 15:39:24 +08:00
|
|
|
|
|
|
|
yielded = kvm_vcpu_yield_to(vcpu);
|
|
|
|
if (yielded > 0) {
|
2011-02-01 22:53:28 +08:00
|
|
|
kvm->last_boosted_vcpu = i;
|
|
|
|
break;
|
2013-01-22 15:39:24 +08:00
|
|
|
} else if (yielded < 0) {
|
|
|
|
try--;
|
|
|
|
if (!try)
|
|
|
|
break;
|
2011-02-01 22:53:28 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2012-07-18 21:37:46 +08:00
|
|
|
kvm_vcpu_set_in_spin_loop(me, false);
|
2012-07-19 17:47:52 +08:00
|
|
|
|
|
|
|
/* Ensure vcpu is not eligible during next spinloop */
|
|
|
|
kvm_vcpu_set_dy_eligible(me, false);
|
2009-10-09 18:03:20 +08:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_vcpu_on_spin);
|
|
|
|
|
2020-10-01 09:22:22 +08:00
|
|
|
static bool kvm_page_in_dirty_ring(struct kvm *kvm, unsigned long pgoff)
|
|
|
|
{
|
|
|
|
#if KVM_DIRTY_LOG_PAGE_OFFSET > 0
|
|
|
|
return (pgoff >= KVM_DIRTY_LOG_PAGE_OFFSET) &&
|
|
|
|
(pgoff < KVM_DIRTY_LOG_PAGE_OFFSET +
|
|
|
|
kvm->dirty_ring_size / PAGE_SIZE);
|
|
|
|
#else
|
|
|
|
return false;
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2018-04-19 03:19:58 +08:00
|
|
|
static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf)
|
2007-02-22 18:58:31 +08:00
|
|
|
{
|
2017-02-25 06:56:41 +08:00
|
|
|
struct kvm_vcpu *vcpu = vmf->vma->vm_file->private_data;
|
2007-02-22 18:58:31 +08:00
|
|
|
struct page *page;
|
|
|
|
|
2007-12-05 15:15:52 +08:00
|
|
|
if (vmf->pgoff == 0)
|
2007-03-20 18:46:50 +08:00
|
|
|
page = virt_to_page(vcpu->run);
|
2008-01-24 00:14:23 +08:00
|
|
|
#ifdef CONFIG_X86
|
2007-12-05 15:15:52 +08:00
|
|
|
else if (vmf->pgoff == KVM_PIO_PAGE_OFFSET)
|
2007-12-13 23:50:52 +08:00
|
|
|
page = virt_to_page(vcpu->arch.pio_data);
|
2008-05-30 22:05:54 +08:00
|
|
|
#endif
|
2017-03-31 19:53:23 +08:00
|
|
|
#ifdef CONFIG_KVM_MMIO
|
2008-05-30 22:05:54 +08:00
|
|
|
else if (vmf->pgoff == KVM_COALESCED_MMIO_PAGE_OFFSET)
|
|
|
|
page = virt_to_page(vcpu->kvm->coalesced_mmio_ring);
|
2008-01-24 00:14:23 +08:00
|
|
|
#endif
|
2020-10-01 09:22:22 +08:00
|
|
|
else if (kvm_page_in_dirty_ring(vcpu->kvm, vmf->pgoff))
|
|
|
|
page = kvm_dirty_ring_get_page(
|
|
|
|
&vcpu->dirty_ring,
|
|
|
|
vmf->pgoff - KVM_DIRTY_LOG_PAGE_OFFSET);
|
2007-03-20 18:46:50 +08:00
|
|
|
else
|
2012-01-04 17:25:23 +08:00
|
|
|
return kvm_arch_vcpu_fault(vcpu, vmf);
|
2007-02-22 18:58:31 +08:00
|
|
|
get_page(page);
|
2007-12-05 15:15:52 +08:00
|
|
|
vmf->page = page;
|
|
|
|
return 0;
|
2007-02-22 18:58:31 +08:00
|
|
|
}
|
|
|
|
|
2009-09-28 02:29:37 +08:00
|
|
|
static const struct vm_operations_struct kvm_vcpu_vm_ops = {
|
2007-12-05 15:15:52 +08:00
|
|
|
.fault = kvm_vcpu_fault,
|
2007-02-22 18:58:31 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
static int kvm_vcpu_mmap(struct file *file, struct vm_area_struct *vma)
|
|
|
|
{
|
2020-10-01 09:22:22 +08:00
|
|
|
struct kvm_vcpu *vcpu = file->private_data;
|
|
|
|
unsigned long pages = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
|
|
|
|
|
|
|
|
if ((kvm_page_in_dirty_ring(vcpu->kvm, vma->vm_pgoff) ||
|
|
|
|
kvm_page_in_dirty_ring(vcpu->kvm, vma->vm_pgoff + pages - 1)) &&
|
|
|
|
((vma->vm_flags & VM_EXEC) || !(vma->vm_flags & VM_SHARED)))
|
|
|
|
return -EINVAL;
|
|
|
|
|
2007-02-22 18:58:31 +08:00
|
|
|
vma->vm_ops = &kvm_vcpu_vm_ops;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-02-22 00:04:26 +08:00
|
|
|
static int kvm_vcpu_release(struct inode *inode, struct file *filp)
|
|
|
|
{
|
|
|
|
struct kvm_vcpu *vcpu = filp->private_data;
|
|
|
|
|
2008-04-20 03:33:56 +08:00
|
|
|
kvm_put_kvm(vcpu->kvm);
|
2007-02-22 00:04:26 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-12-02 18:17:32 +08:00
|
|
|
static struct file_operations kvm_vcpu_fops = {
|
2007-02-22 00:04:26 +08:00
|
|
|
.release = kvm_vcpu_release,
|
|
|
|
.unlocked_ioctl = kvm_vcpu_ioctl,
|
2007-02-22 18:58:31 +08:00
|
|
|
.mmap = kvm_vcpu_mmap,
|
llseek: automatically add .llseek fop
All file_operations should get a .llseek operation so we can make
nonseekable_open the default for future file operations without a
.llseek pointer.
The three cases that we can automatically detect are no_llseek, seq_lseek
and default_llseek. For cases where we can we can automatically prove that
the file offset is always ignored, we use noop_llseek, which maintains
the current behavior of not returning an error from a seek.
New drivers should normally not use noop_llseek but instead use no_llseek
and call nonseekable_open at open time. Existing drivers can be converted
to do the same when the maintainer knows for certain that no user code
relies on calling seek on the device file.
The generated code is often incorrectly indented and right now contains
comments that clarify for each added line why a specific variant was
chosen. In the version that gets submitted upstream, the comments will
be gone and I will manually fix the indentation, because there does not
seem to be a way to do that using coccinelle.
Some amount of new code is currently sitting in linux-next that should get
the same modifications, which I will do at the end of the merge window.
Many thanks to Julia Lawall for helping me learn to write a semantic
patch that does all this.
===== begin semantic patch =====
// This adds an llseek= method to all file operations,
// as a preparation for making no_llseek the default.
//
// The rules are
// - use no_llseek explicitly if we do nonseekable_open
// - use seq_lseek for sequential files
// - use default_llseek if we know we access f_pos
// - use noop_llseek if we know we don't access f_pos,
// but we still want to allow users to call lseek
//
@ open1 exists @
identifier nested_open;
@@
nested_open(...)
{
<+...
nonseekable_open(...)
...+>
}
@ open exists@
identifier open_f;
identifier i, f;
identifier open1.nested_open;
@@
int open_f(struct inode *i, struct file *f)
{
<+...
(
nonseekable_open(...)
|
nested_open(...)
)
...+>
}
@ read disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ read_no_fpos disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
... when != off
}
@ write @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ write_no_fpos @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
... when != off
}
@ fops0 @
identifier fops;
@@
struct file_operations fops = {
...
};
@ has_llseek depends on fops0 @
identifier fops0.fops;
identifier llseek_f;
@@
struct file_operations fops = {
...
.llseek = llseek_f,
...
};
@ has_read depends on fops0 @
identifier fops0.fops;
identifier read_f;
@@
struct file_operations fops = {
...
.read = read_f,
...
};
@ has_write depends on fops0 @
identifier fops0.fops;
identifier write_f;
@@
struct file_operations fops = {
...
.write = write_f,
...
};
@ has_open depends on fops0 @
identifier fops0.fops;
identifier open_f;
@@
struct file_operations fops = {
...
.open = open_f,
...
};
// use no_llseek if we call nonseekable_open
////////////////////////////////////////////
@ nonseekable1 depends on !has_llseek && has_open @
identifier fops0.fops;
identifier nso ~= "nonseekable_open";
@@
struct file_operations fops = {
... .open = nso, ...
+.llseek = no_llseek, /* nonseekable */
};
@ nonseekable2 depends on !has_llseek @
identifier fops0.fops;
identifier open.open_f;
@@
struct file_operations fops = {
... .open = open_f, ...
+.llseek = no_llseek, /* open uses nonseekable */
};
// use seq_lseek for sequential files
/////////////////////////////////////
@ seq depends on !has_llseek @
identifier fops0.fops;
identifier sr ~= "seq_read";
@@
struct file_operations fops = {
... .read = sr, ...
+.llseek = seq_lseek, /* we have seq_read */
};
// use default_llseek if there is a readdir
///////////////////////////////////////////
@ fops1 depends on !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier readdir_e;
@@
// any other fop is used that changes pos
struct file_operations fops = {
... .readdir = readdir_e, ...
+.llseek = default_llseek, /* readdir is present */
};
// use default_llseek if at least one of read/write touches f_pos
/////////////////////////////////////////////////////////////////
@ fops2 depends on !fops1 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read.read_f;
@@
// read fops use offset
struct file_operations fops = {
... .read = read_f, ...
+.llseek = default_llseek, /* read accesses f_pos */
};
@ fops3 depends on !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write.write_f;
@@
// write fops use offset
struct file_operations fops = {
... .write = write_f, ...
+ .llseek = default_llseek, /* write accesses f_pos */
};
// Use noop_llseek if neither read nor write accesses f_pos
///////////////////////////////////////////////////////////
@ fops4 depends on !fops1 && !fops2 && !fops3 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
identifier write_no_fpos.write_f;
@@
// write fops use offset
struct file_operations fops = {
...
.write = write_f,
.read = read_f,
...
+.llseek = noop_llseek, /* read and write both use no f_pos */
};
@ depends on has_write && !has_read && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write_no_fpos.write_f;
@@
struct file_operations fops = {
... .write = write_f, ...
+.llseek = noop_llseek, /* write uses no f_pos */
};
@ depends on has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
@@
struct file_operations fops = {
... .read = read_f, ...
+.llseek = noop_llseek, /* read uses no f_pos */
};
@ depends on !has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
@@
struct file_operations fops = {
...
+.llseek = noop_llseek, /* no read or write fn */
};
===== End semantic patch =====
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Julia Lawall <julia@diku.dk>
Cc: Christoph Hellwig <hch@infradead.org>
2010-08-16 00:52:59 +08:00
|
|
|
.llseek = noop_llseek,
|
2018-06-17 17:16:21 +08:00
|
|
|
KVM_COMPAT(kvm_vcpu_compat_ioctl),
|
2007-02-22 00:04:26 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Allocates an inode for the vcpu.
|
|
|
|
*/
|
|
|
|
static int create_vcpu_fd(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
kvm: embed vcpu id to dentry of vcpu anon inode
All d-entries for vcpu have the same, "anon_inode:kvm-vcpu". That means
it is impossible to know the mapping between fds for vcpu and vcpu
from userland.
# LC_ALL=C ls -l /proc/617/fd | grep vcpu
lrwx------. 1 qemu qemu 64 Jan 7 16:50 18 -> anon_inode:kvm-vcpu
lrwx------. 1 qemu qemu 64 Jan 7 16:50 19 -> anon_inode:kvm-vcpu
It is also impossible to know the mapping between vma for kvm_run
structure and vcpu from userland.
# LC_ALL=C grep vcpu /proc/617/maps
7f9d842d0000-7f9d842d3000 rw-s 00000000 00:0d 20393 anon_inode:kvm-vcpu
7f9d842d3000-7f9d842d6000 rw-s 00000000 00:0d 20393 anon_inode:kvm-vcpu
This change adds vcpu id to d-entries for vcpu. With this change
you can get the following output:
# LC_ALL=C ls -l /proc/617/fd | grep vcpu
lrwx------. 1 qemu qemu 64 Jan 7 16:50 18 -> anon_inode:kvm-vcpu:0
lrwx------. 1 qemu qemu 64 Jan 7 16:50 19 -> anon_inode:kvm-vcpu:1
# LC_ALL=C grep vcpu /proc/617/maps
7f9d842d0000-7f9d842d3000 rw-s 00000000 00:0d 20393 anon_inode:kvm-vcpu:0
7f9d842d3000-7f9d842d6000 rw-s 00000000 00:0d 20393 anon_inode:kvm-vcpu:1
With the mappings known from the output, a tool like strace can report more details
of qemu-kvm process activities. Here is the strace output of my local prototype:
# ./strace -KK -f -p 617 2>&1 | grep 'KVM_RUN\| K'
...
[pid 664] ioctl(18, KVM_RUN, 0) = 0 (KVM_EXIT_MMIO)
K ready_for_interrupt_injection=1, if_flag=0, flags=0, cr8=0000000000000000, apic_base=0x000000fee00d00
K phys_addr=0, len=1634035803, [33, 0, 0, 0, 0, 0, 0, 0], is_write=112
[pid 664] ioctl(18, KVM_RUN, 0) = 0 (KVM_EXIT_MMIO)
K ready_for_interrupt_injection=1, if_flag=1, flags=0, cr8=0000000000000000, apic_base=0x000000fee00d00
K phys_addr=0, len=1634035803, [33, 0, 0, 0, 0, 0, 0, 0], is_write=112
...
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-01-20 03:04:22 +08:00
|
|
|
char name[8 + 1 + ITOA_MAX_LEN + 1];
|
|
|
|
|
|
|
|
snprintf(name, sizeof(name), "kvm-vcpu:%d", vcpu->vcpu_id);
|
|
|
|
return anon_inode_getfd(name, &kvm_vcpu_fops, vcpu, O_RDWR | O_CLOEXEC);
|
2007-02-22 00:04:26 +08:00
|
|
|
}
|
|
|
|
|
2019-08-01 02:56:20 +08:00
|
|
|
static void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu)
|
2016-09-16 22:27:35 +08:00
|
|
|
{
|
2019-08-03 14:14:25 +08:00
|
|
|
#ifdef __KVM_HAVE_ARCH_VCPU_DEBUGFS
|
2020-06-04 21:16:52 +08:00
|
|
|
struct dentry *debugfs_dentry;
|
2016-09-16 22:27:35 +08:00
|
|
|
char dir_name[ITOA_MAX_LEN * 2];
|
|
|
|
|
|
|
|
if (!debugfs_initialized())
|
2019-08-01 02:56:20 +08:00
|
|
|
return;
|
2016-09-16 22:27:35 +08:00
|
|
|
|
|
|
|
snprintf(dir_name, sizeof(dir_name), "vcpu%d", vcpu->vcpu_id);
|
2020-06-04 21:16:52 +08:00
|
|
|
debugfs_dentry = debugfs_create_dir(dir_name,
|
|
|
|
vcpu->kvm->debugfs_dentry);
|
2016-09-16 22:27:35 +08:00
|
|
|
|
2020-06-04 21:16:52 +08:00
|
|
|
kvm_arch_create_vcpu_debugfs(vcpu, debugfs_dentry);
|
2019-08-03 14:14:25 +08:00
|
|
|
#endif
|
2016-09-16 22:27:35 +08:00
|
|
|
}
|
|
|
|
|
2007-02-21 00:41:05 +08:00
|
|
|
/*
|
|
|
|
* Creates some virtual cpus. Good luck creating more than one.
|
|
|
|
*/
|
2009-06-09 20:56:28 +08:00
|
|
|
static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)
|
2007-02-21 00:41:05 +08:00
|
|
|
{
|
|
|
|
int r;
|
2015-11-05 16:03:50 +08:00
|
|
|
struct kvm_vcpu *vcpu;
|
2019-12-19 05:55:30 +08:00
|
|
|
struct page *page;
|
2007-02-21 00:41:05 +08:00
|
|
|
|
2016-05-10 00:13:37 +08:00
|
|
|
if (id >= KVM_MAX_VCPU_ID)
|
2013-11-19 08:09:22 +08:00
|
|
|
return -EINVAL;
|
|
|
|
|
2016-06-13 20:48:25 +08:00
|
|
|
mutex_lock(&kvm->lock);
|
|
|
|
if (kvm->created_vcpus == KVM_MAX_VCPUS) {
|
|
|
|
mutex_unlock(&kvm->lock);
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
kvm->created_vcpus++;
|
|
|
|
mutex_unlock(&kvm->lock);
|
|
|
|
|
2019-12-19 05:55:09 +08:00
|
|
|
r = kvm_arch_vcpu_precreate(kvm, id);
|
|
|
|
if (r)
|
|
|
|
goto vcpu_decrement;
|
|
|
|
|
2021-04-07 03:07:40 +08:00
|
|
|
vcpu = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT);
|
2019-12-19 05:55:15 +08:00
|
|
|
if (!vcpu) {
|
|
|
|
r = -ENOMEM;
|
2016-06-13 20:48:25 +08:00
|
|
|
goto vcpu_decrement;
|
|
|
|
}
|
2007-02-21 00:41:05 +08:00
|
|
|
|
2020-01-09 22:57:12 +08:00
|
|
|
BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE);
|
2020-12-19 06:01:38 +08:00
|
|
|
page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
|
2019-12-19 05:55:30 +08:00
|
|
|
if (!page) {
|
|
|
|
r = -ENOMEM;
|
2019-12-19 05:55:15 +08:00
|
|
|
goto vcpu_free;
|
2019-12-19 05:55:30 +08:00
|
|
|
}
|
|
|
|
vcpu->run = page_address(page);
|
|
|
|
|
|
|
|
kvm_vcpu_init(vcpu, kvm, id);
|
2019-12-19 05:55:15 +08:00
|
|
|
|
|
|
|
r = kvm_arch_vcpu_create(vcpu);
|
|
|
|
if (r)
|
2019-12-19 05:55:30 +08:00
|
|
|
goto vcpu_free_run_page;
|
2019-12-19 05:55:15 +08:00
|
|
|
|
2020-10-01 09:22:22 +08:00
|
|
|
if (kvm->dirty_ring_size) {
|
|
|
|
r = kvm_dirty_ring_alloc(&vcpu->dirty_ring,
|
|
|
|
id, kvm->dirty_ring_size);
|
|
|
|
if (r)
|
|
|
|
goto arch_vcpu_destroy;
|
|
|
|
}
|
|
|
|
|
2007-07-23 14:51:37 +08:00
|
|
|
mutex_lock(&kvm->lock);
|
2015-11-05 16:03:50 +08:00
|
|
|
if (kvm_get_vcpu_by_id(kvm, id)) {
|
|
|
|
r = -EEXIST;
|
|
|
|
goto unlock_vcpu_destroy;
|
|
|
|
}
|
2009-06-09 20:56:28 +08:00
|
|
|
|
2019-11-07 20:53:42 +08:00
|
|
|
vcpu->vcpu_idx = atomic_read(&kvm->online_vcpus);
|
|
|
|
BUG_ON(kvm->vcpus[vcpu->vcpu_idx]);
|
2007-02-21 00:41:05 +08:00
|
|
|
|
2021-06-19 06:27:06 +08:00
|
|
|
/* Fill the stats id string for the vcpu */
|
|
|
|
snprintf(vcpu->stats_id, sizeof(vcpu->stats_id), "kvm-%d/vcpu-%d",
|
|
|
|
task_pid_nr(current), id);
|
|
|
|
|
2007-07-27 15:16:56 +08:00
|
|
|
/* Now it's all set up, let userspace reach it */
|
2008-04-20 03:33:56 +08:00
|
|
|
kvm_get_kvm(kvm);
|
2007-02-22 00:04:26 +08:00
|
|
|
r = create_vcpu_fd(vcpu);
|
2009-06-09 20:56:28 +08:00
|
|
|
if (r < 0) {
|
2019-10-22 06:58:42 +08:00
|
|
|
kvm_put_kvm_no_destroy(kvm);
|
2011-05-23 16:33:05 +08:00
|
|
|
goto unlock_vcpu_destroy;
|
2009-06-09 20:56:28 +08:00
|
|
|
}
|
|
|
|
|
2019-11-07 20:53:42 +08:00
|
|
|
kvm->vcpus[vcpu->vcpu_idx] = vcpu;
|
2015-07-29 17:32:20 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Pairs with smp_rmb() in kvm_get_vcpu. Write kvm->vcpus
|
|
|
|
* before kvm->online_vcpu's incremented value.
|
|
|
|
*/
|
2009-06-09 20:56:28 +08:00
|
|
|
smp_wmb();
|
|
|
|
atomic_inc(&kvm->online_vcpus);
|
|
|
|
|
|
|
|
mutex_unlock(&kvm->lock);
|
2012-11-28 09:29:02 +08:00
|
|
|
kvm_arch_vcpu_postcreate(vcpu);
|
2020-04-01 06:42:22 +08:00
|
|
|
kvm_create_vcpu_debugfs(vcpu);
|
2007-07-27 15:16:56 +08:00
|
|
|
return r;
|
2007-06-08 00:11:53 +08:00
|
|
|
|
2011-05-23 16:33:05 +08:00
|
|
|
unlock_vcpu_destroy:
|
2008-09-18 10:16:59 +08:00
|
|
|
mutex_unlock(&kvm->lock);
|
2020-10-01 09:22:22 +08:00
|
|
|
kvm_dirty_ring_free(&vcpu->dirty_ring);
|
|
|
|
arch_vcpu_destroy:
|
2007-11-20 04:04:43 +08:00
|
|
|
kvm_arch_vcpu_destroy(vcpu);
|
2019-12-19 05:55:30 +08:00
|
|
|
vcpu_free_run_page:
|
|
|
|
free_page((unsigned long)vcpu->run);
|
2019-12-19 05:55:15 +08:00
|
|
|
vcpu_free:
|
|
|
|
kmem_cache_free(kvm_vcpu_cache, vcpu);
|
2016-06-13 20:48:25 +08:00
|
|
|
vcpu_decrement:
|
|
|
|
mutex_lock(&kvm->lock);
|
|
|
|
kvm->created_vcpus--;
|
|
|
|
mutex_unlock(&kvm->lock);
|
2007-02-21 00:41:05 +08:00
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2007-03-06 01:46:05 +08:00
|
|
|
static int kvm_vcpu_ioctl_set_sigmask(struct kvm_vcpu *vcpu, sigset_t *sigset)
|
|
|
|
{
|
|
|
|
if (sigset) {
|
|
|
|
sigdelsetmask(sigset, sigmask(SIGKILL)|sigmask(SIGSTOP));
|
|
|
|
vcpu->sigset_active = 1;
|
|
|
|
vcpu->sigset = *sigset;
|
|
|
|
} else
|
|
|
|
vcpu->sigset_active = 0;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2021-06-19 06:27:06 +08:00
|
|
|
static ssize_t kvm_vcpu_stats_read(struct file *file, char __user *user_buffer,
|
|
|
|
size_t size, loff_t *offset)
|
|
|
|
{
|
|
|
|
struct kvm_vcpu *vcpu = file->private_data;
|
|
|
|
|
|
|
|
return kvm_stats_read(vcpu->stats_id, &kvm_vcpu_stats_header,
|
|
|
|
&kvm_vcpu_stats_desc[0], &vcpu->stat,
|
|
|
|
sizeof(vcpu->stat), user_buffer, size, offset);
|
|
|
|
}
|
|
|
|
|
|
|
|
static const struct file_operations kvm_vcpu_stats_fops = {
|
|
|
|
.read = kvm_vcpu_stats_read,
|
|
|
|
.llseek = noop_llseek,
|
|
|
|
};
|
|
|
|
|
|
|
|
static int kvm_vcpu_ioctl_get_stats_fd(struct kvm_vcpu *vcpu)
|
|
|
|
{
|
|
|
|
int fd;
|
|
|
|
struct file *file;
|
|
|
|
char name[15 + ITOA_MAX_LEN + 1];
|
|
|
|
|
|
|
|
snprintf(name, sizeof(name), "kvm-vcpu-stats:%d", vcpu->vcpu_id);
|
|
|
|
|
|
|
|
fd = get_unused_fd_flags(O_CLOEXEC);
|
|
|
|
if (fd < 0)
|
|
|
|
return fd;
|
|
|
|
|
|
|
|
file = anon_inode_getfile(name, &kvm_vcpu_stats_fops, vcpu, O_RDONLY);
|
|
|
|
if (IS_ERR(file)) {
|
|
|
|
put_unused_fd(fd);
|
|
|
|
return PTR_ERR(file);
|
|
|
|
}
|
|
|
|
file->f_mode |= FMODE_PREAD;
|
|
|
|
fd_install(fd, file);
|
|
|
|
|
|
|
|
return fd;
|
|
|
|
}
|
|
|
|
|
2007-02-22 00:04:26 +08:00
|
|
|
static long kvm_vcpu_ioctl(struct file *filp,
|
|
|
|
unsigned int ioctl, unsigned long arg)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2007-02-22 00:04:26 +08:00
|
|
|
struct kvm_vcpu *vcpu = filp->private_data;
|
2007-02-10 00:38:35 +08:00
|
|
|
void __user *argp = (void __user *)arg;
|
KVM: Portability: split kvm_vcpu_ioctl
This patch splits kvm_vcpu_ioctl into archtecture independent parts, and
x86 specific parts which go to kvm_arch_vcpu_ioctl in x86.c.
Common ioctls for all architectures are:
KVM_RUN, KVM_GET/SET_(S-)REGS, KVM_TRANSLATE, KVM_INTERRUPT,
KVM_DEBUG_GUEST, KVM_SET_SIGNAL_MASK, KVM_GET/SET_FPU
Note that some PPC chips don't have an FPU, so we might need an #ifdef
around KVM_GET/SET_FPU one day.
x86 specific ioctls are:
KVM_GET/SET_LAPIC, KVM_SET_CPUID, KVM_GET/SET_MSRS
An interresting aspect is vcpu_load/vcpu_put. We now have a common
vcpu_load/put which does the preemption stuff, and an architecture
specific kvm_arch_vcpu_load/put. In the x86 case, this one calls the
vmx/svm function defined in kvm_x86_ops.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-10-12 01:16:52 +08:00
|
|
|
int r;
|
2008-08-12 01:01:46 +08:00
|
|
|
struct kvm_fpu *fpu = NULL;
|
|
|
|
struct kvm_sregs *kvm_sregs = NULL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2021-07-03 06:04:23 +08:00
|
|
|
if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)
|
2007-11-21 22:41:05 +08:00
|
|
|
return -EIO;
|
2010-05-13 16:25:04 +08:00
|
|
|
|
2014-09-20 07:03:25 +08:00
|
|
|
if (unlikely(_IOC_TYPE(ioctl) != KVMIO))
|
|
|
|
return -EINVAL;
|
|
|
|
|
2010-05-13 16:25:04 +08:00
|
|
|
/*
|
2017-12-13 00:41:34 +08:00
|
|
|
* Some architectures have vcpu ioctls that are asynchronous to vcpu
|
|
|
|
* execution; mutex_lock() would break them.
|
2010-05-13 16:25:04 +08:00
|
|
|
*/
|
2017-12-13 00:41:34 +08:00
|
|
|
r = kvm_arch_vcpu_async_ioctl(filp, ioctl, arg);
|
|
|
|
if (r != -ENOIOCTLCMD)
|
2012-09-16 16:50:30 +08:00
|
|
|
return r;
|
2010-05-13 16:25:04 +08:00
|
|
|
|
2017-12-05 04:35:23 +08:00
|
|
|
if (mutex_lock_killable(&vcpu->mutex))
|
|
|
|
return -EINTR;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
switch (ioctl) {
|
2017-07-06 20:44:28 +08:00
|
|
|
case KVM_RUN: {
|
|
|
|
struct pid *oldpid;
|
2007-03-07 19:11:17 +08:00
|
|
|
r = -EINVAL;
|
|
|
|
if (arg)
|
|
|
|
goto out;
|
2017-07-06 20:44:28 +08:00
|
|
|
oldpid = rcu_access_pointer(vcpu->pid);
|
2017-07-17 10:39:32 +08:00
|
|
|
if (unlikely(oldpid != task_pid(current))) {
|
2014-08-05 22:44:14 +08:00
|
|
|
/* The thread running this VCPU changed. */
|
2018-02-24 00:23:57 +08:00
|
|
|
struct pid *newpid;
|
2015-02-26 14:58:23 +08:00
|
|
|
|
2018-02-24 00:23:57 +08:00
|
|
|
r = kvm_arch_vcpu_run_pid_change(vcpu);
|
|
|
|
if (r)
|
|
|
|
break;
|
|
|
|
|
|
|
|
newpid = get_task_pid(current, PIDTYPE_PID);
|
2014-08-05 22:44:14 +08:00
|
|
|
rcu_assign_pointer(vcpu->pid, newpid);
|
|
|
|
if (oldpid)
|
|
|
|
synchronize_rcu();
|
|
|
|
put_pid(oldpid);
|
|
|
|
}
|
2020-04-16 13:10:57 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_run(vcpu);
|
2010-10-24 22:49:08 +08:00
|
|
|
trace_kvm_userspace_exit(vcpu->run->exit_reason, r);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
2017-07-06 20:44:28 +08:00
|
|
|
}
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
case KVM_GET_REGS: {
|
2008-02-25 18:52:20 +08:00
|
|
|
struct kvm_regs *kvm_regs;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2008-02-25 18:52:20 +08:00
|
|
|
r = -ENOMEM;
|
2019-02-12 03:02:49 +08:00
|
|
|
kvm_regs = kzalloc(sizeof(struct kvm_regs), GFP_KERNEL_ACCOUNT);
|
2008-02-25 18:52:20 +08:00
|
|
|
if (!kvm_regs)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2008-02-25 18:52:20 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_get_regs(vcpu, kvm_regs);
|
|
|
|
if (r)
|
|
|
|
goto out_free1;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
r = -EFAULT;
|
2008-02-25 18:52:20 +08:00
|
|
|
if (copy_to_user(argp, kvm_regs, sizeof(struct kvm_regs)))
|
|
|
|
goto out_free1;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
r = 0;
|
2008-02-25 18:52:20 +08:00
|
|
|
out_free1:
|
|
|
|
kfree(kvm_regs);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_SET_REGS: {
|
2008-02-25 18:52:20 +08:00
|
|
|
struct kvm_regs *kvm_regs;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2011-12-05 01:36:29 +08:00
|
|
|
kvm_regs = memdup_user(argp, sizeof(*kvm_regs));
|
|
|
|
if (IS_ERR(kvm_regs)) {
|
|
|
|
r = PTR_ERR(kvm_regs);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2011-12-05 01:36:29 +08:00
|
|
|
}
|
2008-02-25 18:52:20 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
|
|
|
|
kfree(kvm_regs);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_GET_SREGS: {
|
2019-02-12 03:02:49 +08:00
|
|
|
kvm_sregs = kzalloc(sizeof(struct kvm_sregs),
|
|
|
|
GFP_KERNEL_ACCOUNT);
|
2008-08-12 01:01:46 +08:00
|
|
|
r = -ENOMEM;
|
|
|
|
if (!kvm_sregs)
|
|
|
|
goto out;
|
|
|
|
r = kvm_arch_vcpu_ioctl_get_sregs(vcpu, kvm_sregs);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
2008-08-12 01:01:46 +08:00
|
|
|
if (copy_to_user(argp, kvm_sregs, sizeof(struct kvm_sregs)))
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_SET_SREGS: {
|
2011-12-05 01:36:29 +08:00
|
|
|
kvm_sregs = memdup_user(argp, sizeof(*kvm_sregs));
|
|
|
|
if (IS_ERR(kvm_sregs)) {
|
|
|
|
r = PTR_ERR(kvm_sregs);
|
2012-11-02 18:33:21 +08:00
|
|
|
kvm_sregs = NULL;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2011-12-05 01:36:29 +08:00
|
|
|
}
|
2008-08-12 01:01:46 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_set_sregs(vcpu, kvm_sregs);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
2008-04-12 00:24:45 +08:00
|
|
|
case KVM_GET_MP_STATE: {
|
|
|
|
struct kvm_mp_state mp_state;
|
|
|
|
|
|
|
|
r = kvm_arch_vcpu_ioctl_get_mpstate(vcpu, &mp_state);
|
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_to_user(argp, &mp_state, sizeof(mp_state)))
|
2008-04-12 00:24:45 +08:00
|
|
|
goto out;
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_SET_MP_STATE: {
|
|
|
|
struct kvm_mp_state mp_state;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&mp_state, argp, sizeof(mp_state)))
|
2008-04-12 00:24:45 +08:00
|
|
|
goto out;
|
|
|
|
r = kvm_arch_vcpu_ioctl_set_mpstate(vcpu, &mp_state);
|
|
|
|
break;
|
|
|
|
}
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
case KVM_TRANSLATE: {
|
|
|
|
struct kvm_translation tr;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&tr, argp, sizeof(tr)))
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2007-11-16 13:05:55 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_translate(vcpu, &tr);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_to_user(argp, &tr, sizeof(tr)))
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
2008-12-15 20:52:10 +08:00
|
|
|
case KVM_SET_GUEST_DEBUG: {
|
|
|
|
struct kvm_guest_debug dbg;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&dbg, argp, sizeof(dbg)))
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2008-12-15 20:52:10 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_set_guest_debug(vcpu, &dbg);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
2007-03-06 01:46:05 +08:00
|
|
|
case KVM_SET_SIGNAL_MASK: {
|
|
|
|
struct kvm_signal_mask __user *sigmask_arg = argp;
|
|
|
|
struct kvm_signal_mask kvm_sigmask;
|
|
|
|
sigset_t sigset, *p;
|
|
|
|
|
|
|
|
p = NULL;
|
|
|
|
if (argp) {
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&kvm_sigmask, argp,
|
2015-02-26 14:58:19 +08:00
|
|
|
sizeof(kvm_sigmask)))
|
2007-03-06 01:46:05 +08:00
|
|
|
goto out;
|
|
|
|
r = -EINVAL;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (kvm_sigmask.len != sizeof(sigset))
|
2007-03-06 01:46:05 +08:00
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&sigset, sigmask_arg->sigset,
|
2015-02-26 14:58:19 +08:00
|
|
|
sizeof(sigset)))
|
2007-03-06 01:46:05 +08:00
|
|
|
goto out;
|
|
|
|
p = &sigset;
|
|
|
|
}
|
2010-06-10 19:10:47 +08:00
|
|
|
r = kvm_vcpu_ioctl_set_sigmask(vcpu, p);
|
2007-03-06 01:46:05 +08:00
|
|
|
break;
|
|
|
|
}
|
2007-04-01 21:34:31 +08:00
|
|
|
case KVM_GET_FPU: {
|
2019-02-12 03:02:49 +08:00
|
|
|
fpu = kzalloc(sizeof(struct kvm_fpu), GFP_KERNEL_ACCOUNT);
|
2008-08-12 01:01:46 +08:00
|
|
|
r = -ENOMEM;
|
|
|
|
if (!fpu)
|
|
|
|
goto out;
|
|
|
|
r = kvm_arch_vcpu_ioctl_get_fpu(vcpu, fpu);
|
2007-04-01 21:34:31 +08:00
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
2008-08-12 01:01:46 +08:00
|
|
|
if (copy_to_user(argp, fpu, sizeof(struct kvm_fpu)))
|
2007-04-01 21:34:31 +08:00
|
|
|
goto out;
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_SET_FPU: {
|
2011-12-05 01:36:29 +08:00
|
|
|
fpu = memdup_user(argp, sizeof(*fpu));
|
|
|
|
if (IS_ERR(fpu)) {
|
|
|
|
r = PTR_ERR(fpu);
|
2012-11-02 18:33:21 +08:00
|
|
|
fpu = NULL;
|
2007-04-01 21:34:31 +08:00
|
|
|
goto out;
|
2011-12-05 01:36:29 +08:00
|
|
|
}
|
2008-08-12 01:01:46 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl_set_fpu(vcpu, fpu);
|
2007-04-01 21:34:31 +08:00
|
|
|
break;
|
|
|
|
}
|
2021-06-19 06:27:06 +08:00
|
|
|
case KVM_GET_STATS_FD: {
|
|
|
|
r = kvm_vcpu_ioctl_get_stats_fd(vcpu);
|
|
|
|
break;
|
|
|
|
}
|
2007-02-22 00:04:26 +08:00
|
|
|
default:
|
KVM: Portability: split kvm_vcpu_ioctl
This patch splits kvm_vcpu_ioctl into archtecture independent parts, and
x86 specific parts which go to kvm_arch_vcpu_ioctl in x86.c.
Common ioctls for all architectures are:
KVM_RUN, KVM_GET/SET_(S-)REGS, KVM_TRANSLATE, KVM_INTERRUPT,
KVM_DEBUG_GUEST, KVM_SET_SIGNAL_MASK, KVM_GET/SET_FPU
Note that some PPC chips don't have an FPU, so we might need an #ifdef
around KVM_GET/SET_FPU one day.
x86 specific ioctls are:
KVM_GET/SET_LAPIC, KVM_SET_CPUID, KVM_GET/SET_MSRS
An interresting aspect is vcpu_load/vcpu_put. We now have a common
vcpu_load/put which does the preemption stuff, and an architecture
specific kvm_arch_vcpu_load/put. In the x86 case, this one calls the
vmx/svm function defined in kvm_x86_ops.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-10-12 01:16:52 +08:00
|
|
|
r = kvm_arch_vcpu_ioctl(filp, ioctl, arg);
|
2007-02-22 00:04:26 +08:00
|
|
|
}
|
|
|
|
out:
|
2017-12-05 04:35:23 +08:00
|
|
|
mutex_unlock(&vcpu->mutex);
|
2008-08-12 01:01:46 +08:00
|
|
|
kfree(fpu);
|
|
|
|
kfree(kvm_sregs);
|
2007-02-22 00:04:26 +08:00
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2015-02-03 16:35:15 +08:00
|
|
|
#ifdef CONFIG_KVM_COMPAT
|
2011-06-08 08:45:37 +08:00
|
|
|
static long kvm_vcpu_compat_ioctl(struct file *filp,
|
|
|
|
unsigned int ioctl, unsigned long arg)
|
|
|
|
{
|
|
|
|
struct kvm_vcpu *vcpu = filp->private_data;
|
|
|
|
void __user *argp = compat_ptr(arg);
|
|
|
|
int r;
|
|
|
|
|
2021-07-03 06:04:23 +08:00
|
|
|
if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)
|
2011-06-08 08:45:37 +08:00
|
|
|
return -EIO;
|
|
|
|
|
|
|
|
switch (ioctl) {
|
|
|
|
case KVM_SET_SIGNAL_MASK: {
|
|
|
|
struct kvm_signal_mask __user *sigmask_arg = argp;
|
|
|
|
struct kvm_signal_mask kvm_sigmask;
|
|
|
|
sigset_t sigset;
|
|
|
|
|
|
|
|
if (argp) {
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&kvm_sigmask, argp,
|
2015-02-26 14:58:19 +08:00
|
|
|
sizeof(kvm_sigmask)))
|
2011-06-08 08:45:37 +08:00
|
|
|
goto out;
|
|
|
|
r = -EINVAL;
|
2017-09-04 09:45:17 +08:00
|
|
|
if (kvm_sigmask.len != sizeof(compat_sigset_t))
|
2011-06-08 08:45:37 +08:00
|
|
|
goto out;
|
|
|
|
r = -EFAULT;
|
2020-07-02 17:39:31 +08:00
|
|
|
if (get_compat_sigset(&sigset,
|
|
|
|
(compat_sigset_t __user *)sigmask_arg->sigset))
|
2011-06-08 08:45:37 +08:00
|
|
|
goto out;
|
2012-08-22 21:34:11 +08:00
|
|
|
r = kvm_vcpu_ioctl_set_sigmask(vcpu, &sigset);
|
|
|
|
} else
|
|
|
|
r = kvm_vcpu_ioctl_set_sigmask(vcpu, NULL);
|
2011-06-08 08:45:37 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
default:
|
|
|
|
r = kvm_vcpu_ioctl(filp, ioctl, arg);
|
|
|
|
}
|
|
|
|
|
|
|
|
out:
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2019-04-18 18:39:36 +08:00
|
|
|
static int kvm_device_mmap(struct file *filp, struct vm_area_struct *vma)
|
|
|
|
{
|
|
|
|
struct kvm_device *dev = filp->private_data;
|
|
|
|
|
|
|
|
if (dev->ops->mmap)
|
|
|
|
return dev->ops->mmap(dev, vma);
|
|
|
|
|
|
|
|
return -ENODEV;
|
|
|
|
}
|
|
|
|
|
2013-04-12 22:08:42 +08:00
|
|
|
static int kvm_device_ioctl_attr(struct kvm_device *dev,
|
|
|
|
int (*accessor)(struct kvm_device *dev,
|
|
|
|
struct kvm_device_attr *attr),
|
|
|
|
unsigned long arg)
|
|
|
|
{
|
|
|
|
struct kvm_device_attr attr;
|
|
|
|
|
|
|
|
if (!accessor)
|
|
|
|
return -EPERM;
|
|
|
|
|
|
|
|
if (copy_from_user(&attr, (void __user *)arg, sizeof(attr)))
|
|
|
|
return -EFAULT;
|
|
|
|
|
|
|
|
return accessor(dev, &attr);
|
|
|
|
}
|
|
|
|
|
|
|
|
static long kvm_device_ioctl(struct file *filp, unsigned int ioctl,
|
|
|
|
unsigned long arg)
|
|
|
|
{
|
|
|
|
struct kvm_device *dev = filp->private_data;
|
|
|
|
|
2021-07-03 06:04:23 +08:00
|
|
|
if (dev->kvm->mm != current->mm || dev->kvm->vm_bugged)
|
2019-02-16 04:48:39 +08:00
|
|
|
return -EIO;
|
|
|
|
|
2013-04-12 22:08:42 +08:00
|
|
|
switch (ioctl) {
|
|
|
|
case KVM_SET_DEVICE_ATTR:
|
|
|
|
return kvm_device_ioctl_attr(dev, dev->ops->set_attr, arg);
|
|
|
|
case KVM_GET_DEVICE_ATTR:
|
|
|
|
return kvm_device_ioctl_attr(dev, dev->ops->get_attr, arg);
|
|
|
|
case KVM_HAS_DEVICE_ATTR:
|
|
|
|
return kvm_device_ioctl_attr(dev, dev->ops->has_attr, arg);
|
|
|
|
default:
|
|
|
|
if (dev->ops->ioctl)
|
|
|
|
return dev->ops->ioctl(dev, ioctl, arg);
|
|
|
|
|
|
|
|
return -ENOTTY;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_device_release(struct inode *inode, struct file *filp)
|
|
|
|
{
|
|
|
|
struct kvm_device *dev = filp->private_data;
|
|
|
|
struct kvm *kvm = dev->kvm;
|
|
|
|
|
2019-04-18 18:39:41 +08:00
|
|
|
if (dev->ops->release) {
|
|
|
|
mutex_lock(&kvm->lock);
|
|
|
|
list_del(&dev->vm_node);
|
|
|
|
dev->ops->release(dev);
|
|
|
|
mutex_unlock(&kvm->lock);
|
|
|
|
}
|
|
|
|
|
2013-04-12 22:08:42 +08:00
|
|
|
kvm_put_kvm(kvm);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static const struct file_operations kvm_device_fops = {
|
|
|
|
.unlocked_ioctl = kvm_device_ioctl,
|
|
|
|
.release = kvm_device_release,
|
2018-06-17 17:16:21 +08:00
|
|
|
KVM_COMPAT(kvm_device_ioctl),
|
2019-04-18 18:39:36 +08:00
|
|
|
.mmap = kvm_device_mmap,
|
2013-04-12 22:08:42 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
struct kvm_device *kvm_device_from_filp(struct file *filp)
|
|
|
|
{
|
|
|
|
if (filp->f_op != &kvm_device_fops)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
return filp->private_data;
|
|
|
|
}
|
|
|
|
|
2019-10-21 23:28:19 +08:00
|
|
|
static const struct kvm_device_ops *kvm_device_ops_table[KVM_DEV_TYPE_MAX] = {
|
2013-04-12 22:08:46 +08:00
|
|
|
#ifdef CONFIG_KVM_MPIC
|
2014-09-02 17:27:33 +08:00
|
|
|
[KVM_DEV_TYPE_FSL_MPIC_20] = &kvm_mpic_ops,
|
|
|
|
[KVM_DEV_TYPE_FSL_MPIC_42] = &kvm_mpic_ops,
|
2013-04-27 08:28:37 +08:00
|
|
|
#endif
|
2014-09-02 17:27:33 +08:00
|
|
|
};
|
|
|
|
|
2019-10-21 23:28:19 +08:00
|
|
|
int kvm_register_device_ops(const struct kvm_device_ops *ops, u32 type)
|
2014-09-02 17:27:33 +08:00
|
|
|
{
|
|
|
|
if (type >= ARRAY_SIZE(kvm_device_ops_table))
|
|
|
|
return -ENOSPC;
|
|
|
|
|
|
|
|
if (kvm_device_ops_table[type] != NULL)
|
|
|
|
return -EEXIST;
|
|
|
|
|
|
|
|
kvm_device_ops_table[type] = ops;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2014-10-09 18:30:08 +08:00
|
|
|
void kvm_unregister_device_ops(u32 type)
|
|
|
|
{
|
|
|
|
if (kvm_device_ops_table[type] != NULL)
|
|
|
|
kvm_device_ops_table[type] = NULL;
|
|
|
|
}
|
|
|
|
|
2013-04-12 22:08:42 +08:00
|
|
|
static int kvm_ioctl_create_device(struct kvm *kvm,
|
|
|
|
struct kvm_create_device *cd)
|
|
|
|
{
|
2019-10-21 23:28:19 +08:00
|
|
|
const struct kvm_device_ops *ops = NULL;
|
2013-04-12 22:08:42 +08:00
|
|
|
struct kvm_device *dev;
|
|
|
|
bool test = cd->flags & KVM_CREATE_DEVICE_TEST;
|
2019-04-11 17:16:47 +08:00
|
|
|
int type;
|
2013-04-12 22:08:42 +08:00
|
|
|
int ret;
|
|
|
|
|
2014-09-02 17:27:33 +08:00
|
|
|
if (cd->type >= ARRAY_SIZE(kvm_device_ops_table))
|
|
|
|
return -ENODEV;
|
|
|
|
|
2019-04-11 17:16:47 +08:00
|
|
|
type = array_index_nospec(cd->type, ARRAY_SIZE(kvm_device_ops_table));
|
|
|
|
ops = kvm_device_ops_table[type];
|
2014-09-02 17:27:33 +08:00
|
|
|
if (ops == NULL)
|
2013-04-12 22:08:42 +08:00
|
|
|
return -ENODEV;
|
|
|
|
|
|
|
|
if (test)
|
|
|
|
return 0;
|
|
|
|
|
2019-02-12 03:02:49 +08:00
|
|
|
dev = kzalloc(sizeof(*dev), GFP_KERNEL_ACCOUNT);
|
2013-04-12 22:08:42 +08:00
|
|
|
if (!dev)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
dev->ops = ops;
|
|
|
|
dev->kvm = kvm;
|
|
|
|
|
2016-08-10 01:13:01 +08:00
|
|
|
mutex_lock(&kvm->lock);
|
2019-04-11 17:16:47 +08:00
|
|
|
ret = ops->create(dev, type);
|
2013-04-12 22:08:42 +08:00
|
|
|
if (ret < 0) {
|
2016-08-10 01:13:01 +08:00
|
|
|
mutex_unlock(&kvm->lock);
|
2013-04-12 22:08:42 +08:00
|
|
|
kfree(dev);
|
|
|
|
return ret;
|
|
|
|
}
|
2016-08-10 01:13:01 +08:00
|
|
|
list_add(&dev->vm_node, &kvm->devices);
|
|
|
|
mutex_unlock(&kvm->lock);
|
2013-04-12 22:08:42 +08:00
|
|
|
|
2016-08-10 01:13:00 +08:00
|
|
|
if (ops->init)
|
|
|
|
ops->init(dev);
|
|
|
|
|
2019-01-26 08:54:33 +08:00
|
|
|
kvm_get_kvm(kvm);
|
2013-08-25 04:14:07 +08:00
|
|
|
ret = anon_inode_getfd(ops->name, &kvm_device_fops, dev, O_RDWR | O_CLOEXEC);
|
2013-04-12 22:08:42 +08:00
|
|
|
if (ret < 0) {
|
2019-10-22 06:58:42 +08:00
|
|
|
kvm_put_kvm_no_destroy(kvm);
|
2016-08-10 01:13:01 +08:00
|
|
|
mutex_lock(&kvm->lock);
|
|
|
|
list_del(&dev->vm_node);
|
|
|
|
mutex_unlock(&kvm->lock);
|
2016-12-01 03:21:05 +08:00
|
|
|
ops->destroy(dev);
|
2013-04-12 22:08:42 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
cd->fd = ret;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2014-07-15 00:33:08 +08:00
|
|
|
static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
|
|
|
|
{
|
|
|
|
switch (arg) {
|
|
|
|
case KVM_CAP_USER_MEMORY:
|
|
|
|
case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
|
|
|
|
case KVM_CAP_JOIN_MEMORY_REGIONS_WORKS:
|
|
|
|
case KVM_CAP_INTERNAL_ERROR_DATA:
|
|
|
|
#ifdef CONFIG_HAVE_KVM_MSI
|
|
|
|
case KVM_CAP_SIGNAL_MSI:
|
|
|
|
#endif
|
2014-06-30 18:51:13 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_IRQFD
|
2015-03-05 18:54:46 +08:00
|
|
|
case KVM_CAP_IRQFD:
|
2014-07-15 00:33:08 +08:00
|
|
|
case KVM_CAP_IRQFD_RESAMPLE:
|
|
|
|
#endif
|
2015-09-15 14:41:59 +08:00
|
|
|
case KVM_CAP_IOEVENTFD_ANY_LENGTH:
|
2014-07-15 00:33:08 +08:00
|
|
|
case KVM_CAP_CHECK_EXTENSION_VM:
|
2017-02-16 17:40:56 +08:00
|
|
|
case KVM_CAP_ENABLE_CAP_VM:
|
2020-04-18 06:14:46 +08:00
|
|
|
case KVM_CAP_HALT_POLL:
|
2014-07-15 00:33:08 +08:00
|
|
|
return 1;
|
2017-03-31 19:53:23 +08:00
|
|
|
#ifdef CONFIG_KVM_MMIO
|
2017-03-31 19:53:22 +08:00
|
|
|
case KVM_CAP_COALESCED_MMIO:
|
|
|
|
return KVM_COALESCED_MMIO_PAGE_OFFSET;
|
2018-10-14 07:09:55 +08:00
|
|
|
case KVM_CAP_COALESCED_PIO:
|
|
|
|
return 1;
|
2017-03-31 19:53:22 +08:00
|
|
|
#endif
|
2020-02-27 09:32:27 +08:00
|
|
|
#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT
|
|
|
|
case KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2:
|
|
|
|
return KVM_DIRTY_LOG_MANUAL_CAPS;
|
|
|
|
#endif
|
2014-07-15 00:33:08 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_IRQ_ROUTING
|
|
|
|
case KVM_CAP_IRQ_ROUTING:
|
|
|
|
return KVM_MAX_IRQ_ROUTES;
|
2015-05-17 23:30:37 +08:00
|
|
|
#endif
|
|
|
|
#if KVM_ADDRESS_SPACE_NUM > 1
|
|
|
|
case KVM_CAP_MULTI_ADDRESS_SPACE:
|
|
|
|
return KVM_ADDRESS_SPACE_NUM;
|
2014-07-15 00:33:08 +08:00
|
|
|
#endif
|
2019-03-29 00:24:03 +08:00
|
|
|
case KVM_CAP_NR_MEMSLOTS:
|
|
|
|
return KVM_USER_MEM_SLOTS;
|
2020-10-01 09:22:22 +08:00
|
|
|
case KVM_CAP_DIRTY_LOG_RING:
|
|
|
|
#if KVM_DIRTY_LOG_PAGE_OFFSET > 0
|
|
|
|
return KVM_DIRTY_RING_MAX_ENTRIES * sizeof(struct kvm_dirty_gfn);
|
|
|
|
#else
|
|
|
|
return 0;
|
|
|
|
#endif
|
2021-06-19 06:27:06 +08:00
|
|
|
case KVM_CAP_BINARY_STATS_FD:
|
|
|
|
return 1;
|
2014-07-15 00:33:08 +08:00
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
return kvm_vm_ioctl_check_extension(kvm, arg);
|
|
|
|
}
|
|
|
|
|
2020-10-01 09:22:22 +08:00
|
|
|
static int kvm_vm_ioctl_enable_dirty_log_ring(struct kvm *kvm, u32 size)
|
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
|
|
|
if (!KVM_DIRTY_LOG_PAGE_OFFSET)
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
/* the size should be power of 2 */
|
|
|
|
if (!size || (size & (size - 1)))
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
/* Should be bigger to keep the reserved entries, or a page */
|
|
|
|
if (size < kvm_dirty_ring_get_rsvd_entries() *
|
|
|
|
sizeof(struct kvm_dirty_gfn) || size < PAGE_SIZE)
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
if (size > KVM_DIRTY_RING_MAX_ENTRIES *
|
|
|
|
sizeof(struct kvm_dirty_gfn))
|
|
|
|
return -E2BIG;
|
|
|
|
|
|
|
|
/* We only allow it to set once */
|
|
|
|
if (kvm->dirty_ring_size)
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
mutex_lock(&kvm->lock);
|
|
|
|
|
|
|
|
if (kvm->created_vcpus) {
|
|
|
|
/* We don't allow to change this value after vcpu created */
|
|
|
|
r = -EINVAL;
|
|
|
|
} else {
|
|
|
|
kvm->dirty_ring_size = size;
|
|
|
|
r = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
mutex_unlock(&kvm->lock);
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_vm_ioctl_reset_dirty_pages(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
struct kvm_vcpu *vcpu;
|
|
|
|
int cleared = 0;
|
|
|
|
|
|
|
|
if (!kvm->dirty_ring_size)
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
mutex_lock(&kvm->slots_lock);
|
|
|
|
|
|
|
|
kvm_for_each_vcpu(i, vcpu, kvm)
|
|
|
|
cleared += kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring);
|
|
|
|
|
|
|
|
mutex_unlock(&kvm->slots_lock);
|
|
|
|
|
|
|
|
if (cleared)
|
|
|
|
kvm_flush_remote_tlbs(kvm);
|
|
|
|
|
|
|
|
return cleared;
|
|
|
|
}
|
|
|
|
|
2017-02-16 17:40:56 +08:00
|
|
|
int __attribute__((weak)) kvm_vm_ioctl_enable_cap(struct kvm *kvm,
|
|
|
|
struct kvm_enable_cap *cap)
|
|
|
|
{
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm,
|
|
|
|
struct kvm_enable_cap *cap)
|
|
|
|
{
|
|
|
|
switch (cap->cap) {
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT
|
2020-02-27 09:32:27 +08:00
|
|
|
case KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2: {
|
|
|
|
u64 allowed_options = KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE;
|
|
|
|
|
|
|
|
if (cap->args[0] & KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE)
|
|
|
|
allowed_options = KVM_DIRTY_LOG_MANUAL_CAPS;
|
|
|
|
|
|
|
|
if (cap->flags || (cap->args[0] & ~allowed_options))
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
return -EINVAL;
|
|
|
|
kvm->manual_dirty_log_protect = cap->args[0];
|
|
|
|
return 0;
|
2020-02-27 09:32:27 +08:00
|
|
|
}
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
#endif
|
2020-04-18 06:14:46 +08:00
|
|
|
case KVM_CAP_HALT_POLL: {
|
|
|
|
if (cap->flags || cap->args[0] != (unsigned int)cap->args[0])
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
kvm->max_halt_poll_ns = cap->args[0];
|
|
|
|
return 0;
|
|
|
|
}
|
2020-10-01 09:22:22 +08:00
|
|
|
case KVM_CAP_DIRTY_LOG_RING:
|
|
|
|
return kvm_vm_ioctl_enable_dirty_log_ring(kvm, cap->args[0]);
|
2017-02-16 17:40:56 +08:00
|
|
|
default:
|
|
|
|
return kvm_vm_ioctl_enable_cap(kvm, cap);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-06-19 06:27:05 +08:00
|
|
|
static ssize_t kvm_vm_stats_read(struct file *file, char __user *user_buffer,
|
|
|
|
size_t size, loff_t *offset)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = file->private_data;
|
|
|
|
|
|
|
|
return kvm_stats_read(kvm->stats_id, &kvm_vm_stats_header,
|
|
|
|
&kvm_vm_stats_desc[0], &kvm->stat,
|
|
|
|
sizeof(kvm->stat), user_buffer, size, offset);
|
|
|
|
}
|
|
|
|
|
|
|
|
static const struct file_operations kvm_vm_stats_fops = {
|
|
|
|
.read = kvm_vm_stats_read,
|
|
|
|
.llseek = noop_llseek,
|
|
|
|
};
|
|
|
|
|
|
|
|
static int kvm_vm_ioctl_get_stats_fd(struct kvm *kvm)
|
|
|
|
{
|
|
|
|
int fd;
|
|
|
|
struct file *file;
|
|
|
|
|
|
|
|
fd = get_unused_fd_flags(O_CLOEXEC);
|
|
|
|
if (fd < 0)
|
|
|
|
return fd;
|
|
|
|
|
|
|
|
file = anon_inode_getfile("kvm-vm-stats",
|
|
|
|
&kvm_vm_stats_fops, kvm, O_RDONLY);
|
|
|
|
if (IS_ERR(file)) {
|
|
|
|
put_unused_fd(fd);
|
|
|
|
return PTR_ERR(file);
|
|
|
|
}
|
|
|
|
file->f_mode |= FMODE_PREAD;
|
|
|
|
fd_install(fd, file);
|
|
|
|
|
|
|
|
return fd;
|
|
|
|
}
|
|
|
|
|
2007-02-22 00:04:26 +08:00
|
|
|
static long kvm_vm_ioctl(struct file *filp,
|
|
|
|
unsigned int ioctl, unsigned long arg)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = filp->private_data;
|
|
|
|
void __user *argp = (void __user *)arg;
|
2007-10-29 23:08:35 +08:00
|
|
|
int r;
|
2007-02-22 00:04:26 +08:00
|
|
|
|
2021-07-03 06:04:23 +08:00
|
|
|
if (kvm->mm != current->mm || kvm->vm_bugged)
|
2007-11-21 22:41:05 +08:00
|
|
|
return -EIO;
|
2007-02-22 00:04:26 +08:00
|
|
|
switch (ioctl) {
|
|
|
|
case KVM_CREATE_VCPU:
|
|
|
|
r = kvm_vm_ioctl_create_vcpu(kvm, arg);
|
|
|
|
break;
|
2017-02-16 17:40:56 +08:00
|
|
|
case KVM_ENABLE_CAP: {
|
|
|
|
struct kvm_enable_cap cap;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&cap, argp, sizeof(cap)))
|
|
|
|
goto out;
|
|
|
|
r = kvm_vm_ioctl_enable_cap_generic(kvm, &cap);
|
|
|
|
break;
|
|
|
|
}
|
2007-10-10 01:20:39 +08:00
|
|
|
case KVM_SET_USER_MEMORY_REGION: {
|
|
|
|
struct kvm_userspace_memory_region kvm_userspace_mem;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&kvm_userspace_mem, argp,
|
2015-02-26 14:58:19 +08:00
|
|
|
sizeof(kvm_userspace_mem)))
|
2007-10-10 01:20:39 +08:00
|
|
|
goto out;
|
|
|
|
|
2013-02-27 18:43:00 +08:00
|
|
|
r = kvm_vm_ioctl_set_memory_region(kvm, &kvm_userspace_mem);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_GET_DIRTY_LOG: {
|
|
|
|
struct kvm_dirty_log log;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&log, argp, sizeof(log)))
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
goto out;
|
2007-02-21 00:27:58 +08:00
|
|
|
r = kvm_vm_ioctl_get_dirty_log(kvm, &log);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
break;
|
|
|
|
}
|
kvm: introduce manual dirty log reprotect
There are two problems with KVM_GET_DIRTY_LOG. First, and less important,
it can take kvm->mmu_lock for an extended period of time. Second, its user
can actually see many false positives in some cases. The latter is due
to a benign race like this:
1. KVM_GET_DIRTY_LOG returns a set of dirty pages and write protects
them.
2. The guest modifies the pages, causing them to be marked ditry.
3. Userspace actually copies the pages.
4. KVM_GET_DIRTY_LOG returns those pages as dirty again, even though
they were not written to since (3).
This is especially a problem for large guests, where the time between
(1) and (3) can be substantial. This patch introduces a new
capability which, when enabled, makes KVM_GET_DIRTY_LOG not
write-protect the pages it returns. Instead, userspace has to
explicitly clear the dirty log bits just before using the content
of the page. The new KVM_CLEAR_DIRTY_LOG ioctl can also operate on a
64-page granularity rather than requiring to sync a full memslot;
this way, the mmu_lock is taken for small amounts of time, and
only a small amount of time will pass between write protection
of pages and the sending of their content.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-23 08:36:47 +08:00
|
|
|
#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT
|
|
|
|
case KVM_CLEAR_DIRTY_LOG: {
|
|
|
|
struct kvm_clear_dirty_log log;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&log, argp, sizeof(log)))
|
|
|
|
goto out;
|
|
|
|
r = kvm_vm_ioctl_clear_dirty_log(kvm, &log);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
2017-03-31 19:53:23 +08:00
|
|
|
#ifdef CONFIG_KVM_MMIO
|
2008-05-30 22:05:54 +08:00
|
|
|
case KVM_REGISTER_COALESCED_MMIO: {
|
|
|
|
struct kvm_coalesced_mmio_zone zone;
|
2015-02-26 14:58:23 +08:00
|
|
|
|
2008-05-30 22:05:54 +08:00
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&zone, argp, sizeof(zone)))
|
2008-05-30 22:05:54 +08:00
|
|
|
goto out;
|
|
|
|
r = kvm_vm_ioctl_register_coalesced_mmio(kvm, &zone);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
case KVM_UNREGISTER_COALESCED_MMIO: {
|
|
|
|
struct kvm_coalesced_mmio_zone zone;
|
2015-02-26 14:58:23 +08:00
|
|
|
|
2008-05-30 22:05:54 +08:00
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&zone, argp, sizeof(zone)))
|
2008-05-30 22:05:54 +08:00
|
|
|
goto out;
|
|
|
|
r = kvm_vm_ioctl_unregister_coalesced_mmio(kvm, &zone);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
2009-05-20 22:30:49 +08:00
|
|
|
case KVM_IRQFD: {
|
|
|
|
struct kvm_irqfd data;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&data, argp, sizeof(data)))
|
2009-05-20 22:30:49 +08:00
|
|
|
goto out;
|
2012-06-29 23:56:08 +08:00
|
|
|
r = kvm_irqfd(kvm, &data);
|
2009-05-20 22:30:49 +08:00
|
|
|
break;
|
|
|
|
}
|
KVM: add ioeventfd support
ioeventfd is a mechanism to register PIO/MMIO regions to trigger an eventfd
signal when written to by a guest. Host userspace can register any
arbitrary IO address with a corresponding eventfd and then pass the eventfd
to a specific end-point of interest for handling.
Normal IO requires a blocking round-trip since the operation may cause
side-effects in the emulated model or may return data to the caller.
Therefore, an IO in KVM traps from the guest to the host, causes a VMX/SVM
"heavy-weight" exit back to userspace, and is ultimately serviced by qemu's
device model synchronously before returning control back to the vcpu.
However, there is a subclass of IO which acts purely as a trigger for
other IO (such as to kick off an out-of-band DMA request, etc). For these
patterns, the synchronous call is particularly expensive since we really
only want to simply get our notification transmitted asychronously and
return as quickly as possible. All the sychronous infrastructure to ensure
proper data-dependencies are met in the normal IO case are just unecessary
overhead for signalling. This adds additional computational load on the
system, as well as latency to the signalling path.
Therefore, we provide a mechanism for registration of an in-kernel trigger
point that allows the VCPU to only require a very brief, lightweight
exit just long enough to signal an eventfd. This also means that any
clients compatible with the eventfd interface (which includes userspace
and kernelspace equally well) can now register to be notified. The end
result should be a more flexible and higher performance notification API
for the backend KVM hypervisor and perhipheral components.
To test this theory, we built a test-harness called "doorbell". This
module has a function called "doorbell_ring()" which simply increments a
counter for each time the doorbell is signaled. It supports signalling
from either an eventfd, or an ioctl().
We then wired up two paths to the doorbell: One via QEMU via a registered
io region and through the doorbell ioctl(). The other is direct via
ioeventfd.
You can download this test harness here:
ftp://ftp.novell.com/dev/ghaskins/doorbell.tar.bz2
The measured results are as follows:
qemu-mmio: 110000 iops, 9.09us rtt
ioeventfd-mmio: 200100 iops, 5.00us rtt
ioeventfd-pio: 367300 iops, 2.72us rtt
I didn't measure qemu-pio, because I have to figure out how to register a
PIO region with qemu's device model, and I got lazy. However, for now we
can extrapolate based on the data from the NULLIO runs of +2.56us for MMIO,
and -350ns for HC, we get:
qemu-pio: 153139 iops, 6.53us rtt
ioeventfd-hc: 412585 iops, 2.37us rtt
these are just for fun, for now, until I can gather more data.
Here is a graph for your convenience:
http://developer.novell.com/wiki/images/7/76/Iofd-chart.png
The conclusion to draw is that we save about 4us by skipping the userspace
hop.
--------------------
Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Avi Kivity <avi@redhat.com>
2009-07-08 05:08:49 +08:00
|
|
|
case KVM_IOEVENTFD: {
|
|
|
|
struct kvm_ioeventfd data;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&data, argp, sizeof(data)))
|
KVM: add ioeventfd support
ioeventfd is a mechanism to register PIO/MMIO regions to trigger an eventfd
signal when written to by a guest. Host userspace can register any
arbitrary IO address with a corresponding eventfd and then pass the eventfd
to a specific end-point of interest for handling.
Normal IO requires a blocking round-trip since the operation may cause
side-effects in the emulated model or may return data to the caller.
Therefore, an IO in KVM traps from the guest to the host, causes a VMX/SVM
"heavy-weight" exit back to userspace, and is ultimately serviced by qemu's
device model synchronously before returning control back to the vcpu.
However, there is a subclass of IO which acts purely as a trigger for
other IO (such as to kick off an out-of-band DMA request, etc). For these
patterns, the synchronous call is particularly expensive since we really
only want to simply get our notification transmitted asychronously and
return as quickly as possible. All the sychronous infrastructure to ensure
proper data-dependencies are met in the normal IO case are just unecessary
overhead for signalling. This adds additional computational load on the
system, as well as latency to the signalling path.
Therefore, we provide a mechanism for registration of an in-kernel trigger
point that allows the VCPU to only require a very brief, lightweight
exit just long enough to signal an eventfd. This also means that any
clients compatible with the eventfd interface (which includes userspace
and kernelspace equally well) can now register to be notified. The end
result should be a more flexible and higher performance notification API
for the backend KVM hypervisor and perhipheral components.
To test this theory, we built a test-harness called "doorbell". This
module has a function called "doorbell_ring()" which simply increments a
counter for each time the doorbell is signaled. It supports signalling
from either an eventfd, or an ioctl().
We then wired up two paths to the doorbell: One via QEMU via a registered
io region and through the doorbell ioctl(). The other is direct via
ioeventfd.
You can download this test harness here:
ftp://ftp.novell.com/dev/ghaskins/doorbell.tar.bz2
The measured results are as follows:
qemu-mmio: 110000 iops, 9.09us rtt
ioeventfd-mmio: 200100 iops, 5.00us rtt
ioeventfd-pio: 367300 iops, 2.72us rtt
I didn't measure qemu-pio, because I have to figure out how to register a
PIO region with qemu's device model, and I got lazy. However, for now we
can extrapolate based on the data from the NULLIO runs of +2.56us for MMIO,
and -350ns for HC, we get:
qemu-pio: 153139 iops, 6.53us rtt
ioeventfd-hc: 412585 iops, 2.37us rtt
these are just for fun, for now, until I can gather more data.
Here is a graph for your convenience:
http://developer.novell.com/wiki/images/7/76/Iofd-chart.png
The conclusion to draw is that we save about 4us by skipping the userspace
hop.
--------------------
Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Avi Kivity <avi@redhat.com>
2009-07-08 05:08:49 +08:00
|
|
|
goto out;
|
|
|
|
r = kvm_ioeventfd(kvm, &data);
|
|
|
|
break;
|
|
|
|
}
|
2012-03-30 03:14:12 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_MSI
|
|
|
|
case KVM_SIGNAL_MSI: {
|
|
|
|
struct kvm_msi msi;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&msi, argp, sizeof(msi)))
|
2012-03-30 03:14:12 +08:00
|
|
|
goto out;
|
|
|
|
r = kvm_send_userspace_msi(kvm, &msi);
|
|
|
|
break;
|
|
|
|
}
|
2012-07-24 20:51:20 +08:00
|
|
|
#endif
|
|
|
|
#ifdef __KVM_HAVE_IRQ_LINE
|
|
|
|
case KVM_IRQ_LINE_STATUS:
|
|
|
|
case KVM_IRQ_LINE: {
|
|
|
|
struct kvm_irq_level irq_event;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_from_user(&irq_event, argp, sizeof(irq_event)))
|
2012-07-24 20:51:20 +08:00
|
|
|
goto out;
|
|
|
|
|
2013-04-11 19:21:40 +08:00
|
|
|
r = kvm_vm_ioctl_irq_line(kvm, &irq_event,
|
|
|
|
ioctl == KVM_IRQ_LINE_STATUS);
|
2012-07-24 20:51:20 +08:00
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (ioctl == KVM_IRQ_LINE_STATUS) {
|
2015-02-26 14:58:19 +08:00
|
|
|
if (copy_to_user(argp, &irq_event, sizeof(irq_event)))
|
2012-07-24 20:51:20 +08:00
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
2009-06-09 20:56:28 +08:00
|
|
|
#endif
|
2013-04-16 03:12:53 +08:00
|
|
|
#ifdef CONFIG_HAVE_KVM_IRQ_ROUTING
|
|
|
|
case KVM_SET_GSI_ROUTING: {
|
|
|
|
struct kvm_irq_routing routing;
|
|
|
|
struct kvm_irq_routing __user *urouting;
|
2016-06-01 20:09:22 +08:00
|
|
|
struct kvm_irq_routing_entry *entries = NULL;
|
2013-04-16 03:12:53 +08:00
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&routing, argp, sizeof(routing)))
|
|
|
|
goto out;
|
|
|
|
r = -EINVAL;
|
2017-04-28 23:06:20 +08:00
|
|
|
if (!kvm_arch_can_set_irq_routing(kvm))
|
|
|
|
goto out;
|
kvm: Fix irq route entries exceeding KVM_MAX_IRQ_ROUTES
These days, we experienced one guest crash with 8 cores and 3 disks,
with qemu error logs as bellow:
qemu-system-x86_64: /build/qemu-2.0.0/kvm-all.c:984:
kvm_irqchip_commit_routes: Assertion `ret == 0' failed.
And then we found one patch(bdf026317d) in qemu tree, which said
could fix this bug.
Execute the following script will reproduce the BUG quickly:
irq_affinity.sh
========================================================================
vda_irq_num=25
vdb_irq_num=27
while [ 1 ]
do
for irq in {1,2,4,8,10,20,40,80}
do
echo $irq > /proc/irq/$vda_irq_num/smp_affinity
echo $irq > /proc/irq/$vdb_irq_num/smp_affinity
dd if=/dev/vda of=/dev/zero bs=4K count=100 iflag=direct
dd if=/dev/vdb of=/dev/zero bs=4K count=100 iflag=direct
done
done
========================================================================
The following qemu log is added in the qemu code and is displayed when
this bug reproduced:
kvm_irqchip_commit_routes: max gsi: 1008, nr_allocated_irq_routes: 1024,
irq_routes->nr: 1024, gsi_count: 1024.
That's to say when irq_routes->nr == 1024, there are 1024 routing entries,
but in the kernel code when routes->nr >= 1024, will just return -EINVAL;
The nr is the number of the routing entries which is in of
[1 ~ KVM_MAX_IRQ_ROUTES], not the index in [0 ~ KVM_MAX_IRQ_ROUTES - 1].
This patch fix the BUG above.
Cc: stable@vger.kernel.org
Signed-off-by: Xiubo Li <lixiubo@cmss.chinamobile.com>
Signed-off-by: Wei Tang <tangwei@cmss.chinamobile.com>
Signed-off-by: Zhang Zhuoyu <zhangzhuoyu@cmss.chinamobile.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-06-15 18:00:33 +08:00
|
|
|
if (routing.nr > KVM_MAX_IRQ_ROUTES)
|
2013-04-16 03:12:53 +08:00
|
|
|
goto out;
|
|
|
|
if (routing.flags)
|
|
|
|
goto out;
|
2016-06-01 20:09:22 +08:00
|
|
|
if (routing.nr) {
|
|
|
|
urouting = argp;
|
2020-06-03 18:11:31 +08:00
|
|
|
entries = vmemdup_user(urouting->entries,
|
|
|
|
array_size(sizeof(*entries),
|
|
|
|
routing.nr));
|
|
|
|
if (IS_ERR(entries)) {
|
|
|
|
r = PTR_ERR(entries);
|
|
|
|
goto out;
|
|
|
|
}
|
2016-06-01 20:09:22 +08:00
|
|
|
}
|
2013-04-16 03:12:53 +08:00
|
|
|
r = kvm_set_irq_routing(kvm, entries, routing.nr,
|
|
|
|
routing.flags);
|
2020-06-03 18:11:31 +08:00
|
|
|
kvfree(entries);
|
2013-04-16 03:12:53 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif /* CONFIG_HAVE_KVM_IRQ_ROUTING */
|
2013-04-12 22:08:42 +08:00
|
|
|
case KVM_CREATE_DEVICE: {
|
|
|
|
struct kvm_create_device cd;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_from_user(&cd, argp, sizeof(cd)))
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
r = kvm_ioctl_create_device(kvm, &cd);
|
|
|
|
if (r)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
r = -EFAULT;
|
|
|
|
if (copy_to_user(argp, &cd, sizeof(cd)))
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
r = 0;
|
|
|
|
break;
|
|
|
|
}
|
2014-07-15 00:33:08 +08:00
|
|
|
case KVM_CHECK_EXTENSION:
|
|
|
|
r = kvm_vm_ioctl_check_extension_generic(kvm, arg);
|
|
|
|
break;
|
2020-10-01 09:22:22 +08:00
|
|
|
case KVM_RESET_DIRTY_RINGS:
|
|
|
|
r = kvm_vm_ioctl_reset_dirty_pages(kvm);
|
|
|
|
break;
|
2021-06-19 06:27:05 +08:00
|
|
|
case KVM_GET_STATS_FD:
|
|
|
|
r = kvm_vm_ioctl_get_stats_fd(kvm);
|
|
|
|
break;
|
2007-02-22 01:28:04 +08:00
|
|
|
default:
|
2007-10-29 23:08:35 +08:00
|
|
|
r = kvm_arch_vm_ioctl(filp, ioctl, arg);
|
2007-02-22 01:28:04 +08:00
|
|
|
}
|
|
|
|
out:
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2015-02-03 16:35:15 +08:00
|
|
|
#ifdef CONFIG_KVM_COMPAT
|
2009-10-22 20:19:27 +08:00
|
|
|
struct compat_kvm_dirty_log {
|
|
|
|
__u32 slot;
|
|
|
|
__u32 padding1;
|
|
|
|
union {
|
|
|
|
compat_uptr_t dirty_bitmap; /* one bit per page */
|
|
|
|
__u64 padding2;
|
|
|
|
};
|
|
|
|
};
|
|
|
|
|
2021-07-27 20:43:10 +08:00
|
|
|
struct compat_kvm_clear_dirty_log {
|
|
|
|
__u32 slot;
|
|
|
|
__u32 num_pages;
|
|
|
|
__u64 first_page;
|
|
|
|
union {
|
|
|
|
compat_uptr_t dirty_bitmap; /* one bit per page */
|
|
|
|
__u64 padding2;
|
|
|
|
};
|
|
|
|
};
|
|
|
|
|
2009-10-22 20:19:27 +08:00
|
|
|
static long kvm_vm_compat_ioctl(struct file *filp,
|
|
|
|
unsigned int ioctl, unsigned long arg)
|
|
|
|
{
|
|
|
|
struct kvm *kvm = filp->private_data;
|
|
|
|
int r;
|
|
|
|
|
2021-07-03 06:04:23 +08:00
|
|
|
if (kvm->mm != current->mm || kvm->vm_bugged)
|
2009-10-22 20:19:27 +08:00
|
|
|
return -EIO;
|
|
|
|
switch (ioctl) {
|
2021-07-27 20:43:10 +08:00
|
|
|
#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT
|
|
|
|
case KVM_CLEAR_DIRTY_LOG: {
|
|
|
|
struct compat_kvm_clear_dirty_log compat_log;
|
|
|
|
struct kvm_clear_dirty_log log;
|
|
|
|
|
|
|
|
if (copy_from_user(&compat_log, (void __user *)arg,
|
|
|
|
sizeof(compat_log)))
|
|
|
|
return -EFAULT;
|
|
|
|
log.slot = compat_log.slot;
|
|
|
|
log.num_pages = compat_log.num_pages;
|
|
|
|
log.first_page = compat_log.first_page;
|
|
|
|
log.padding2 = compat_log.padding2;
|
|
|
|
log.dirty_bitmap = compat_ptr(compat_log.dirty_bitmap);
|
|
|
|
|
|
|
|
r = kvm_vm_ioctl_clear_dirty_log(kvm, &log);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
#endif
|
2009-10-22 20:19:27 +08:00
|
|
|
case KVM_GET_DIRTY_LOG: {
|
|
|
|
struct compat_kvm_dirty_log compat_log;
|
|
|
|
struct kvm_dirty_log log;
|
|
|
|
|
|
|
|
if (copy_from_user(&compat_log, (void __user *)arg,
|
|
|
|
sizeof(compat_log)))
|
2017-01-22 18:30:21 +08:00
|
|
|
return -EFAULT;
|
2009-10-22 20:19:27 +08:00
|
|
|
log.slot = compat_log.slot;
|
|
|
|
log.padding1 = compat_log.padding1;
|
|
|
|
log.padding2 = compat_log.padding2;
|
|
|
|
log.dirty_bitmap = compat_ptr(compat_log.dirty_bitmap);
|
|
|
|
|
|
|
|
r = kvm_vm_ioctl_get_dirty_log(kvm, &log);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
default:
|
|
|
|
r = kvm_vm_ioctl(filp, ioctl, arg);
|
|
|
|
}
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2008-12-02 18:17:32 +08:00
|
|
|
static struct file_operations kvm_vm_fops = {
|
2007-02-22 01:28:04 +08:00
|
|
|
.release = kvm_vm_release,
|
|
|
|
.unlocked_ioctl = kvm_vm_ioctl,
|
llseek: automatically add .llseek fop
All file_operations should get a .llseek operation so we can make
nonseekable_open the default for future file operations without a
.llseek pointer.
The three cases that we can automatically detect are no_llseek, seq_lseek
and default_llseek. For cases where we can we can automatically prove that
the file offset is always ignored, we use noop_llseek, which maintains
the current behavior of not returning an error from a seek.
New drivers should normally not use noop_llseek but instead use no_llseek
and call nonseekable_open at open time. Existing drivers can be converted
to do the same when the maintainer knows for certain that no user code
relies on calling seek on the device file.
The generated code is often incorrectly indented and right now contains
comments that clarify for each added line why a specific variant was
chosen. In the version that gets submitted upstream, the comments will
be gone and I will manually fix the indentation, because there does not
seem to be a way to do that using coccinelle.
Some amount of new code is currently sitting in linux-next that should get
the same modifications, which I will do at the end of the merge window.
Many thanks to Julia Lawall for helping me learn to write a semantic
patch that does all this.
===== begin semantic patch =====
// This adds an llseek= method to all file operations,
// as a preparation for making no_llseek the default.
//
// The rules are
// - use no_llseek explicitly if we do nonseekable_open
// - use seq_lseek for sequential files
// - use default_llseek if we know we access f_pos
// - use noop_llseek if we know we don't access f_pos,
// but we still want to allow users to call lseek
//
@ open1 exists @
identifier nested_open;
@@
nested_open(...)
{
<+...
nonseekable_open(...)
...+>
}
@ open exists@
identifier open_f;
identifier i, f;
identifier open1.nested_open;
@@
int open_f(struct inode *i, struct file *f)
{
<+...
(
nonseekable_open(...)
|
nested_open(...)
)
...+>
}
@ read disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ read_no_fpos disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
... when != off
}
@ write @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ write_no_fpos @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
... when != off
}
@ fops0 @
identifier fops;
@@
struct file_operations fops = {
...
};
@ has_llseek depends on fops0 @
identifier fops0.fops;
identifier llseek_f;
@@
struct file_operations fops = {
...
.llseek = llseek_f,
...
};
@ has_read depends on fops0 @
identifier fops0.fops;
identifier read_f;
@@
struct file_operations fops = {
...
.read = read_f,
...
};
@ has_write depends on fops0 @
identifier fops0.fops;
identifier write_f;
@@
struct file_operations fops = {
...
.write = write_f,
...
};
@ has_open depends on fops0 @
identifier fops0.fops;
identifier open_f;
@@
struct file_operations fops = {
...
.open = open_f,
...
};
// use no_llseek if we call nonseekable_open
////////////////////////////////////////////
@ nonseekable1 depends on !has_llseek && has_open @
identifier fops0.fops;
identifier nso ~= "nonseekable_open";
@@
struct file_operations fops = {
... .open = nso, ...
+.llseek = no_llseek, /* nonseekable */
};
@ nonseekable2 depends on !has_llseek @
identifier fops0.fops;
identifier open.open_f;
@@
struct file_operations fops = {
... .open = open_f, ...
+.llseek = no_llseek, /* open uses nonseekable */
};
// use seq_lseek for sequential files
/////////////////////////////////////
@ seq depends on !has_llseek @
identifier fops0.fops;
identifier sr ~= "seq_read";
@@
struct file_operations fops = {
... .read = sr, ...
+.llseek = seq_lseek, /* we have seq_read */
};
// use default_llseek if there is a readdir
///////////////////////////////////////////
@ fops1 depends on !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier readdir_e;
@@
// any other fop is used that changes pos
struct file_operations fops = {
... .readdir = readdir_e, ...
+.llseek = default_llseek, /* readdir is present */
};
// use default_llseek if at least one of read/write touches f_pos
/////////////////////////////////////////////////////////////////
@ fops2 depends on !fops1 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read.read_f;
@@
// read fops use offset
struct file_operations fops = {
... .read = read_f, ...
+.llseek = default_llseek, /* read accesses f_pos */
};
@ fops3 depends on !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write.write_f;
@@
// write fops use offset
struct file_operations fops = {
... .write = write_f, ...
+ .llseek = default_llseek, /* write accesses f_pos */
};
// Use noop_llseek if neither read nor write accesses f_pos
///////////////////////////////////////////////////////////
@ fops4 depends on !fops1 && !fops2 && !fops3 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
identifier write_no_fpos.write_f;
@@
// write fops use offset
struct file_operations fops = {
...
.write = write_f,
.read = read_f,
...
+.llseek = noop_llseek, /* read and write both use no f_pos */
};
@ depends on has_write && !has_read && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write_no_fpos.write_f;
@@
struct file_operations fops = {
... .write = write_f, ...
+.llseek = noop_llseek, /* write uses no f_pos */
};
@ depends on has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
@@
struct file_operations fops = {
... .read = read_f, ...
+.llseek = noop_llseek, /* read uses no f_pos */
};
@ depends on !has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
@@
struct file_operations fops = {
...
+.llseek = noop_llseek, /* no read or write fn */
};
===== End semantic patch =====
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Julia Lawall <julia@diku.dk>
Cc: Christoph Hellwig <hch@infradead.org>
2010-08-16 00:52:59 +08:00
|
|
|
.llseek = noop_llseek,
|
2018-06-17 17:16:21 +08:00
|
|
|
KVM_COMPAT(kvm_vm_compat_ioctl),
|
2007-02-22 01:28:04 +08:00
|
|
|
};
|
|
|
|
|
2021-04-09 06:32:14 +08:00
|
|
|
bool file_is_kvm(struct file *file)
|
|
|
|
{
|
|
|
|
return file && file->f_op == &kvm_vm_fops;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(file_is_kvm);
|
|
|
|
|
2012-01-04 17:25:20 +08:00
|
|
|
static int kvm_dev_ioctl_create_vm(unsigned long type)
|
2007-02-22 01:28:04 +08:00
|
|
|
{
|
2010-10-27 23:22:10 +08:00
|
|
|
int r;
|
2007-02-22 01:28:04 +08:00
|
|
|
struct kvm *kvm;
|
2016-07-15 00:54:17 +08:00
|
|
|
struct file *file;
|
2007-02-22 01:28:04 +08:00
|
|
|
|
2012-01-04 17:25:20 +08:00
|
|
|
kvm = kvm_create_vm(type);
|
2007-06-28 20:38:16 +08:00
|
|
|
if (IS_ERR(kvm))
|
|
|
|
return PTR_ERR(kvm);
|
2017-03-31 19:53:23 +08:00
|
|
|
#ifdef CONFIG_KVM_MMIO
|
2010-03-15 21:13:30 +08:00
|
|
|
r = kvm_coalesced_mmio_init(kvm);
|
2017-11-21 20:40:17 +08:00
|
|
|
if (r < 0)
|
|
|
|
goto put_kvm;
|
2010-03-15 21:13:30 +08:00
|
|
|
#endif
|
2016-07-15 00:54:17 +08:00
|
|
|
r = get_unused_fd_flags(O_CLOEXEC);
|
2017-11-21 20:40:17 +08:00
|
|
|
if (r < 0)
|
|
|
|
goto put_kvm;
|
|
|
|
|
2021-06-19 06:27:05 +08:00
|
|
|
snprintf(kvm->stats_id, sizeof(kvm->stats_id),
|
|
|
|
"kvm-%d", task_pid_nr(current));
|
|
|
|
|
2016-07-15 00:54:17 +08:00
|
|
|
file = anon_inode_getfile("kvm-vm", &kvm_vm_fops, kvm, O_RDWR);
|
|
|
|
if (IS_ERR(file)) {
|
|
|
|
put_unused_fd(r);
|
2017-11-21 20:40:17 +08:00
|
|
|
r = PTR_ERR(file);
|
|
|
|
goto put_kvm;
|
2016-07-15 00:54:17 +08:00
|
|
|
}
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2017-06-27 21:45:09 +08:00
|
|
|
/*
|
|
|
|
* Don't call kvm_put_kvm anymore at this point; file->f_op is
|
|
|
|
* already set, with ->release() being kvm_vm_release(). In error
|
|
|
|
* cases it will be called by the final fput(file) and will take
|
|
|
|
* care of doing kvm_put_kvm(kvm).
|
|
|
|
*/
|
2016-05-18 19:26:23 +08:00
|
|
|
if (kvm_create_vm_debugfs(kvm, r) < 0) {
|
2016-07-15 00:54:17 +08:00
|
|
|
put_unused_fd(r);
|
|
|
|
fput(file);
|
2016-05-18 19:26:23 +08:00
|
|
|
return -ENOMEM;
|
|
|
|
}
|
2017-07-12 23:56:44 +08:00
|
|
|
kvm_uevent_notify_change(KVM_EVENT_CREATE_VM, kvm);
|
2007-02-22 01:28:04 +08:00
|
|
|
|
2016-07-15 00:54:17 +08:00
|
|
|
fd_install(r, file);
|
2010-10-27 23:22:10 +08:00
|
|
|
return r;
|
2017-11-21 20:40:17 +08:00
|
|
|
|
|
|
|
put_kvm:
|
|
|
|
kvm_put_kvm(kvm);
|
|
|
|
return r;
|
2007-02-22 01:28:04 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static long kvm_dev_ioctl(struct file *filp,
|
|
|
|
unsigned int ioctl, unsigned long arg)
|
|
|
|
{
|
2007-03-07 19:05:38 +08:00
|
|
|
long r = -EINVAL;
|
2007-02-22 01:28:04 +08:00
|
|
|
|
|
|
|
switch (ioctl) {
|
|
|
|
case KVM_GET_API_VERSION:
|
2007-03-07 19:11:17 +08:00
|
|
|
if (arg)
|
|
|
|
goto out;
|
2007-02-22 01:28:04 +08:00
|
|
|
r = KVM_API_VERSION;
|
|
|
|
break;
|
|
|
|
case KVM_CREATE_VM:
|
2012-01-04 17:25:20 +08:00
|
|
|
r = kvm_dev_ioctl_create_vm(arg);
|
2007-02-22 01:28:04 +08:00
|
|
|
break;
|
2007-11-15 23:07:47 +08:00
|
|
|
case KVM_CHECK_EXTENSION:
|
2014-07-15 00:27:35 +08:00
|
|
|
r = kvm_vm_ioctl_check_extension_generic(NULL, arg);
|
2007-03-01 23:56:20 +08:00
|
|
|
break;
|
2007-03-07 19:05:38 +08:00
|
|
|
case KVM_GET_VCPU_MMAP_SIZE:
|
|
|
|
if (arg)
|
|
|
|
goto out;
|
2008-01-24 21:13:08 +08:00
|
|
|
r = PAGE_SIZE; /* struct kvm_run */
|
|
|
|
#ifdef CONFIG_X86
|
|
|
|
r += PAGE_SIZE; /* pio data page */
|
2008-05-30 22:05:54 +08:00
|
|
|
#endif
|
2017-03-31 19:53:23 +08:00
|
|
|
#ifdef CONFIG_KVM_MMIO
|
2008-05-30 22:05:54 +08:00
|
|
|
r += PAGE_SIZE; /* coalesced mmio ring page */
|
2008-01-24 21:13:08 +08:00
|
|
|
#endif
|
2007-03-07 19:05:38 +08:00
|
|
|
break;
|
2008-04-10 20:47:53 +08:00
|
|
|
case KVM_TRACE_ENABLE:
|
|
|
|
case KVM_TRACE_PAUSE:
|
|
|
|
case KVM_TRACE_DISABLE:
|
2009-06-18 22:47:28 +08:00
|
|
|
r = -EOPNOTSUPP;
|
2008-04-10 20:47:53 +08:00
|
|
|
break;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
default:
|
2007-10-10 23:16:19 +08:00
|
|
|
return kvm_arch_dev_ioctl(filp, ioctl, arg);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
out:
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct file_operations kvm_chardev_ops = {
|
|
|
|
.unlocked_ioctl = kvm_dev_ioctl,
|
llseek: automatically add .llseek fop
All file_operations should get a .llseek operation so we can make
nonseekable_open the default for future file operations without a
.llseek pointer.
The three cases that we can automatically detect are no_llseek, seq_lseek
and default_llseek. For cases where we can we can automatically prove that
the file offset is always ignored, we use noop_llseek, which maintains
the current behavior of not returning an error from a seek.
New drivers should normally not use noop_llseek but instead use no_llseek
and call nonseekable_open at open time. Existing drivers can be converted
to do the same when the maintainer knows for certain that no user code
relies on calling seek on the device file.
The generated code is often incorrectly indented and right now contains
comments that clarify for each added line why a specific variant was
chosen. In the version that gets submitted upstream, the comments will
be gone and I will manually fix the indentation, because there does not
seem to be a way to do that using coccinelle.
Some amount of new code is currently sitting in linux-next that should get
the same modifications, which I will do at the end of the merge window.
Many thanks to Julia Lawall for helping me learn to write a semantic
patch that does all this.
===== begin semantic patch =====
// This adds an llseek= method to all file operations,
// as a preparation for making no_llseek the default.
//
// The rules are
// - use no_llseek explicitly if we do nonseekable_open
// - use seq_lseek for sequential files
// - use default_llseek if we know we access f_pos
// - use noop_llseek if we know we don't access f_pos,
// but we still want to allow users to call lseek
//
@ open1 exists @
identifier nested_open;
@@
nested_open(...)
{
<+...
nonseekable_open(...)
...+>
}
@ open exists@
identifier open_f;
identifier i, f;
identifier open1.nested_open;
@@
int open_f(struct inode *i, struct file *f)
{
<+...
(
nonseekable_open(...)
|
nested_open(...)
)
...+>
}
@ read disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ read_no_fpos disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
... when != off
}
@ write @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
<+...
(
*off = E
|
*off += E
|
func(..., off, ...)
|
E = *off
)
...+>
}
@ write_no_fpos @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
... when != off
}
@ fops0 @
identifier fops;
@@
struct file_operations fops = {
...
};
@ has_llseek depends on fops0 @
identifier fops0.fops;
identifier llseek_f;
@@
struct file_operations fops = {
...
.llseek = llseek_f,
...
};
@ has_read depends on fops0 @
identifier fops0.fops;
identifier read_f;
@@
struct file_operations fops = {
...
.read = read_f,
...
};
@ has_write depends on fops0 @
identifier fops0.fops;
identifier write_f;
@@
struct file_operations fops = {
...
.write = write_f,
...
};
@ has_open depends on fops0 @
identifier fops0.fops;
identifier open_f;
@@
struct file_operations fops = {
...
.open = open_f,
...
};
// use no_llseek if we call nonseekable_open
////////////////////////////////////////////
@ nonseekable1 depends on !has_llseek && has_open @
identifier fops0.fops;
identifier nso ~= "nonseekable_open";
@@
struct file_operations fops = {
... .open = nso, ...
+.llseek = no_llseek, /* nonseekable */
};
@ nonseekable2 depends on !has_llseek @
identifier fops0.fops;
identifier open.open_f;
@@
struct file_operations fops = {
... .open = open_f, ...
+.llseek = no_llseek, /* open uses nonseekable */
};
// use seq_lseek for sequential files
/////////////////////////////////////
@ seq depends on !has_llseek @
identifier fops0.fops;
identifier sr ~= "seq_read";
@@
struct file_operations fops = {
... .read = sr, ...
+.llseek = seq_lseek, /* we have seq_read */
};
// use default_llseek if there is a readdir
///////////////////////////////////////////
@ fops1 depends on !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier readdir_e;
@@
// any other fop is used that changes pos
struct file_operations fops = {
... .readdir = readdir_e, ...
+.llseek = default_llseek, /* readdir is present */
};
// use default_llseek if at least one of read/write touches f_pos
/////////////////////////////////////////////////////////////////
@ fops2 depends on !fops1 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read.read_f;
@@
// read fops use offset
struct file_operations fops = {
... .read = read_f, ...
+.llseek = default_llseek, /* read accesses f_pos */
};
@ fops3 depends on !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write.write_f;
@@
// write fops use offset
struct file_operations fops = {
... .write = write_f, ...
+ .llseek = default_llseek, /* write accesses f_pos */
};
// Use noop_llseek if neither read nor write accesses f_pos
///////////////////////////////////////////////////////////
@ fops4 depends on !fops1 && !fops2 && !fops3 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
identifier write_no_fpos.write_f;
@@
// write fops use offset
struct file_operations fops = {
...
.write = write_f,
.read = read_f,
...
+.llseek = noop_llseek, /* read and write both use no f_pos */
};
@ depends on has_write && !has_read && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write_no_fpos.write_f;
@@
struct file_operations fops = {
... .write = write_f, ...
+.llseek = noop_llseek, /* write uses no f_pos */
};
@ depends on has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
@@
struct file_operations fops = {
... .read = read_f, ...
+.llseek = noop_llseek, /* read uses no f_pos */
};
@ depends on !has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
@@
struct file_operations fops = {
...
+.llseek = noop_llseek, /* no read or write fn */
};
===== End semantic patch =====
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Julia Lawall <julia@diku.dk>
Cc: Christoph Hellwig <hch@infradead.org>
2010-08-16 00:52:59 +08:00
|
|
|
.llseek = noop_llseek,
|
2018-06-17 17:16:21 +08:00
|
|
|
KVM_COMPAT(kvm_dev_ioctl),
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
static struct miscdevice kvm_dev = {
|
2007-03-04 19:27:36 +08:00
|
|
|
KVM_MINOR,
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
"kvm",
|
|
|
|
&kvm_chardev_ops,
|
|
|
|
};
|
|
|
|
|
2010-11-16 16:37:41 +08:00
|
|
|
static void hardware_enable_nolock(void *junk)
|
2007-05-24 18:03:52 +08:00
|
|
|
{
|
|
|
|
int cpu = raw_smp_processor_id();
|
2009-09-15 17:37:46 +08:00
|
|
|
int r;
|
2007-05-24 18:03:52 +08:00
|
|
|
|
2008-12-07 18:55:45 +08:00
|
|
|
if (cpumask_test_cpu(cpu, cpus_hardware_enabled))
|
2007-05-24 18:03:52 +08:00
|
|
|
return;
|
2009-09-15 17:37:46 +08:00
|
|
|
|
2008-12-07 18:55:45 +08:00
|
|
|
cpumask_set_cpu(cpu, cpus_hardware_enabled);
|
2009-09-15 17:37:46 +08:00
|
|
|
|
2014-08-28 21:13:03 +08:00
|
|
|
r = kvm_arch_hardware_enable();
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
if (r) {
|
|
|
|
cpumask_clear_cpu(cpu, cpus_hardware_enabled);
|
|
|
|
atomic_inc(&hardware_enable_failed);
|
2015-02-26 14:58:26 +08:00
|
|
|
pr_info("kvm: enabling virtualization on CPU%d failed\n", cpu);
|
2009-09-15 17:37:46 +08:00
|
|
|
}
|
2007-05-24 18:03:52 +08:00
|
|
|
}
|
|
|
|
|
2016-07-14 01:16:37 +08:00
|
|
|
static int kvm_starting_cpu(unsigned int cpu)
|
2010-11-16 16:37:41 +08:00
|
|
|
{
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_lock(&kvm_count_lock);
|
2013-09-10 18:57:17 +08:00
|
|
|
if (kvm_usage_count)
|
|
|
|
hardware_enable_nolock(NULL);
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_unlock(&kvm_count_lock);
|
2016-07-14 01:16:37 +08:00
|
|
|
return 0;
|
2010-11-16 16:37:41 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static void hardware_disable_nolock(void *junk)
|
2007-05-24 18:03:52 +08:00
|
|
|
{
|
|
|
|
int cpu = raw_smp_processor_id();
|
|
|
|
|
2008-12-07 18:55:45 +08:00
|
|
|
if (!cpumask_test_cpu(cpu, cpus_hardware_enabled))
|
2007-05-24 18:03:52 +08:00
|
|
|
return;
|
2008-12-07 18:55:45 +08:00
|
|
|
cpumask_clear_cpu(cpu, cpus_hardware_enabled);
|
2014-08-28 21:13:03 +08:00
|
|
|
kvm_arch_hardware_disable();
|
2007-05-24 18:03:52 +08:00
|
|
|
}
|
|
|
|
|
2016-07-14 01:16:37 +08:00
|
|
|
static int kvm_dying_cpu(unsigned int cpu)
|
2010-11-16 16:37:41 +08:00
|
|
|
{
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_lock(&kvm_count_lock);
|
2013-09-10 18:57:17 +08:00
|
|
|
if (kvm_usage_count)
|
|
|
|
hardware_disable_nolock(NULL);
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_unlock(&kvm_count_lock);
|
2016-07-14 01:16:37 +08:00
|
|
|
return 0;
|
2010-11-16 16:37:41 +08:00
|
|
|
}
|
|
|
|
|
2009-09-15 17:37:46 +08:00
|
|
|
static void hardware_disable_all_nolock(void)
|
|
|
|
{
|
|
|
|
BUG_ON(!kvm_usage_count);
|
|
|
|
|
|
|
|
kvm_usage_count--;
|
|
|
|
if (!kvm_usage_count)
|
2010-11-16 16:37:41 +08:00
|
|
|
on_each_cpu(hardware_disable_nolock, NULL, 1);
|
2009-09-15 17:37:46 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static void hardware_disable_all(void)
|
|
|
|
{
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_lock(&kvm_count_lock);
|
2009-09-15 17:37:46 +08:00
|
|
|
hardware_disable_all_nolock();
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_unlock(&kvm_count_lock);
|
2009-09-15 17:37:46 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static int hardware_enable_all(void)
|
|
|
|
{
|
|
|
|
int r = 0;
|
|
|
|
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_lock(&kvm_count_lock);
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
kvm_usage_count++;
|
|
|
|
if (kvm_usage_count == 1) {
|
|
|
|
atomic_set(&hardware_enable_failed, 0);
|
2010-11-16 16:37:41 +08:00
|
|
|
on_each_cpu(hardware_enable_nolock, NULL, 1);
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
if (atomic_read(&hardware_enable_failed)) {
|
|
|
|
hardware_disable_all_nolock();
|
|
|
|
r = -EBUSY;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2013-09-10 18:58:35 +08:00
|
|
|
raw_spin_unlock(&kvm_count_lock);
|
2009-09-15 17:37:46 +08:00
|
|
|
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2007-07-17 21:17:55 +08:00
|
|
|
static int kvm_reboot(struct notifier_block *notifier, unsigned long val,
|
2007-10-08 21:02:08 +08:00
|
|
|
void *v)
|
2007-07-17 21:17:55 +08:00
|
|
|
{
|
2009-04-29 11:09:04 +08:00
|
|
|
/*
|
|
|
|
* Some (well, at least mine) BIOSes hang on reboot if
|
|
|
|
* in vmx root mode.
|
|
|
|
*
|
|
|
|
* And Intel TXT required VMX off for all cpu when system shutdown.
|
|
|
|
*/
|
2015-02-26 14:58:26 +08:00
|
|
|
pr_info("kvm: exiting hardware virtualization\n");
|
2009-04-29 11:09:04 +08:00
|
|
|
kvm_rebooting = true;
|
2010-11-16 16:37:41 +08:00
|
|
|
on_each_cpu(hardware_disable_nolock, NULL, 1);
|
2007-07-17 21:17:55 +08:00
|
|
|
return NOTIFY_OK;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct notifier_block kvm_reboot_notifier = {
|
|
|
|
.notifier_call = kvm_reboot,
|
|
|
|
.priority = 0,
|
|
|
|
};
|
|
|
|
|
2009-12-24 00:35:24 +08:00
|
|
|
static void kvm_io_bus_destroy(struct kvm_io_bus *bus)
|
2007-06-01 02:08:53 +08:00
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < bus->dev_count; i++) {
|
2011-07-27 21:00:48 +08:00
|
|
|
struct kvm_io_device *pos = bus->range[i].dev;
|
2007-06-01 02:08:53 +08:00
|
|
|
|
|
|
|
kvm_iodevice_destructor(pos);
|
|
|
|
}
|
2009-12-24 00:35:24 +08:00
|
|
|
kfree(bus);
|
2007-06-01 02:08:53 +08:00
|
|
|
}
|
|
|
|
|
2013-08-27 21:41:41 +08:00
|
|
|
static inline int kvm_io_bus_cmp(const struct kvm_io_range *r1,
|
2015-02-26 14:58:25 +08:00
|
|
|
const struct kvm_io_range *r2)
|
2011-07-27 21:00:48 +08:00
|
|
|
{
|
2015-09-15 14:41:57 +08:00
|
|
|
gpa_t addr1 = r1->addr;
|
|
|
|
gpa_t addr2 = r2->addr;
|
|
|
|
|
|
|
|
if (addr1 < addr2)
|
2011-07-27 21:00:48 +08:00
|
|
|
return -1;
|
2015-09-15 14:41:57 +08:00
|
|
|
|
|
|
|
/* If r2->len == 0, match the exact address. If r2->len != 0,
|
|
|
|
* accept any overlapping write. Any order is acceptable for
|
|
|
|
* overlapping ranges, because kvm_io_bus_get_first_dev ensures
|
|
|
|
* we process all of them.
|
|
|
|
*/
|
|
|
|
if (r2->len) {
|
|
|
|
addr1 += r1->len;
|
|
|
|
addr2 += r2->len;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (addr1 > addr2)
|
2011-07-27 21:00:48 +08:00
|
|
|
return 1;
|
2015-09-15 14:41:57 +08:00
|
|
|
|
2011-07-27 21:00:48 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2013-07-16 19:03:29 +08:00
|
|
|
static int kvm_io_bus_sort_cmp(const void *p1, const void *p2)
|
|
|
|
{
|
2013-08-27 21:41:41 +08:00
|
|
|
return kvm_io_bus_cmp(p1, p2);
|
2013-07-16 19:03:29 +08:00
|
|
|
}
|
|
|
|
|
2013-04-06 03:20:30 +08:00
|
|
|
static int kvm_io_bus_get_first_dev(struct kvm_io_bus *bus,
|
2011-07-27 21:00:48 +08:00
|
|
|
gpa_t addr, int len)
|
|
|
|
{
|
|
|
|
struct kvm_io_range *range, key;
|
|
|
|
int off;
|
|
|
|
|
|
|
|
key = (struct kvm_io_range) {
|
|
|
|
.addr = addr,
|
|
|
|
.len = len,
|
|
|
|
};
|
|
|
|
|
|
|
|
range = bsearch(&key, bus->range, bus->dev_count,
|
|
|
|
sizeof(struct kvm_io_range), kvm_io_bus_sort_cmp);
|
|
|
|
if (range == NULL)
|
|
|
|
return -ENOENT;
|
|
|
|
|
|
|
|
off = range - bus->range;
|
|
|
|
|
2013-08-27 21:41:41 +08:00
|
|
|
while (off > 0 && kvm_io_bus_cmp(&key, &bus->range[off-1]) == 0)
|
2011-07-27 21:00:48 +08:00
|
|
|
off--;
|
|
|
|
|
|
|
|
return off;
|
|
|
|
}
|
|
|
|
|
2015-03-26 22:39:28 +08:00
|
|
|
static int __kvm_io_bus_write(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
|
2013-07-03 22:30:53 +08:00
|
|
|
struct kvm_io_range *range, const void *val)
|
|
|
|
{
|
|
|
|
int idx;
|
|
|
|
|
|
|
|
idx = kvm_io_bus_get_first_dev(bus, range->addr, range->len);
|
|
|
|
if (idx < 0)
|
|
|
|
return -EOPNOTSUPP;
|
|
|
|
|
|
|
|
while (idx < bus->dev_count &&
|
2013-08-27 21:41:41 +08:00
|
|
|
kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
|
2015-03-26 22:39:28 +08:00
|
|
|
if (!kvm_iodevice_write(vcpu, bus->range[idx].dev, range->addr,
|
2013-07-03 22:30:53 +08:00
|
|
|
range->len, val))
|
|
|
|
return idx;
|
|
|
|
idx++;
|
|
|
|
}
|
|
|
|
|
|
|
|
return -EOPNOTSUPP;
|
|
|
|
}
|
|
|
|
|
2009-06-30 03:24:32 +08:00
|
|
|
/* kvm_io_bus_write - called under kvm->slots_lock */
|
2015-03-26 22:39:28 +08:00
|
|
|
int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
|
2009-06-30 03:24:32 +08:00
|
|
|
int len, const void *val)
|
2007-06-01 02:08:53 +08:00
|
|
|
{
|
2010-04-19 17:41:23 +08:00
|
|
|
struct kvm_io_bus *bus;
|
2011-07-27 21:00:48 +08:00
|
|
|
struct kvm_io_range range;
|
2013-07-03 22:30:53 +08:00
|
|
|
int r;
|
2011-07-27 21:00:48 +08:00
|
|
|
|
|
|
|
range = (struct kvm_io_range) {
|
|
|
|
.addr = addr,
|
|
|
|
.len = len,
|
|
|
|
};
|
2010-04-19 17:41:23 +08:00
|
|
|
|
2015-03-26 22:39:28 +08:00
|
|
|
bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
|
2017-03-24 01:24:19 +08:00
|
|
|
if (!bus)
|
|
|
|
return -ENOMEM;
|
2015-03-26 22:39:28 +08:00
|
|
|
r = __kvm_io_bus_write(vcpu, bus, &range, val);
|
2013-07-03 22:30:53 +08:00
|
|
|
return r < 0 ? r : 0;
|
|
|
|
}
|
2019-02-22 16:10:09 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_io_bus_write);
|
2013-07-03 22:30:53 +08:00
|
|
|
|
|
|
|
/* kvm_io_bus_write_cookie - called under kvm->slots_lock */
|
2015-03-26 22:39:28 +08:00
|
|
|
int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
|
|
|
|
gpa_t addr, int len, const void *val, long cookie)
|
2013-07-03 22:30:53 +08:00
|
|
|
{
|
|
|
|
struct kvm_io_bus *bus;
|
|
|
|
struct kvm_io_range range;
|
|
|
|
|
|
|
|
range = (struct kvm_io_range) {
|
|
|
|
.addr = addr,
|
|
|
|
.len = len,
|
|
|
|
};
|
|
|
|
|
2015-03-26 22:39:28 +08:00
|
|
|
bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
|
2017-03-24 01:24:19 +08:00
|
|
|
if (!bus)
|
|
|
|
return -ENOMEM;
|
2013-07-03 22:30:53 +08:00
|
|
|
|
|
|
|
/* First try the device referenced by cookie. */
|
|
|
|
if ((cookie >= 0) && (cookie < bus->dev_count) &&
|
2013-08-27 21:41:41 +08:00
|
|
|
(kvm_io_bus_cmp(&range, &bus->range[cookie]) == 0))
|
2015-03-26 22:39:28 +08:00
|
|
|
if (!kvm_iodevice_write(vcpu, bus->range[cookie].dev, addr, len,
|
2013-07-03 22:30:53 +08:00
|
|
|
val))
|
|
|
|
return cookie;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* cookie contained garbage; fall back to search and return the
|
|
|
|
* correct cookie value.
|
|
|
|
*/
|
2015-03-26 22:39:28 +08:00
|
|
|
return __kvm_io_bus_write(vcpu, bus, &range, val);
|
2013-07-03 22:30:53 +08:00
|
|
|
}
|
|
|
|
|
2015-03-26 22:39:28 +08:00
|
|
|
static int __kvm_io_bus_read(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
|
|
|
|
struct kvm_io_range *range, void *val)
|
2013-07-03 22:30:53 +08:00
|
|
|
{
|
|
|
|
int idx;
|
|
|
|
|
|
|
|
idx = kvm_io_bus_get_first_dev(bus, range->addr, range->len);
|
2011-07-27 21:00:48 +08:00
|
|
|
if (idx < 0)
|
|
|
|
return -EOPNOTSUPP;
|
|
|
|
|
|
|
|
while (idx < bus->dev_count &&
|
2013-08-27 21:41:41 +08:00
|
|
|
kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
|
2015-03-26 22:39:28 +08:00
|
|
|
if (!kvm_iodevice_read(vcpu, bus->range[idx].dev, range->addr,
|
2013-07-03 22:30:53 +08:00
|
|
|
range->len, val))
|
|
|
|
return idx;
|
2011-07-27 21:00:48 +08:00
|
|
|
idx++;
|
|
|
|
}
|
|
|
|
|
2009-06-30 03:24:32 +08:00
|
|
|
return -EOPNOTSUPP;
|
|
|
|
}
|
2007-06-01 02:08:53 +08:00
|
|
|
|
2009-06-30 03:24:32 +08:00
|
|
|
/* kvm_io_bus_read - called under kvm->slots_lock */
|
2015-03-26 22:39:28 +08:00
|
|
|
int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
|
2009-12-24 00:35:24 +08:00
|
|
|
int len, void *val)
|
2009-06-30 03:24:32 +08:00
|
|
|
{
|
2010-04-19 17:41:23 +08:00
|
|
|
struct kvm_io_bus *bus;
|
2011-07-27 21:00:48 +08:00
|
|
|
struct kvm_io_range range;
|
2013-07-03 22:30:53 +08:00
|
|
|
int r;
|
2011-07-27 21:00:48 +08:00
|
|
|
|
|
|
|
range = (struct kvm_io_range) {
|
|
|
|
.addr = addr,
|
|
|
|
.len = len,
|
|
|
|
};
|
2009-12-24 00:35:24 +08:00
|
|
|
|
2015-03-26 22:39:28 +08:00
|
|
|
bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
|
2017-03-24 01:24:19 +08:00
|
|
|
if (!bus)
|
|
|
|
return -ENOMEM;
|
2015-03-26 22:39:28 +08:00
|
|
|
r = __kvm_io_bus_read(vcpu, bus, &range, val);
|
2013-07-03 22:30:53 +08:00
|
|
|
return r < 0 ? r : 0;
|
|
|
|
}
|
2011-07-27 21:00:48 +08:00
|
|
|
|
2009-12-24 00:35:26 +08:00
|
|
|
/* Caller must hold slots_lock. */
|
2011-07-27 21:00:48 +08:00
|
|
|
int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
|
|
|
|
int len, struct kvm_io_device *dev)
|
2009-06-30 03:24:26 +08:00
|
|
|
{
|
2018-01-16 21:34:41 +08:00
|
|
|
int i;
|
2009-12-24 00:35:24 +08:00
|
|
|
struct kvm_io_bus *new_bus, *bus;
|
2018-01-16 21:34:41 +08:00
|
|
|
struct kvm_io_range range;
|
2009-07-08 05:08:44 +08:00
|
|
|
|
2017-07-07 16:51:38 +08:00
|
|
|
bus = kvm_get_bus(kvm, bus_idx);
|
2017-03-24 01:24:19 +08:00
|
|
|
if (!bus)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
2013-05-25 06:44:15 +08:00
|
|
|
/* exclude ioeventfd which is limited by maximum fd */
|
|
|
|
if (bus->dev_count - bus->ioeventfd_count > NR_IOBUS_DEVS - 1)
|
2009-07-08 05:08:44 +08:00
|
|
|
return -ENOSPC;
|
2007-06-01 02:08:53 +08:00
|
|
|
|
2019-01-31 00:07:47 +08:00
|
|
|
new_bus = kmalloc(struct_size(bus, range, bus->dev_count + 1),
|
2019-02-12 03:02:49 +08:00
|
|
|
GFP_KERNEL_ACCOUNT);
|
2009-12-24 00:35:24 +08:00
|
|
|
if (!new_bus)
|
|
|
|
return -ENOMEM;
|
2018-01-16 21:34:41 +08:00
|
|
|
|
|
|
|
range = (struct kvm_io_range) {
|
|
|
|
.addr = addr,
|
|
|
|
.len = len,
|
|
|
|
.dev = dev,
|
|
|
|
};
|
|
|
|
|
|
|
|
for (i = 0; i < bus->dev_count; i++)
|
|
|
|
if (kvm_io_bus_cmp(&bus->range[i], &range) > 0)
|
|
|
|
break;
|
|
|
|
|
|
|
|
memcpy(new_bus, bus, sizeof(*bus) + i * sizeof(struct kvm_io_range));
|
|
|
|
new_bus->dev_count++;
|
|
|
|
new_bus->range[i] = range;
|
|
|
|
memcpy(new_bus->range + i + 1, bus->range + i,
|
|
|
|
(bus->dev_count - i) * sizeof(struct kvm_io_range));
|
2009-12-24 00:35:24 +08:00
|
|
|
rcu_assign_pointer(kvm->buses[bus_idx], new_bus);
|
|
|
|
synchronize_srcu_expedited(&kvm->srcu);
|
|
|
|
kfree(bus);
|
2009-07-08 05:08:44 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2021-04-13 06:20:49 +08:00
|
|
|
int kvm_io_bus_unregister_dev(struct kvm *kvm, enum kvm_bus bus_idx,
|
|
|
|
struct kvm_io_device *dev)
|
2009-07-08 05:08:44 +08:00
|
|
|
{
|
2020-09-08 02:55:35 +08:00
|
|
|
int i, j;
|
2009-12-24 00:35:24 +08:00
|
|
|
struct kvm_io_bus *new_bus, *bus;
|
2009-07-08 05:08:44 +08:00
|
|
|
|
2021-04-13 06:20:50 +08:00
|
|
|
lockdep_assert_held(&kvm->slots_lock);
|
|
|
|
|
2017-07-07 16:51:38 +08:00
|
|
|
bus = kvm_get_bus(kvm, bus_idx);
|
2017-03-15 16:01:17 +08:00
|
|
|
if (!bus)
|
2021-04-13 06:20:49 +08:00
|
|
|
return 0;
|
2017-03-15 16:01:17 +08:00
|
|
|
|
2021-04-13 06:20:50 +08:00
|
|
|
for (i = 0; i < bus->dev_count; i++) {
|
2012-03-09 12:17:32 +08:00
|
|
|
if (bus->range[i].dev == dev) {
|
2009-07-08 05:08:44 +08:00
|
|
|
break;
|
|
|
|
}
|
2021-04-13 06:20:50 +08:00
|
|
|
}
|
2009-12-24 00:35:24 +08:00
|
|
|
|
2017-03-24 01:24:19 +08:00
|
|
|
if (i == bus->dev_count)
|
2021-04-13 06:20:49 +08:00
|
|
|
return 0;
|
2012-03-09 12:17:32 +08:00
|
|
|
|
2019-01-31 00:07:47 +08:00
|
|
|
new_bus = kmalloc(struct_size(bus, range, bus->dev_count - 1),
|
2019-02-12 03:02:49 +08:00
|
|
|
GFP_KERNEL_ACCOUNT);
|
2020-09-08 02:55:35 +08:00
|
|
|
if (new_bus) {
|
2020-09-18 20:05:00 +08:00
|
|
|
memcpy(new_bus, bus, struct_size(bus, range, i));
|
2020-09-08 02:55:35 +08:00
|
|
|
new_bus->dev_count--;
|
|
|
|
memcpy(new_bus->range + i, bus->range + i + 1,
|
2020-09-18 20:05:00 +08:00
|
|
|
flex_array_size(new_bus, range, new_bus->dev_count - i));
|
2021-04-13 06:20:48 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
rcu_assign_pointer(kvm->buses[bus_idx], new_bus);
|
|
|
|
synchronize_srcu_expedited(&kvm->srcu);
|
|
|
|
|
|
|
|
/* Destroy the old bus _after_ installing the (null) bus. */
|
|
|
|
if (!new_bus) {
|
2017-03-24 01:24:19 +08:00
|
|
|
pr_err("kvm: failed to shrink bus, removing it completely\n");
|
2020-09-08 02:55:35 +08:00
|
|
|
for (j = 0; j < bus->dev_count; j++) {
|
|
|
|
if (j == i)
|
|
|
|
continue;
|
|
|
|
kvm_iodevice_destructor(bus->range[j].dev);
|
|
|
|
}
|
2017-03-24 01:24:19 +08:00
|
|
|
}
|
2012-03-09 12:17:32 +08:00
|
|
|
|
2009-12-24 00:35:24 +08:00
|
|
|
kfree(bus);
|
2021-04-13 06:20:49 +08:00
|
|
|
return new_bus ? 0 : -ENOMEM;
|
2007-06-01 02:08:53 +08:00
|
|
|
}
|
|
|
|
|
2016-07-15 19:43:26 +08:00
|
|
|
struct kvm_io_device *kvm_io_bus_get_dev(struct kvm *kvm, enum kvm_bus bus_idx,
|
|
|
|
gpa_t addr)
|
|
|
|
{
|
|
|
|
struct kvm_io_bus *bus;
|
|
|
|
int dev_idx, srcu_idx;
|
|
|
|
struct kvm_io_device *iodev = NULL;
|
|
|
|
|
|
|
|
srcu_idx = srcu_read_lock(&kvm->srcu);
|
|
|
|
|
|
|
|
bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
|
2017-03-24 01:24:19 +08:00
|
|
|
if (!bus)
|
|
|
|
goto out_unlock;
|
2016-07-15 19:43:26 +08:00
|
|
|
|
|
|
|
dev_idx = kvm_io_bus_get_first_dev(bus, addr, 1);
|
|
|
|
if (dev_idx < 0)
|
|
|
|
goto out_unlock;
|
|
|
|
|
|
|
|
iodev = bus->range[dev_idx].dev;
|
|
|
|
|
|
|
|
out_unlock:
|
|
|
|
srcu_read_unlock(&kvm->srcu, srcu_idx);
|
|
|
|
|
|
|
|
return iodev;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(kvm_io_bus_get_dev);
|
|
|
|
|
2016-05-18 19:26:23 +08:00
|
|
|
static int kvm_debugfs_open(struct inode *inode, struct file *file,
|
|
|
|
int (*get)(void *, u64 *), int (*set)(void *, u64),
|
|
|
|
const char *fmt)
|
|
|
|
{
|
|
|
|
struct kvm_stat_data *stat_data = (struct kvm_stat_data *)
|
|
|
|
inode->i_private;
|
|
|
|
|
2021-06-25 23:32:07 +08:00
|
|
|
/*
|
|
|
|
* The debugfs files are a reference to the kvm struct which
|
|
|
|
* is still valid when kvm_destroy_vm is called. kvm_get_kvm_safe
|
|
|
|
* avoids the race between open and the removal of the debugfs directory.
|
2016-05-18 19:26:23 +08:00
|
|
|
*/
|
2021-06-25 23:32:07 +08:00
|
|
|
if (!kvm_get_kvm_safe(stat_data->kvm))
|
2016-05-18 19:26:23 +08:00
|
|
|
return -ENOENT;
|
|
|
|
|
2019-10-01 00:48:44 +08:00
|
|
|
if (simple_attr_open(inode, file, get,
|
2021-06-24 05:28:46 +08:00
|
|
|
kvm_stats_debugfs_mode(stat_data->desc) & 0222
|
2019-12-13 21:07:21 +08:00
|
|
|
? set : NULL,
|
|
|
|
fmt)) {
|
2016-05-18 19:26:23 +08:00
|
|
|
kvm_put_kvm(stat_data->kvm);
|
|
|
|
return -ENOMEM;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_debugfs_release(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
struct kvm_stat_data *stat_data = (struct kvm_stat_data *)
|
|
|
|
inode->i_private;
|
|
|
|
|
|
|
|
simple_attr_release(inode, file);
|
|
|
|
kvm_put_kvm(stat_data->kvm);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_get_stat_per_vm(struct kvm *kvm, size_t offset, u64 *val)
|
2016-05-18 19:26:23 +08:00
|
|
|
{
|
2021-06-24 05:28:46 +08:00
|
|
|
*val = *(u64 *)((void *)(&kvm->stat) + offset);
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int kvm_clear_stat_per_vm(struct kvm *kvm, size_t offset)
|
|
|
|
{
|
2021-06-24 05:28:46 +08:00
|
|
|
*(u64 *)((void *)(&kvm->stat) + offset) = 0;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_get_stat_per_vcpu(struct kvm *kvm, size_t offset, u64 *val)
|
2016-10-19 10:49:47 +08:00
|
|
|
{
|
2019-12-13 21:07:21 +08:00
|
|
|
int i;
|
|
|
|
struct kvm_vcpu *vcpu;
|
2016-10-19 10:49:47 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
*val = 0;
|
2016-10-19 10:49:47 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_for_each_vcpu(i, vcpu, kvm)
|
2021-06-24 05:28:46 +08:00
|
|
|
*val += *(u64 *)((void *)(&vcpu->stat) + offset);
|
2016-10-19 10:49:47 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_clear_stat_per_vcpu(struct kvm *kvm, size_t offset)
|
2016-05-18 19:26:23 +08:00
|
|
|
{
|
2019-12-13 21:07:21 +08:00
|
|
|
int i;
|
|
|
|
struct kvm_vcpu *vcpu;
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_for_each_vcpu(i, vcpu, kvm)
|
2021-06-24 05:28:46 +08:00
|
|
|
*(u64 *)((void *)(&vcpu->stat) + offset) = 0;
|
2019-12-13 21:07:21 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_stat_data_get(void *data, u64 *val)
|
2016-05-18 19:26:23 +08:00
|
|
|
{
|
2019-12-13 21:07:21 +08:00
|
|
|
int r = -EFAULT;
|
2016-05-18 19:26:23 +08:00
|
|
|
struct kvm_stat_data *stat_data = (struct kvm_stat_data *)data;
|
|
|
|
|
2021-06-24 05:28:46 +08:00
|
|
|
switch (stat_data->kind) {
|
2019-12-13 21:07:21 +08:00
|
|
|
case KVM_STAT_VM:
|
|
|
|
r = kvm_get_stat_per_vm(stat_data->kvm,
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc->desc.offset, val);
|
2019-12-13 21:07:21 +08:00
|
|
|
break;
|
|
|
|
case KVM_STAT_VCPU:
|
|
|
|
r = kvm_get_stat_per_vcpu(stat_data->kvm,
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc->desc.offset, val);
|
2019-12-13 21:07:21 +08:00
|
|
|
break;
|
|
|
|
}
|
2016-05-18 19:26:23 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
return r;
|
2016-05-18 19:26:23 +08:00
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_stat_data_clear(void *data, u64 val)
|
2016-10-19 10:49:47 +08:00
|
|
|
{
|
2019-12-13 21:07:21 +08:00
|
|
|
int r = -EFAULT;
|
2016-10-19 10:49:47 +08:00
|
|
|
struct kvm_stat_data *stat_data = (struct kvm_stat_data *)data;
|
|
|
|
|
|
|
|
if (val)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2021-06-24 05:28:46 +08:00
|
|
|
switch (stat_data->kind) {
|
2019-12-13 21:07:21 +08:00
|
|
|
case KVM_STAT_VM:
|
|
|
|
r = kvm_clear_stat_per_vm(stat_data->kvm,
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc->desc.offset);
|
2019-12-13 21:07:21 +08:00
|
|
|
break;
|
|
|
|
case KVM_STAT_VCPU:
|
|
|
|
r = kvm_clear_stat_per_vcpu(stat_data->kvm,
|
2021-06-24 05:28:46 +08:00
|
|
|
stat_data->desc->desc.offset);
|
2019-12-13 21:07:21 +08:00
|
|
|
break;
|
|
|
|
}
|
2016-10-19 10:49:47 +08:00
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
return r;
|
2016-10-19 10:49:47 +08:00
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static int kvm_stat_data_open(struct inode *inode, struct file *file)
|
2016-05-18 19:26:23 +08:00
|
|
|
{
|
|
|
|
__simple_attr_check_format("%llu\n", 0ull);
|
2019-12-13 21:07:21 +08:00
|
|
|
return kvm_debugfs_open(inode, file, kvm_stat_data_get,
|
|
|
|
kvm_stat_data_clear, "%llu\n");
|
2016-05-18 19:26:23 +08:00
|
|
|
}
|
|
|
|
|
2019-12-13 21:07:21 +08:00
|
|
|
static const struct file_operations stat_fops_per_vm = {
|
|
|
|
.owner = THIS_MODULE,
|
|
|
|
.open = kvm_stat_data_open,
|
2016-05-18 19:26:23 +08:00
|
|
|
.release = kvm_debugfs_release,
|
2019-12-13 21:07:21 +08:00
|
|
|
.read = simple_attr_read,
|
|
|
|
.write = simple_attr_write,
|
|
|
|
.llseek = no_llseek,
|
2016-05-18 19:26:23 +08:00
|
|
|
};
|
|
|
|
|
2008-02-08 20:20:26 +08:00
|
|
|
static int vm_stat_get(void *_offset, u64 *val)
|
2007-11-18 22:24:12 +08:00
|
|
|
{
|
|
|
|
unsigned offset = (long)_offset;
|
|
|
|
struct kvm *kvm;
|
2016-05-18 19:26:23 +08:00
|
|
|
u64 tmp_val;
|
2007-11-18 22:24:12 +08:00
|
|
|
|
2008-02-08 20:20:26 +08:00
|
|
|
*val = 0;
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2016-05-18 19:26:23 +08:00
|
|
|
list_for_each_entry(kvm, &vm_list, vm_list) {
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_get_stat_per_vm(kvm, offset, &tmp_val);
|
2016-05-18 19:26:23 +08:00
|
|
|
*val += tmp_val;
|
|
|
|
}
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2008-02-08 20:20:26 +08:00
|
|
|
return 0;
|
2007-11-18 22:24:12 +08:00
|
|
|
}
|
|
|
|
|
2016-10-19 10:49:47 +08:00
|
|
|
static int vm_stat_clear(void *_offset, u64 val)
|
|
|
|
{
|
|
|
|
unsigned offset = (long)_offset;
|
|
|
|
struct kvm *kvm;
|
|
|
|
|
|
|
|
if (val)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2016-10-19 10:49:47 +08:00
|
|
|
list_for_each_entry(kvm, &vm_list, vm_list) {
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_clear_stat_per_vm(kvm, offset);
|
2016-10-19 10:49:47 +08:00
|
|
|
}
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2016-10-19 10:49:47 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
DEFINE_SIMPLE_ATTRIBUTE(vm_stat_fops, vm_stat_get, vm_stat_clear, "%llu\n");
|
2021-06-24 05:28:46 +08:00
|
|
|
DEFINE_SIMPLE_ATTRIBUTE(vm_stat_readonly_fops, vm_stat_get, NULL, "%llu\n");
|
2007-11-18 22:24:12 +08:00
|
|
|
|
2008-02-08 20:20:26 +08:00
|
|
|
static int vcpu_stat_get(void *_offset, u64 *val)
|
2007-04-19 22:27:43 +08:00
|
|
|
{
|
|
|
|
unsigned offset = (long)_offset;
|
|
|
|
struct kvm *kvm;
|
2016-05-18 19:26:23 +08:00
|
|
|
u64 tmp_val;
|
2007-04-19 22:27:43 +08:00
|
|
|
|
2008-02-08 20:20:26 +08:00
|
|
|
*val = 0;
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2016-05-18 19:26:23 +08:00
|
|
|
list_for_each_entry(kvm, &vm_list, vm_list) {
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_get_stat_per_vcpu(kvm, offset, &tmp_val);
|
2016-05-18 19:26:23 +08:00
|
|
|
*val += tmp_val;
|
|
|
|
}
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2008-02-08 20:20:26 +08:00
|
|
|
return 0;
|
2007-04-19 22:27:43 +08:00
|
|
|
}
|
|
|
|
|
2016-10-19 10:49:47 +08:00
|
|
|
static int vcpu_stat_clear(void *_offset, u64 val)
|
|
|
|
{
|
|
|
|
unsigned offset = (long)_offset;
|
|
|
|
struct kvm *kvm;
|
|
|
|
|
|
|
|
if (val)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2016-10-19 10:49:47 +08:00
|
|
|
list_for_each_entry(kvm, &vm_list, vm_list) {
|
2019-12-13 21:07:21 +08:00
|
|
|
kvm_clear_stat_per_vcpu(kvm, offset);
|
2016-10-19 10:49:47 +08:00
|
|
|
}
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2016-10-19 10:49:47 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
DEFINE_SIMPLE_ATTRIBUTE(vcpu_stat_fops, vcpu_stat_get, vcpu_stat_clear,
|
|
|
|
"%llu\n");
|
2021-06-24 05:28:46 +08:00
|
|
|
DEFINE_SIMPLE_ATTRIBUTE(vcpu_stat_readonly_fops, vcpu_stat_get, NULL, "%llu\n");
|
2007-04-19 22:27:43 +08:00
|
|
|
|
2017-07-12 23:56:44 +08:00
|
|
|
static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm)
|
|
|
|
{
|
|
|
|
struct kobj_uevent_env *env;
|
|
|
|
unsigned long long created, active;
|
|
|
|
|
|
|
|
if (!kvm_dev.this_device || !kvm)
|
|
|
|
return;
|
|
|
|
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_lock(&kvm_lock);
|
2017-07-12 23:56:44 +08:00
|
|
|
if (type == KVM_EVENT_CREATE_VM) {
|
|
|
|
kvm_createvm_count++;
|
|
|
|
kvm_active_vms++;
|
|
|
|
} else if (type == KVM_EVENT_DESTROY_VM) {
|
|
|
|
kvm_active_vms--;
|
|
|
|
}
|
|
|
|
created = kvm_createvm_count;
|
|
|
|
active = kvm_active_vms;
|
2019-01-04 09:14:28 +08:00
|
|
|
mutex_unlock(&kvm_lock);
|
2017-07-12 23:56:44 +08:00
|
|
|
|
2019-02-12 03:02:49 +08:00
|
|
|
env = kzalloc(sizeof(*env), GFP_KERNEL_ACCOUNT);
|
2017-07-12 23:56:44 +08:00
|
|
|
if (!env)
|
|
|
|
return;
|
|
|
|
|
|
|
|
add_uevent_var(env, "CREATED=%llu", created);
|
|
|
|
add_uevent_var(env, "COUNT=%llu", active);
|
|
|
|
|
2017-07-24 19:40:03 +08:00
|
|
|
if (type == KVM_EVENT_CREATE_VM) {
|
2017-07-12 23:56:44 +08:00
|
|
|
add_uevent_var(env, "EVENT=create");
|
2017-07-24 19:40:03 +08:00
|
|
|
kvm->userspace_pid = task_pid_nr(current);
|
|
|
|
} else if (type == KVM_EVENT_DESTROY_VM) {
|
2017-07-12 23:56:44 +08:00
|
|
|
add_uevent_var(env, "EVENT=destroy");
|
2017-07-24 19:40:03 +08:00
|
|
|
}
|
|
|
|
add_uevent_var(env, "PID=%d", kvm->userspace_pid);
|
2017-07-12 23:56:44 +08:00
|
|
|
|
2021-08-04 17:28:52 +08:00
|
|
|
if (kvm->debugfs_dentry) {
|
2019-02-12 03:02:49 +08:00
|
|
|
char *tmp, *p = kmalloc(PATH_MAX, GFP_KERNEL_ACCOUNT);
|
2017-07-24 19:40:03 +08:00
|
|
|
|
|
|
|
if (p) {
|
|
|
|
tmp = dentry_path_raw(kvm->debugfs_dentry, p, PATH_MAX);
|
|
|
|
if (!IS_ERR(tmp))
|
|
|
|
add_uevent_var(env, "STATS_PATH=%s", tmp);
|
|
|
|
kfree(p);
|
2017-07-12 23:56:44 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
/* no need for checks, since we are adding at most only 5 keys */
|
|
|
|
env->envp[env->envp_idx++] = NULL;
|
|
|
|
kobject_uevent_env(&kvm_dev.this_device->kobj, KOBJ_CHANGE, env->envp);
|
|
|
|
kfree(env);
|
|
|
|
}
|
|
|
|
|
2018-05-30 00:22:04 +08:00
|
|
|
static void kvm_init_debug(void)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2021-06-24 05:28:46 +08:00
|
|
|
const struct file_operations *fops;
|
|
|
|
const struct _kvm_stats_desc *pdesc;
|
|
|
|
int i;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2008-04-16 05:05:42 +08:00
|
|
|
kvm_debugfs_dir = debugfs_create_dir("kvm", NULL);
|
2011-12-15 14:23:16 +08:00
|
|
|
|
2021-06-24 05:28:46 +08:00
|
|
|
for (i = 0; i < kvm_vm_stats_header.num_desc; ++i) {
|
|
|
|
pdesc = &kvm_vm_stats_desc[i];
|
|
|
|
if (kvm_stats_debugfs_mode(pdesc) & 0222)
|
|
|
|
fops = &vm_stat_fops;
|
|
|
|
else
|
|
|
|
fops = &vm_stat_readonly_fops;
|
|
|
|
debugfs_create_file(pdesc->name, kvm_stats_debugfs_mode(pdesc),
|
|
|
|
kvm_debugfs_dir,
|
|
|
|
(void *)(long)pdesc->desc.offset, fops);
|
|
|
|
}
|
|
|
|
|
|
|
|
for (i = 0; i < kvm_vcpu_stats_header.num_desc; ++i) {
|
|
|
|
pdesc = &kvm_vcpu_stats_desc[i];
|
|
|
|
if (kvm_stats_debugfs_mode(pdesc) & 0222)
|
|
|
|
fops = &vcpu_stat_fops;
|
|
|
|
else
|
|
|
|
fops = &vcpu_stat_readonly_fops;
|
|
|
|
debugfs_create_file(pdesc->name, kvm_stats_debugfs_mode(pdesc),
|
|
|
|
kvm_debugfs_dir,
|
|
|
|
(void *)(long)pdesc->desc.offset, fops);
|
2011-12-15 14:23:16 +08:00
|
|
|
}
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
|
2011-03-24 05:16:23 +08:00
|
|
|
static int kvm_suspend(void)
|
2007-02-12 16:54:48 +08:00
|
|
|
{
|
2009-09-15 17:37:46 +08:00
|
|
|
if (kvm_usage_count)
|
2010-11-16 16:37:41 +08:00
|
|
|
hardware_disable_nolock(NULL);
|
2007-02-12 16:54:48 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2011-03-24 05:16:23 +08:00
|
|
|
static void kvm_resume(void)
|
2007-02-12 16:54:48 +08:00
|
|
|
{
|
2010-08-20 16:07:28 +08:00
|
|
|
if (kvm_usage_count) {
|
2019-05-17 16:49:49 +08:00
|
|
|
#ifdef CONFIG_LOCKDEP
|
|
|
|
WARN_ON(lockdep_is_held(&kvm_count_lock));
|
|
|
|
#endif
|
2010-11-16 16:37:41 +08:00
|
|
|
hardware_enable_nolock(NULL);
|
2010-08-20 16:07:28 +08:00
|
|
|
}
|
2007-02-12 16:54:48 +08:00
|
|
|
}
|
|
|
|
|
2011-03-24 05:16:23 +08:00
|
|
|
static struct syscore_ops kvm_syscore_ops = {
|
2007-02-12 16:54:48 +08:00
|
|
|
.suspend = kvm_suspend,
|
|
|
|
.resume = kvm_resume,
|
|
|
|
};
|
|
|
|
|
2007-07-11 23:17:21 +08:00
|
|
|
static inline
|
|
|
|
struct kvm_vcpu *preempt_notifier_to_vcpu(struct preempt_notifier *pn)
|
|
|
|
{
|
|
|
|
return container_of(pn, struct kvm_vcpu, preempt_notifier);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_sched_in(struct preempt_notifier *pn, int cpu)
|
|
|
|
{
|
|
|
|
struct kvm_vcpu *vcpu = preempt_notifier_to_vcpu(pn);
|
2015-02-26 14:58:23 +08:00
|
|
|
|
2019-08-01 11:30:14 +08:00
|
|
|
WRITE_ONCE(vcpu->preempted, false);
|
KVM: Boost vCPUs that are delivering interrupts
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 155%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 179%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-18 19:39:06 +08:00
|
|
|
WRITE_ONCE(vcpu->ready, false);
|
2007-07-11 23:17:21 +08:00
|
|
|
|
2020-01-09 22:57:19 +08:00
|
|
|
__this_cpu_write(kvm_running_vcpu, vcpu);
|
2014-08-22 00:08:05 +08:00
|
|
|
kvm_arch_sched_in(vcpu, cpu);
|
2007-11-14 20:38:21 +08:00
|
|
|
kvm_arch_vcpu_load(vcpu, cpu);
|
2007-07-11 23:17:21 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static void kvm_sched_out(struct preempt_notifier *pn,
|
|
|
|
struct task_struct *next)
|
|
|
|
{
|
|
|
|
struct kvm_vcpu *vcpu = preempt_notifier_to_vcpu(pn);
|
|
|
|
|
2021-06-11 16:28:13 +08:00
|
|
|
if (current->on_rq) {
|
2019-08-01 11:30:14 +08:00
|
|
|
WRITE_ONCE(vcpu->preempted, true);
|
KVM: Boost vCPUs that are delivering interrupts
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 155%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 179%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-18 19:39:06 +08:00
|
|
|
WRITE_ONCE(vcpu->ready, true);
|
|
|
|
}
|
2007-11-14 20:38:21 +08:00
|
|
|
kvm_arch_vcpu_put(vcpu);
|
2020-01-09 22:57:19 +08:00
|
|
|
__this_cpu_write(kvm_running_vcpu, NULL);
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* kvm_get_running_vcpu - get the vcpu running on the current CPU.
|
2020-02-08 00:34:10 +08:00
|
|
|
*
|
|
|
|
* We can disable preemption locally around accessing the per-CPU variable,
|
|
|
|
* and use the resolved vcpu pointer after enabling preemption again,
|
|
|
|
* because even if the current thread is migrated to another CPU, reading
|
|
|
|
* the per-CPU value later will give us the same value as we update the
|
|
|
|
* per-CPU variable in the preempt notifier handlers.
|
2020-01-09 22:57:19 +08:00
|
|
|
*/
|
|
|
|
struct kvm_vcpu *kvm_get_running_vcpu(void)
|
|
|
|
{
|
2020-02-08 00:34:10 +08:00
|
|
|
struct kvm_vcpu *vcpu;
|
|
|
|
|
|
|
|
preempt_disable();
|
|
|
|
vcpu = __this_cpu_read(kvm_running_vcpu);
|
|
|
|
preempt_enable();
|
|
|
|
|
|
|
|
return vcpu;
|
2020-01-09 22:57:19 +08:00
|
|
|
}
|
2020-04-28 14:23:27 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_get_running_vcpu);
|
2020-01-09 22:57:19 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* kvm_get_running_vcpus - get the per-CPU array of currently running vcpus.
|
|
|
|
*/
|
|
|
|
struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void)
|
|
|
|
{
|
|
|
|
return &kvm_running_vcpu;
|
2007-07-11 23:17:21 +08:00
|
|
|
}
|
|
|
|
|
2020-03-22 04:25:55 +08:00
|
|
|
struct kvm_cpu_compat_check {
|
|
|
|
void *opaque;
|
|
|
|
int *ret;
|
|
|
|
};
|
|
|
|
|
|
|
|
static void check_processor_compat(void *data)
|
2019-04-20 13:18:17 +08:00
|
|
|
{
|
2020-03-22 04:25:55 +08:00
|
|
|
struct kvm_cpu_compat_check *c = data;
|
|
|
|
|
|
|
|
*c->ret = kvm_arch_check_processor_compat(c->opaque);
|
2019-04-20 13:18:17 +08:00
|
|
|
}
|
|
|
|
|
2010-04-28 20:39:01 +08:00
|
|
|
int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
|
2007-07-30 19:12:19 +08:00
|
|
|
struct module *module)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2020-03-22 04:25:55 +08:00
|
|
|
struct kvm_cpu_compat_check c;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
int r;
|
2007-07-31 19:23:01 +08:00
|
|
|
int cpu;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-11-14 20:40:21 +08:00
|
|
|
r = kvm_arch_init(opaque);
|
|
|
|
if (r)
|
2007-11-29 15:35:39 +08:00
|
|
|
goto out_fail;
|
2007-11-14 20:39:31 +08:00
|
|
|
|
2013-05-08 10:57:29 +08:00
|
|
|
/*
|
|
|
|
* kvm_arch_init makes sure there's at most one caller
|
|
|
|
* for architectures that support multiple implementations,
|
|
|
|
* like intel and amd on x86.
|
2016-10-26 19:35:56 +08:00
|
|
|
* kvm_arch_init must be called before kvm_irqfd_init to avoid creating
|
|
|
|
* conflicts in case kvm is already setup for another implementation.
|
2013-05-08 10:57:29 +08:00
|
|
|
*/
|
2016-10-26 19:35:56 +08:00
|
|
|
r = kvm_irqfd_init();
|
|
|
|
if (r)
|
|
|
|
goto out_irqfd;
|
2013-05-08 10:57:29 +08:00
|
|
|
|
2009-06-07 05:52:35 +08:00
|
|
|
if (!zalloc_cpumask_var(&cpus_hardware_enabled, GFP_KERNEL)) {
|
2008-12-07 18:55:45 +08:00
|
|
|
r = -ENOMEM;
|
|
|
|
goto out_free_0;
|
|
|
|
}
|
|
|
|
|
2020-03-22 04:25:55 +08:00
|
|
|
r = kvm_arch_hardware_setup(opaque);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
if (r < 0)
|
2019-11-23 10:45:50 +08:00
|
|
|
goto out_free_1;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2020-03-22 04:25:55 +08:00
|
|
|
c.ret = &r;
|
|
|
|
c.opaque = opaque;
|
2007-07-31 19:23:01 +08:00
|
|
|
for_each_online_cpu(cpu) {
|
2020-03-22 04:25:55 +08:00
|
|
|
smp_call_function_single(cpu, check_processor_compat, &c, 1);
|
2007-07-31 19:23:01 +08:00
|
|
|
if (r < 0)
|
2019-11-23 10:45:50 +08:00
|
|
|
goto out_free_2;
|
2007-07-31 19:23:01 +08:00
|
|
|
}
|
|
|
|
|
2016-12-22 03:19:54 +08:00
|
|
|
r = cpuhp_setup_state_nocalls(CPUHP_AP_KVM_STARTING, "kvm/cpu:starting",
|
2016-07-14 01:16:37 +08:00
|
|
|
kvm_starting_cpu, kvm_dying_cpu);
|
2007-02-12 16:54:47 +08:00
|
|
|
if (r)
|
2007-11-29 15:35:39 +08:00
|
|
|
goto out_free_2;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
register_reboot_notifier(&kvm_reboot_notifier);
|
|
|
|
|
2007-07-30 19:12:19 +08:00
|
|
|
/* A kmem cache lets us meet the alignment requirements of fx_save. */
|
2010-04-28 20:39:01 +08:00
|
|
|
if (!vcpu_align)
|
|
|
|
vcpu_align = __alignof__(struct kvm_vcpu);
|
2017-10-26 21:45:46 +08:00
|
|
|
kvm_vcpu_cache =
|
|
|
|
kmem_cache_create_usercopy("kvm_vcpu", vcpu_size, vcpu_align,
|
|
|
|
SLAB_ACCOUNT,
|
|
|
|
offsetof(struct kvm_vcpu, arch),
|
2021-06-19 06:27:06 +08:00
|
|
|
offsetofend(struct kvm_vcpu, stats_id)
|
|
|
|
- offsetof(struct kvm_vcpu, arch),
|
2017-10-26 21:45:46 +08:00
|
|
|
NULL);
|
2007-07-30 19:12:19 +08:00
|
|
|
if (!kvm_vcpu_cache) {
|
|
|
|
r = -ENOMEM;
|
2011-03-24 05:16:23 +08:00
|
|
|
goto out_free_3;
|
2007-07-30 19:12:19 +08:00
|
|
|
}
|
|
|
|
|
2010-10-14 17:22:46 +08:00
|
|
|
r = kvm_async_pf_init();
|
|
|
|
if (r)
|
|
|
|
goto out_free;
|
|
|
|
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
kvm_chardev_ops.owner = module;
|
2008-12-02 18:17:32 +08:00
|
|
|
kvm_vm_fops.owner = module;
|
|
|
|
kvm_vcpu_fops.owner = module;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
|
|
|
r = misc_register(&kvm_dev);
|
|
|
|
if (r) {
|
2015-02-26 14:58:26 +08:00
|
|
|
pr_err("kvm: misc device register failed\n");
|
2010-10-14 17:22:46 +08:00
|
|
|
goto out_unreg;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
|
|
|
|
2011-03-24 05:16:23 +08:00
|
|
|
register_syscore_ops(&kvm_syscore_ops);
|
|
|
|
|
2007-07-11 23:17:21 +08:00
|
|
|
kvm_preempt_ops.sched_in = kvm_sched_in;
|
|
|
|
kvm_preempt_ops.sched_out = kvm_sched_out;
|
|
|
|
|
2018-05-30 00:22:04 +08:00
|
|
|
kvm_init_debug();
|
2009-10-15 07:21:00 +08:00
|
|
|
|
2014-09-24 19:02:46 +08:00
|
|
|
r = kvm_vfio_ops_init();
|
|
|
|
WARN_ON(r);
|
|
|
|
|
KVM: Allow not-present guest page faults to bypass kvm
There are two classes of page faults trapped by kvm:
- host page faults, where the fault is needed to allow kvm to install
the shadow pte or update the guest accessed and dirty bits
- guest page faults, where the guest has faulted and kvm simply injects
the fault back into the guest to handle
The second class, guest page faults, is pure overhead. We can eliminate
some of it on vmx using the following evil trick:
- when we set up a shadow page table entry, if the corresponding guest pte
is not present, set up the shadow pte as not present
- if the guest pte _is_ present, mark the shadow pte as present but also
set one of the reserved bits in the shadow pte
- tell the vmx hardware not to trap faults which have the present bit clear
With this, normal page-not-present faults go directly to the guest,
bypassing kvm entirely.
Unfortunately, this trick only works on Intel hardware, as AMD lacks a
way to discriminate among page faults based on error code. It is also
a little risky since it uses reserved bits which might become unreserved
in the future, so a module parameter is provided to disable it.
Signed-off-by: Avi Kivity <avi@qumranet.com>
2007-09-17 00:58:32 +08:00
|
|
|
return 0;
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2010-10-14 17:22:46 +08:00
|
|
|
out_unreg:
|
|
|
|
kvm_async_pf_deinit();
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
out_free:
|
2007-07-30 19:12:19 +08:00
|
|
|
kmem_cache_destroy(kvm_vcpu_cache);
|
2007-11-29 15:35:39 +08:00
|
|
|
out_free_3:
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
unregister_reboot_notifier(&kvm_reboot_notifier);
|
2016-07-14 01:16:37 +08:00
|
|
|
cpuhp_remove_state_nocalls(CPUHP_AP_KVM_STARTING);
|
2007-11-29 15:35:39 +08:00
|
|
|
out_free_2:
|
2007-11-14 20:38:21 +08:00
|
|
|
kvm_arch_hardware_unsetup();
|
2019-11-23 10:45:50 +08:00
|
|
|
out_free_1:
|
2008-12-07 18:55:45 +08:00
|
|
|
free_cpumask_var(cpus_hardware_enabled);
|
2007-11-29 15:35:39 +08:00
|
|
|
out_free_0:
|
2013-02-28 19:33:18 +08:00
|
|
|
kvm_irqfd_exit();
|
2016-10-26 19:35:56 +08:00
|
|
|
out_irqfd:
|
2013-05-08 10:57:29 +08:00
|
|
|
kvm_arch_exit();
|
|
|
|
out_fail:
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
return r;
|
|
|
|
}
|
2007-11-14 20:39:31 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_init);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
|
2007-11-14 20:39:31 +08:00
|
|
|
void kvm_exit(void)
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
{
|
2015-10-14 18:37:35 +08:00
|
|
|
debugfs_remove_recursive(kvm_debugfs_dir);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
misc_deregister(&kvm_dev);
|
2007-07-30 19:12:19 +08:00
|
|
|
kmem_cache_destroy(kvm_vcpu_cache);
|
2010-10-14 17:22:46 +08:00
|
|
|
kvm_async_pf_deinit();
|
2011-03-24 05:16:23 +08:00
|
|
|
unregister_syscore_ops(&kvm_syscore_ops);
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
unregister_reboot_notifier(&kvm_reboot_notifier);
|
2016-07-14 01:16:37 +08:00
|
|
|
cpuhp_remove_state_nocalls(CPUHP_AP_KVM_STARTING);
|
2010-11-16 16:37:41 +08:00
|
|
|
on_each_cpu(hardware_disable_nolock, NULL, 1);
|
2007-11-14 20:38:21 +08:00
|
|
|
kvm_arch_hardware_unsetup();
|
2007-11-14 20:40:21 +08:00
|
|
|
kvm_arch_exit();
|
2013-02-28 19:33:18 +08:00
|
|
|
kvm_irqfd_exit();
|
2008-12-07 18:55:45 +08:00
|
|
|
free_cpumask_var(cpus_hardware_enabled);
|
2014-10-09 18:30:08 +08:00
|
|
|
kvm_vfio_ops_exit();
|
[PATCH] kvm: userspace interface
web site: http://kvm.sourceforge.net
mailing list: kvm-devel@lists.sourceforge.net
(http://lists.sourceforge.net/lists/listinfo/kvm-devel)
The following patchset adds a driver for Intel's hardware virtualization
extensions to the x86 architecture. The driver adds a character device
(/dev/kvm) that exposes the virtualization capabilities to userspace. Using
this driver, a process can run a virtual machine (a "guest") in a fully
virtualized PC containing its own virtual hard disks, network adapters, and
display.
Using this driver, one can start multiple virtual machines on a host.
Each virtual machine is a process on the host; a virtual cpu is a thread in
that process. kill(1), nice(1), top(1) work as expected. In effect, the
driver adds a third execution mode to the existing two: we now have kernel
mode, user mode, and guest mode. Guest mode has its own address space mapping
guest physical memory (which is accessible to user mode by mmap()ing
/dev/kvm). Guest mode has no access to any I/O devices; any such access is
intercepted and directed to user mode for emulation.
The driver supports i386 and x86_64 hosts and guests. All combinations are
allowed except x86_64 guest on i386 host. For i386 guests and hosts, both pae
and non-pae paging modes are supported.
SMP hosts and UP guests are supported. At the moment only Intel
hardware is supported, but AMD virtualization support is being worked on.
Performance currently is non-stellar due to the naive implementation of the
mmu virtualization, which throws away most of the shadow page table entries
every context switch. We plan to address this in two ways:
- cache shadow page tables across tlb flushes
- wait until AMD and Intel release processors with nested page tables
Currently a virtual desktop is responsive but consumes a lot of CPU. Under
Windows I tried playing pinball and watching a few flash movies; with a recent
CPU one can hardly feel the virtualization. Linux/X is slower, probably due
to X being in a separate process.
In addition to the driver, you need a slightly modified qemu to provide I/O
device emulation and the BIOS.
Caveats (akpm: might no longer be true):
- The Windows install currently bluescreens due to a problem with the
virtual APIC. We are working on a fix. A temporary workaround is to
use an existing image or install through qemu
- Windows 64-bit does not work. That's also true for qemu, so it's
probably a problem with the device model.
[bero@arklinux.org: build fix]
[simon.kagstrom@bth.se: build fix, other fixes]
[uril@qumranet.com: KVM: Expose interrupt bitmap]
[akpm@osdl.org: i386 build fix]
[mingo@elte.hu: i386 fixes]
[rdreier@cisco.com: add log levels to all printks]
[randy.dunlap@oracle.com: Fix sparse NULL and C99 struct init warnings]
[anthony@codemonkey.ws: KVM: AMD SVM: 32-bit host support]
Signed-off-by: Yaniv Kamay <yaniv@qumranet.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Cc: Simon Kagstrom <simon.kagstrom@bth.se>
Cc: Bernhard Rosenkraenzer <bero@arklinux.org>
Signed-off-by: Uri Lublin <uril@qumranet.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-10 18:21:36 +08:00
|
|
|
}
|
2007-11-14 20:39:31 +08:00
|
|
|
EXPORT_SYMBOL_GPL(kvm_exit);
|
2019-11-04 19:22:02 +08:00
|
|
|
|
|
|
|
struct kvm_vm_worker_thread_context {
|
|
|
|
struct kvm *kvm;
|
|
|
|
struct task_struct *parent;
|
|
|
|
struct completion init_done;
|
|
|
|
kvm_vm_thread_fn_t thread_fn;
|
|
|
|
uintptr_t data;
|
|
|
|
int err;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int kvm_vm_worker_thread(void *context)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* The init_context is allocated on the stack of the parent thread, so
|
|
|
|
* we have to locally copy anything that is needed beyond initialization
|
|
|
|
*/
|
|
|
|
struct kvm_vm_worker_thread_context *init_context = context;
|
|
|
|
struct kvm *kvm = init_context->kvm;
|
|
|
|
kvm_vm_thread_fn_t thread_fn = init_context->thread_fn;
|
|
|
|
uintptr_t data = init_context->data;
|
|
|
|
int err;
|
|
|
|
|
|
|
|
err = kthread_park(current);
|
|
|
|
/* kthread_park(current) is never supposed to return an error */
|
|
|
|
WARN_ON(err != 0);
|
|
|
|
if (err)
|
|
|
|
goto init_complete;
|
|
|
|
|
|
|
|
err = cgroup_attach_task_all(init_context->parent, current);
|
|
|
|
if (err) {
|
|
|
|
kvm_err("%s: cgroup_attach_task_all failed with err %d\n",
|
|
|
|
__func__, err);
|
|
|
|
goto init_complete;
|
|
|
|
}
|
|
|
|
|
|
|
|
set_user_nice(current, task_nice(init_context->parent));
|
|
|
|
|
|
|
|
init_complete:
|
|
|
|
init_context->err = err;
|
|
|
|
complete(&init_context->init_done);
|
|
|
|
init_context = NULL;
|
|
|
|
|
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
|
|
|
/* Wait to be woken up by the spawner before proceeding. */
|
|
|
|
kthread_parkme();
|
|
|
|
|
|
|
|
if (!kthread_should_stop())
|
|
|
|
err = thread_fn(kvm, data);
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
|
|
|
int kvm_vm_create_worker_thread(struct kvm *kvm, kvm_vm_thread_fn_t thread_fn,
|
|
|
|
uintptr_t data, const char *name,
|
|
|
|
struct task_struct **thread_ptr)
|
|
|
|
{
|
|
|
|
struct kvm_vm_worker_thread_context init_context = {};
|
|
|
|
struct task_struct *thread;
|
|
|
|
|
|
|
|
*thread_ptr = NULL;
|
|
|
|
init_context.kvm = kvm;
|
|
|
|
init_context.parent = current;
|
|
|
|
init_context.thread_fn = thread_fn;
|
|
|
|
init_context.data = data;
|
|
|
|
init_completion(&init_context.init_done);
|
|
|
|
|
|
|
|
thread = kthread_run(kvm_vm_worker_thread, &init_context,
|
|
|
|
"%s-%d", name, task_pid_nr(current));
|
|
|
|
if (IS_ERR(thread))
|
|
|
|
return PTR_ERR(thread);
|
|
|
|
|
|
|
|
/* kthread_run is never supposed to return NULL */
|
|
|
|
WARN_ON(thread == NULL);
|
|
|
|
|
|
|
|
wait_for_completion(&init_context.init_done);
|
|
|
|
|
|
|
|
if (!init_context.err)
|
|
|
|
*thread_ptr = thread;
|
|
|
|
|
|
|
|
return init_context.err;
|
|
|
|
}
|