hollalinux
/
linux-sg2042
mirror of https://github.com/sophgo/linux-riscv.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
linux-sg2042/arch/powerpc/kvm/book3s_rmhandlers.S

170 lines
4.6 KiB
ArmAsm
Raw Normal View History

Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright SUSE Linux Products GmbH 2009
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
KVM: PPC: Assemble book3s{,_hv}_rmhandlers.S separately This makes arch/powerpc/kvm/book3s_rmhandlers.S and arch/powerpc/kvm/book3s_hv_rmhandlers.S be assembled as separate compilation units rather than having them #included in arch/powerpc/kernel/exceptions-64s.S. We no longer have any conditional branches between the exception prologs in exceptions-64s.S and the KVM handlers, so there is no need to keep their contents close together in the vmlinux image. In their current location, they are using up part of the limited space between the first-level interrupt handlers and the firmware NMI data area at offset 0x7000, and with some kernel configurations this area will overflow (e.g. allyesconfig), leading to an "attempt to .org backwards" error when compiling exceptions-64s.S. Moving them out requires that we add some #includes that the book3s_{,hv_}rmhandlers.S code was previously getting implicitly via exceptions-64s.S. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 15:41:11 +08:00
#include <asm/mmu.h>
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
#include <asm/page.h>
#include <asm/asm-offsets.h>
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
#ifdef CONFIG_PPC_BOOK3S_64
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
#include <asm/exception-64s.h>
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
#endif
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/*****************************************************************************
* *
* Real Mode handlers that need to be in low physical memory *
* *
****************************************************************************/
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
#if defined(CONFIG_PPC_BOOK3S_64)
KVM: PPC: Book3S PR: Fix ABIv2 on LE We switched to ABIv2 on Little Endian systems now which gets rid of the dotted function names. Branch to the actual functions when we see such a system. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-06-16 20:37:53 +08:00
#if defined(_CALL_ELF) && _CALL_ELF == 2
#define FUNC(name) name
#else
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
#define FUNC(name) GLUE(.,name)
KVM: PPC: Book3S PR: Fix ABIv2 on LE We switched to ABIv2 on Little Endian systems now which gets rid of the dotted function names. Branch to the actual functions when we see such a system. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-06-16 20:37:53 +08:00
#endif
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
#elif defined(CONFIG_PPC_BOOK3S_32)
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
#define FUNC(name) name
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
.macro INTERRUPT_TRAMPOLINE intno
.global kvmppc_trampoline_\intno
kvmppc_trampoline_\intno:
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
mtspr SPRN_SPRG_SCRATCH0, r13 /* Save r13 */
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/*
* First thing to do is to find out if we're coming
* from a KVM guest or a Linux process.
*
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
* To distinguish, we check a magic byte in the PACA/current
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
*/
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
mfspr r13, SPRN_SPRG_THREAD
lwz r13, THREAD_KVM_SVCPU(r13)
/* PPC32 can have a NULL pointer - let's check for that */
mtspr SPRN_SPRG_SCRATCH1, r12 /* Save r12 */
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
mfcr r12
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
cmpwi r13, 0
bne 1f
2: mtcr r12
mfspr r12, SPRN_SPRG_SCRATCH1
mfspr r13, SPRN_SPRG_SCRATCH0 /* r13 = original r13 */
b kvmppc_resume_\intno /* Get back original handler */
1: tophys(r13, r13)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
stw r12, HSTATE_SCRATCH1(r13)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
mfspr r12, SPRN_SPRG_SCRATCH1
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
stw r12, HSTATE_SCRATCH0(r13)
lbz r12, HSTATE_IN_GUEST(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
cmpwi r12, KVM_GUEST_MODE_NONE
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
bne ..kvmppc_handler_hasmagic_\intno
/* No KVM guest? Then jump back to the Linux handler! */
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
lwz r12, HSTATE_SCRATCH1(r13)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
b 2b
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/* Now we know we're handling a KVM guest */
..kvmppc_handler_hasmagic_\intno:
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
/* Should we just skip the faulting instruction? */
cmpwi r12, KVM_GUEST_MODE_SKIP
beq kvmppc_handler_skip_ins
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/* Let's store which interrupt we're handling */
li r12, \intno
/* Jump into the SLB exit code that goes to the highmem handler */
b kvmppc_handler_trampoline_exit
.endm
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_SYSTEM_RESET
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_MACHINE_CHECK
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_DATA_STORAGE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_INST_STORAGE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_EXTERNAL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_ALIGNMENT
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_PROGRAM
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_FP_UNAVAIL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_DECREMENTER
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_SYSCALL
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_TRACE
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_PERFMON
INTERRUPT_TRAMPOLINE BOOK3S_INTERRUPT_ALTIVEC
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
/*
* Bring us back to the faulting code, but skip the
* faulting instruction.
*
* This is a generic exit path from the interrupt
* trampolines above.
*
* Input Registers:
*
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
* R12 = free
* R13 = Shadow VCPU (PACA)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
* HSTATE.SCRATCH0 = guest R12
* HSTATE.SCRATCH1 = guest CR
KVM: PPC: Make real mode handler generic The real mode handler code was originally writen for 64 bit Book3S only. But since we not add 32 bit functionality too, we need to make some tweaks to it. This patch basically combines using the "long" access defines and using fields from the shadow VCPU we just moved there. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:46 +08:00
* SPRG_SCRATCH0 = guest R13
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
*
*/
kvmppc_handler_skip_ins:
/* Patch the IP to the next instruction */
mfsrr0 r12
addi r12, r12, 4
mtsrr0 r12
/* Clean up all state */
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
lwz r12, HSTATE_SCRATCH1(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
mtcr r12
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:20:58 +08:00
PPC_LL r12, HSTATE_SCRATCH0(r13)
powerpc: Always use SPRN_SPRG_HSCRATCH0 when running in HV mode This uses feature sections to arrange that we always use HSPRG1 as the scratch register in the interrupt entry code rather than SPRG2 when we're running in hypervisor mode on POWER7. This will ensure that we don't trash the guest's SPRG2 when we are running KVM guests. To simplify the code, we define GET_SCRATCH0() and SET_SCRATCH0() macros like the GET_PACA/SET_PACA macros. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-05 11:59:58 +08:00
GET_SCRATCH0(r13)
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
/* And get back into the code */
RFI
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-29 08:18:26 +08:00
#endif
KVM: PPC: Implement 'skip instruction' mode To fetch the last instruction we were interrupted on, we enable DR in early exit code, where we are still in a very transitional phase between guest and host state. Most of the time this seemed to work, but another CPU can easily flush our TLB and HTAB which makes us go in the Linux page fault handler which totally breaks because we still use the guest's SLB entries. To work around that, let's introduce a second KVM guest mode that defines that whenever we get a trap, we don't call the Linux handler or go into the KVM exit code, but just jump over the faulting instruction. That way a potentially bad lwz doesn't trigger any faults and we can later on interpret the invalid instruction we fetched as "fetch didn't work". Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:04 +08:00
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
/*
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 15:41:44 +08:00
* Call kvmppc_handler_trampoline_enter in real mode
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
*
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 15:41:44 +08:00
* On entry, r4 contains the guest shadow MSR
KVM: PPC: Book3S: PR: Rework irq disabling Today, we disable preemption while inside guest context, because we need to expose to the world that we are not in a preemptible context. However, during that time we already have interrupts disabled, which would indicate that we are in a non-preemptible context. The reason the checks for irqs_disabled() fail for us though is that we manually control hard IRQs and ignore all the lazy EE framework. Let's stop doing that. Instead, let's always use lazy EE to indicate when we want to disable IRQs, but do a special final switch that gets us into EE disabled, but soft enabled state. That way when we get back out of guest state, we are immediately ready to process interrupts. This simplifies the code drastically and reduces the time that we appear as preempt disabled. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-08-13 07:04:19 +08:00
* MSR.EE has to be 0 when calling this function
Add interrupt handling code Getting from host state to the guest is only half the story. We also need to return to our host context and handle whatever happened to get us out of the guest. On PowerPC every guest exit is an interrupt. So all we need to do is trap the host's interrupt handlers and get into our #VMEXIT code to handle it. PowerPCs also have a register that can add an offset to the interrupt handlers' adresses which is what the booke KVM code uses. Unfortunately that is a hypervisor ressource and we also want to be able to run KVM when we're running in an LPAR. So we have to hook into the Linux interrupt handlers. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-30 13:47:09 +08:00
*/
KVM: PPC: Assembly functions exported to modules need _GLOBAL_TOC() Both kvmppc_hv_entry_trampoline and kvmppc_entry_trampoline are assembly functions that are exported to modules and also require a valid r2. As such we need to use _GLOBAL_TOC so we provide a global entry point that establishes the TOC (r2). Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2014-06-12 16:16:53 +08:00
_GLOBAL_TOC(kvmppc_entry_trampoline)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 15:41:44 +08:00
mfmsr r5
LOAD_REG_ADDR(r7, kvmppc_handler_trampoline_enter)
toreal(r7)
li r6, MSR_IR | MSR_DR
KVM: PPC: Book3S: PR: Rework irq disabling Today, we disable preemption while inside guest context, because we need to expose to the world that we are not in a preemptible context. However, during that time we already have interrupts disabled, which would indicate that we are in a non-preemptible context. The reason the checks for irqs_disabled() fail for us though is that we manually control hard IRQs and ignore all the lazy EE framework. Let's stop doing that. Instead, let's always use lazy EE to indicate when we want to disable IRQs, but do a special final switch that gets us into EE disabled, but soft enabled state. That way when we get back out of guest state, we are immediately ready to process interrupts. This simplifies the code drastically and reduces the time that we appear as preempt disabled. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-08-13 07:04:19 +08:00
andc r6, r5, r6 /* Clear DR and IR in MSR value */
/*
* Set EE in HOST_MSR so that it's enabled when we get into our
KVM: PPC: Book3S: PR: Enable interrupts earlier Now that the svcpu sync is interrupt aware we can enable interrupts earlier in the exit code path again, moving 32bit and 64bit closer together. While at it, document the fact that we're always executing the exit path with interrupts enabled so that the next person doesn't trap over this. Signed-off-by: Alexander Graf <agraf@suse.de>
2013-11-29 09:32:31 +08:00
* C exit handler function.
KVM: PPC: Book3S: PR: Rework irq disabling Today, we disable preemption while inside guest context, because we need to expose to the world that we are not in a preemptible context. However, during that time we already have interrupts disabled, which would indicate that we are in a non-preemptible context. The reason the checks for irqs_disabled() fail for us though is that we manually control hard IRQs and ignore all the lazy EE framework. Let's stop doing that. Instead, let's always use lazy EE to indicate when we want to disable IRQs, but do a special final switch that gets us into EE disabled, but soft enabled state. That way when we get back out of guest state, we are immediately ready to process interrupts. This simplifies the code drastically and reduces the time that we appear as preempt disabled. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-08-13 07:04:19 +08:00
*/
ori r5, r5, MSR_EE
mtsrr0 r7
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:03 +08:00
mtsrr1 r6
KVM: PPC: Call SLB patching code in interrupt safe manner Currently we're racy when doing the transition from IR=1 to IR=0, from the module memory entry code to the real mode SLB switching code. To work around that I took a look at the RTAS entry code which is faced with a similar problem and did the same thing: A small helper in linear mapped memory that does mtmsr with IR=0 and then RFIs info the actual handler. Thanks to that trick we can safely take page faults in the entry code and only need to be really wary of what to do as of the SLB switching part. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:06 +08:00
RFI
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
#include "book3s_segment.S"