UbiquitousOS
/
OpenCloudOS-Kernel
mirror of https://gitee.com/OpenCloudOS/OpenCloudOS-Kernel.git
11
0
Fork 0
Code Issues Projects Releases Wiki Activity
OpenCloudOS-Kernel/arch/powerpc/kvm/book3s_64_slb.S

161 lines
4.0 KiB
ArmAsm
Raw Normal View History

Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, version 2, as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Copyright SUSE Linux Products GmbH 2009
*
* Authors: Alexander Graf <agraf@suse.de>
*/
powerpc: Fix little endian lppaca, slb_shadow and dtl_entry The lppaca, slb_shadow and dtl_entry hypervisor structures are big endian, so we have to byte swap them in little endian builds. LE KVM hosts will also need to be fixed but for now add an #error to remind us. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2013-08-07 00:01:46 +08:00
#ifdef __LITTLE_ENDIAN__
#error Need to fix SLB shadow accesses in little endian mode
#endif
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
#define SHADOW_SLB_ESID(num) (SLBSHADOW_SAVEAREA + (num * 0x10))
#define SHADOW_SLB_VSID(num) (SLBSHADOW_SAVEAREA + (num * 0x10) + 0x8)
#define UNBOLT_SLB_ENTRY(num) \
ld r9, SHADOW_SLB_ESID(num)(r12); \
/* Invalid? Skip. */; \
rldicl. r0, r9, 37, 63; \
beq slb_entry_skip_ ## num; \
xoris r9, r9, SLB_ESID_V@h; \
std r9, SHADOW_SLB_ESID(num)(r12); \
slb_entry_skip_ ## num:
#define REBOLT_SLB_ENTRY(num) \
ld r10, SHADOW_SLB_ESID(num)(r11); \
cmpdi r10, 0; \
KVM: PPC: Fix typo in rebolting code When we're loading bolted entries into the SLB again, we're checking if an entry is in use and only slbmte it when it is. Unfortunately, the check always goes to the skip label of the first entry, resulting in an endless loop when it actually gets triggered. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-05 05:19:22 +08:00
beq slb_exit_skip_ ## num; \
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
oris r10, r10, SLB_ESID_V@h; \
ld r9, SHADOW_SLB_VSID(num)(r11); \
slbmte r9, r10; \
std r10, SHADOW_SLB_ESID(num)(r11); \
slb_exit_skip_ ## num:
/******************************************************************************
* *
* Entry code *
* *
*****************************************************************************/
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
.macro LOAD_GUEST_SEGMENTS
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
/* Required state:
*
* MSR = ~IR|DR
* R13 = PACA
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-08 09:58:03 +08:00
* R1 = host R1
* R2 = host R2
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
* R3 = shadow vcpu
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 15:41:44 +08:00
* all other volatile GPRS = free except R4, R6
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
* SVCPU[CR] = guest CR
* SVCPU[XER] = guest XER
* SVCPU[CTR] = guest CTR
* SVCPU[LR] = guest LR
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
*/
/* Remove LPAR shadow entries */
#if SLB_NUM_BOLTED == 3
ld r12, PACA_SLBSHADOWPTR(r13)
/* Remove bolted entries */
UNBOLT_SLB_ENTRY(0)
UNBOLT_SLB_ENTRY(1)
UNBOLT_SLB_ENTRY(2)
#else
#error unknown number of bolted entries
#endif
/* Flush SLB */
KVM: PPC: Book3S PR: Fix invalidation of SLB entry 0 on guest entry On entering a PR KVM guest, we invalidate the whole SLB before loading up the guest entries. We do this using an slbia instruction, which invalidates all entries except entry 0, followed by an slbie to invalidate entry 0. However, the slbie turns out to be ineffective in some circumstances (specifically when the host linear mapping uses 64k pages) because of errors in computing the parameter to the slbie. The result is that the guest kernel hangs very early in boot because it takes a DSI the first time it tries to access kernel data using a linear mapping address in real mode. Currently we construct bits 36 - 43 (big-endian numbering) of the slbie parameter by taking bits 56 - 63 of the SLB VSID doubleword. These bits for the tlbie are C (class, 1 bit), B (segment size, 2 bits) and 5 reserved bits. For the SLB VSID doubleword these are C (class, 1 bit), reserved (1 bit), LP (large page size, 2 bits), and 4 reserved bits. Thus we are not setting the B field correctly, and when LP = 01 as it is for 64k pages, we are setting a reserved bit. Rather than add more instructions to calculate the slbie parameter correctly, this takes a simpler approach, which is to set entry 0 to zeroes explicitly. Normally slbmte should not be used to invalidate an entry, since it doesn't invalidate the ERATs, but it is OK to use it to invalidate an entry if it is immediately followed by slbia, which does invalidate the ERATs. (This has been confirmed with the Power architects.) This approach takes fewer instructions and will work whatever the contents of entry 0. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-06-22 15:14:11 +08:00
li r10, 0
slbmte r10, r10
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
slbia
/* Fill SLB with our shadow */
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
lbz r12, SVCPU_SLB_MAX(r3)
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
mulli r12, r12, 16
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
addi r12, r12, SVCPU_SLB
add r12, r12, r3
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
/* for (r11 = kvm_slb; r11 < kvm_slb + kvm_slb_size; r11+=slb_entry) */
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
li r11, SVCPU_SLB
add r11, r11, r3
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
slb_loop_enter:
ld r10, 0(r11)
rldicl. r0, r10, 37, 63
beq slb_loop_enter_skip
ld r9, 8(r11)
slbmte r9, r10
slb_loop_enter_skip:
addi r11, r11, 16
cmpd cr0, r11, r12
blt slb_loop_enter
slb_do_enter:
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
.endm
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
/******************************************************************************
* *
* Exit code *
* *
*****************************************************************************/
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
.macro LOAD_HOST_SEGMENTS
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
/* Register usage at this point:
*
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
* R1 = host R1
* R2 = host R2
* R12 = exit handler id
* R13 = shadow vcpu - SHADOW_VCPU_OFF [=PACA on PPC64]
* SVCPU.* = guest *
* SVCPU[CR] = guest CR
* SVCPU[XER] = guest XER
* SVCPU[CTR] = guest CTR
* SVCPU[LR] = guest LR
Add SLB switching code for entry/exit This is the really low level of guest entry/exit code. Book3s_64 has an SLB, which stores all ESID -> VSID mappings we're currently aware of. The segments in the guest differ from the ones on the host, so we need to switch the SLB to tell the MMU that we're in a new context. So we store a shadow of the guest's SLB in the PACA, switch to that on entry and only restore bolted entries on exit, leaving the rest to the Linux SLB fault handler. That way we get a really clean way of switching the SLB. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-11-02 20:02:29 +08:00
*
*/
/* Restore bolted entries from the shadow and fix it along the way */
/* We don't store anything in entry 0, so we don't need to take care of it */
slbia
isync
#if SLB_NUM_BOLTED == 3
ld r11, PACA_SLBSHADOWPTR(r13)
REBOLT_SLB_ENTRY(0)
REBOLT_SLB_ENTRY(1)
REBOLT_SLB_ENTRY(2)
#else
#error unknown number of bolted entries
#endif
slb_do_exit:
KVM: PPC: Make SLB switching code the new segment framework We just introduced generic segment switching code that only needs to call small macros to do the actual switching, but keeps most of the entry / exit code generic. So let's move the SLB switching code over to use this new mechanism. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-16 06:11:48 +08:00
.endm