If for any reason the interrupt enable for an ap queue fails the
state machine run for the queue returned wrong return codes to the
caller. So the caller assumed interrupt support for this queue in
enabled and thus did not re-establish the high resolution timer used
for polling. In the end this let to a hang for the user space process
waiting "forever" for the reply.
This patch reworks these return codes to return correct indications
for the caller to re-establish the timer when a queue runs without
interrupt support.
Please note that this is fixing a wrong behavior after a first
failure (enable interrupt support for the queue) failed. However,
looks like this occasionally happens on KVM systems.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
The device struct provides a pointer for driver-private data. Use this
in the zcrypt drivers (as vfio_ap already does), and then remove the
custom pointer from the AP device structs.
As really_probe() will always clear the drvdata pointer on error, we
no longer have to do so ourselves.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
The device struct itself already contains a pointer to its driver.
Use this consistently, instead of duplicating it.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
This patch provides support for new dynamic AP bus message limit
with the existing zcrypt device driver and AP bus core code.
There is support for a new field 'ml' from TAPQ query. The field
gives if != 0 the AP bus limit for this card in 4k chunk units.
The actual message size limit per card is shown as a new read-only
sysfs attribute. The sysfs attribute
/sys/devices/ap/cardxx/max_msg_size
shows the upper limit in bytes used by the AP bus and zcrypt device
driver for requests and replies send to and received from this card.
Currently up to CEX7 support only max 12kB msg size and thus the field
shows 12288 meaning the upper limit of a valid msg for this card is
12kB. Please note that the usable payload is somewhat lower and
depends on the msg type and thus the header struct which is to be
prepended by the zcrypt dd.
The dispatcher responsible for choosing the right card and queue is
aware of the individual card AP bus message limit. So a request is
only assigned to a queue of a card which is able to handle the size of
the request (e.g. a 14kB request will never go to a max 12kB card).
If no such card is found the ioctl will fail with ENODEV.
The reply buffer held by the device driver is determined by the ml
field of the TAPQ for this card. If a response from the card exceeds
this limit however, the response is not truncated but the ioctl for
this request will fail with errno EMSGSIZE to indicate that the device
driver has dropped the response because it would overflow the buffer
limit.
If the request size does not indicate to the dispatcher that an
adapter with extended limit is to be used, a random card will be
chosen when no specific card is addressed (ANY addressing). This may
result in an ioctl failure when the reply size needs an adapter with
extended limit but the randomly chosen one is not capable of handling
the broader reply size. The user space application needs to use
dedicated addressing to forward such a request only to suitable cards
to get requests like this processed properly.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Tuchscherer <ingo.tuchscherer@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This patch brings 3 reworked/new uevent changes:
* All AP uevents caused by an ap card or queue device now carry an
additional uevent env value MODE=<accel|cca|ep11>. Here is an
example:
KERNEL[1267.301292] add /devices/ap/card0a (ap)
ACTION=add
DEVPATH=/devices/ap/card0a
SUBSYSTEM=ap
DEVTYPE=ap_card
DEV_TYPE=000D
MODALIAS=ap:t0D
MODE=ep11 <- this is new
SEQNUM=1095
This is true for bind, unbind, add, remove, and change uevents
related to ap card or ap queue devices.
* On a change of the soft online attribute on a zcrypt queue or card
device a new CHANGE uevent is sent with an env value ONLINE=<0|1>.
Example uevent:
KERNEL[613.067531] change /devices/ap/card09/09.0011 (ap)
ACTION=change
DEVPATH=/devices/ap/card09/09.0011
SUBSYSTEM=ap
ONLINE=0 <- this is new
DEVTYPE=ap_queue
DRIVER=cex4queue
MODE=cca
SEQNUM=1070
- On a change of the config state of an zcrypt card device a new
CHANGE uevent is sent with an env value CONFIG=<0|1>.
Example uevent:
KERNEL[876.258680] change /devices/ap/card09 (ap)
ACTION=change
DEVPATH=/devices/ap/card09
SUBSYSTEM=ap
CONFIG=0 <- this is new
DEVTYPE=ap_card
DRIVER=cex4card
DEV_TYPE=000D
MODALIAS=ap:t0D
MODE=cca
SEQNUM=1073
Setting a card config on/off causes the dependent queue devices to
follow the config state change and thus uevents informing about the
config state change for the queue devices are also emitted.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This patch adds notifications to userspace for two important
conditions of the ap bus:
I) Initial ap bus scan done. This indicates that the initial
scan of all the ap devices (cards, queues) is complete and
ap devices have been build up for all the hardware found.
This condition is signaled with
1) An ap bus change uevent send to userspace with an environment
key/value pair "INITSCAN=done":
# udevadm monitor -k -p
...
KERNEL[97.830919] change /devices/ap (ap)
ACTION=change
DEVPATH=/devices/ap
SUBSYSTEM=ap
INITSCAN=done
SEQNUM=10421
2) A sysfs attribute /sys/bus/ap/scans which shows the
number of completed ap bus scans done since bus init.
So a value of 1 or greater signals that the initial
ap bus scan is complete.
Note: The initial ap bus scan complete condition is fulfilled
and will be signaled even if there was no ap resource found.
II) APQN driver bindings complete. This indicates that all
APQNs have been bound to an zcrypt or alternate device
driver. Only with the help of an device driver an APQN
can be used for crypto load. So the binding complete
condition is the starting point for user space to be
sure all crypto resources on the ap bus are available
for use.
This condition is signaled with
1) An ap bus change uevent send to userspace with an environment
key/value pair "BINDINGS=complete":
# udevadm monitor -k -p
...
KERNEL[97.830975] change /devices/ap (ap)
ACTION=change
DEVPATH=/devices/ap
SUBSYSTEM=ap
BINDINGS=complete
SEQNUM=10422
2) A sysfs attribute /sys/bus/ap/bindings showing
"<nr of bound apqns>/<total nr of apqns> (complete)"
when all available apqns have been bound to device drivers, or
"<nr of bound apqns>/<total nr of apqns>"
when there are some apqns not bound to an device driver.
Note: The binding complete condition is also fulfilled, when
there are no apqns available to bind any device driver. In
this case the binding complete will be signaled AFTER init
scan is done.
Note: This condition may arise multiple times when after
initial scan modifications on the bindings take place. For
example a manual unbind of an APQN switches the binding
complete condition off. When at a later time the unbound APQNs
are bound with an device driver the binding is (again) complete
resulting in another uevent and marking the bindings sysfs
attribute with '(complete)'.
There is also a new function to be used within the kernel:
int ap_wait_init_apqn_bindings_complete(unsigned long timeout)
Interface to wait for the AP bus to have done one initial ap bus
scan and all detected APQNs have been bound to device drivers.
If these both conditions are not fulfilled, this function blocks
on a condition with wait_for_completion_interruptible_timeout().
If these both conditions are fulfilled (before the timeout hits)
the return value is 0. If the timeout (in jiffies) hits instead
-ETIME is returned. On failures negative return values are
returned to the caller. Please note that further unbind/bind
actions after initial binding complete is through do not cause this
function to block again.
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Introduce a way to specify additional debug flags with an crpyto
request to be able to trigger certain failures within the zcrypt
device drivers and/or ap core code.
This failure injection possibility is only enabled with a kernel debug
build CONFIG_ZCRYPT_DEBUG) and should never be available on a regular
kernel running in production environment.
Details:
* The ioctl(ICARSAMODEXPO) get's a struct ica_rsa_modexpo. If the
leftmost bit of the 32 bit unsigned int inputdatalength field is
set, the uppermost 16 bits are separated and used as debug flag
value. The process is checked to have the CAP_SYS_ADMIN capability
enabled or EPERM is returned.
* The ioctl(ICARSACRT) get's a struct ica_rsa_modexpo_crt. If the
leftmost bit of the 32 bit unsigned int inputdatalength field is set,
the uppermost 16 bits are separated and used als debug flag
value. The process is checked to have the CAP_SYS_ADMIN capability
enabled or EPERM is returned.
* The ioctl(ZSECSENDCPRB) used to send CCA CPRBs get's a struct
ica_xcRB. If the leftmost bit of the 32 bit unsigned int status
field is set, the uppermost 16 bits of this field are used as debug
flag value. The process is checked to have the CAP_SYS_ADMIN
capability enabled or EPERM is returned.
* The ioctl(ZSENDEP11CPRB) used to send EP11 CPRBs get's a struct
ep11_urb. If the leftmost bit of the 64 bit unsigned int req_len
field is set, the uppermost 16 bits of this field are used as debug
flag value. The process is checked to have the CAP_SYS_ADMIN
capability enabled or EPERM is returned.
So it is possible to send an additional 16 bit value to the zcrypt API
to be used to carry a failure injection command which may trigger
special behavior within the zcrypt API and layers below. This 16 bit
value is for the rest of the test referred as 'fi command' for Failure
Injection.
The lower 8 bits of the fi command construct a numerical argument in
the range of 1-255 and is the 'fi action' to be performed with the
request or the resulting reply:
* 0x00 (all requests): No failure injection action but flags may be
provided which may affect the processing of the request or reply.
* 0x01 (only CCA CPRBs): The CPRB's agent_ID field is set to
'FF'. This results in an reply code 0x90 (Transport-Protocol
Failure).
* 0x02 (only CCA CPRBs): After the APQN to send to has been chosen,
the domain field within the CPRB is overwritten with value 99 to
enforce an reply with RY 0x8A.
* 0x03 (all requests): At NQAP invocation the invalid qid value 0xFF00
is used causing an response code of 0x01 (AP queue not valid).
The upper 8 bits of the fi command may carry bit flags which may
influence the processing of an request or response:
* 0x01: No retry. If this bit is set, the usual loop in the zcrypt API
which retries an CPRB up to 10 times when the lower layers return
with EAGAIN is abandoned after the first attempt to send the CPRB.
* 0x02: Toggle special. Toggles the special bit on this request. This
should result in an reply code RY~0x41 and result in an ioctl
failure with errno EINVAL.
This failure injection possibilities may get some further extensions
in the future. As of now this is a starting point for Continuous Test
and Integration to trigger some failures and watch for the reaction of
the ap bus and zcrypt device driver code.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Revisit the ap queue error handling: Based on discussions and
evaluatios with the firmware folk here is now a rework of the response
code handling for all the AP instructions. The idea is to distinguish
between failures because of some kind of invalid request where a retry
does not make any sense and a failure where another attempt to send
the very same request may succeed. The first case is handled by
returning EINVAL to the userspace application. The second case results
in retries within the zcrypt API controlled by a per message retry
counter.
Revisit the zcrpyt error handling: Similar here, based on discussions
with the firmware people here comes a rework of the handling of all
the reply codes. Main point here is that there are only very few
cases left, where a zcrypt device queue is switched to offline. It
should never be the case that an AP reply message is 'unknown' to the
device driver as it indicates a total mismatch between device driver
and crypto card firmware. In all other cases, the code distinguishes
between failure because of invalid message (see above - EINVAL) or
failures of the infrastructure (see above - EAGAIN).
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This patch adds a new config state to the ap card and queue
devices. This state reflects the response code
0x03 "AP deconfigured" on TQAP invocation and is tracked with
every ap bus scan.
Together with this new state now a card/queue device which
is 'deconfigured' is not disposed any more. However, for backward
compatibility the online state now needs to take this state into
account. So a card/queue is offline when the device is not configured.
Furthermore a device can't get switched from offline to online state
when not configured.
The config state is shown in sysfs at
/sys/devices/ap/cardxx/config
for the card and
/sys/devices/ap/cardxx/xx.yyyy/config
for each queue within each card.
It is a read-only attribute reflecting the negation of the
'AP deconfig' state as it is noted in the AP documents.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
On AP instruction failures the last response code is now
kept in the struct ap_queue. There is also a new sysfs
attribute showing this field (enabled only on debug kernels).
Also slight rework of the AP_DBF macros to get some more
content into one debug feature message line.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The state machine for each ap queue covered a mixture of
device states and state machine (firmware queue state) states.
This patch splits the device states and the state machine
states into two different enums and variables. The major
state is the device state with currently these values:
AP_DEV_STATE_UNINITIATED - fresh and virgin, not touched
AP_DEV_STATE_OPERATING - queue dev is working normal
AP_DEV_STATE_SHUTDOWN - remove/unbind/shutdown in progress
AP_DEV_STATE_ERROR - device is in error state
only when the device state is > UNINITIATED the state machine
is run. The state machine represents the states of the firmware
queue:
AP_SM_STATE_RESET_START - starting point, reset (RAPQ) ap queue
AP_SM_STATE_RESET_WAIT - reset triggered, waiting to be finished
if irqs enabled, set up irq (AQIC)
AP_SM_STATE_SETIRQ_WAIT - enable irq triggered, waiting to be
finished, then go to IDLE
AP_SM_STATE_IDLE - queue is operational but empty
AP_SM_STATE_WORKING - queue is operational, requests are stored
and replies may wait for getting fetched
AP_SM_STATE_QUEUE_FULL - firmware queue is full, so only replies
can get fetched
For debugging each ap queue shows a sysfs attribute 'states' which
displays the device and state machine state and is only available
when the kernel is build with CONFIG_ZCRYPT_DEBUG enabled.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
As said by Linus:
A symmetric naming is only helpful if it implies symmetries in use.
Otherwise it's actively misleading.
In "kzalloc()", the z is meaningful and an important part of what the
caller wants.
In "kzfree()", the z is actively detrimental, because maybe in the
future we really _might_ want to use that "memfill(0xdeadbeef)" or
something. The "zero" part of the interface isn't even _relevant_.
The main reason that kzfree() exists is to clear sensitive information
that should not be leaked to other future users of the same memory
objects.
Rename kzfree() to kfree_sensitive() to follow the example of the recently
added kvfree_sensitive() and make the intention of the API more explicit.
In addition, memzero_explicit() is used to clear the memory to make sure
that it won't get optimized away by the compiler.
The renaming is done by using the command sequence:
git grep -w --name-only kzfree |\
xargs sed -i 's/kzfree/kfree_sensitive/'
followed by some editing of the kfree_sensitive() kerneldoc and adding
a kzfree backward compatibility macro in slab.h.
[akpm@linux-foundation.org: fs/crypto/inline_crypt.c needs linux/slab.h]
[akpm@linux-foundation.org: fix fs/crypto/inline_crypt.c some more]
Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Howells <dhowells@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: Joe Perches <joe@perches.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Dan Carpenter <dan.carpenter@oracle.com>
Cc: "Jason A . Donenfeld" <Jason@zx2c4.com>
Link: http://lkml.kernel.org/r/20200616154311.12314-3-longman@redhat.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
There is a state machine held for each ap queue device.
The states and functions related to this where somethimes
noted with _sm_ somethimes without. This patch clarifies
and renames all the ap queue state machine related functions,
enums and defines to have a _sm_ in the name.
There is no functional change coming with this patch - it's
only beautifying code.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Some beautifications related to the internal only used
struct ap_message and related code. Instead of one int carrying
only the special flag now a u32 flags field is used.
At struct CPRBX the pointers to additional data are now marked
with __user. This caused some changes needed on code, where
these structs are also used within the zcrypt misc functions.
The ica_rsa_* structs now use the generic types __u8, __u32, ...
instead of char, unsigned int.
zcrypt_msg6 and zcrypt_msg50 use min_t() instead of min().
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Provide a new interface function to be used by the ap drivers:
struct ap_queue *ap_get_qdev(ap_qid_t qid);
Returns ptr to the struct ap_queue device or NULL if there
was no ap_queue device with this qid found. When something is
found, the reference count of the embedded device is increased.
So the caller has to decrease the reference count after use
with a call to put_device(&aq->ap_dev.device).
With this patch also the ap_card_list is removed from the
ap core code and a new hashtable is introduced which stores
hnodes of all the ap queues known to the ap bus.
The hashtable approach and a first implementation of this
interface comes from a previous patch from
Anthony Krowiak and an idea from Halil Pasic.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Suggested-by: Tony Krowiak <akrowiak@linux.ibm.com>
Suggested-by: Halil Pasic <pasic@linux.ibm.com>
Reviewed-by: Tony Krowiak <akrowiak@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The s390 power management support has been removed. So the
api registration and the suspend and resume callbacks and
all the code related to this for the ap bus and the ap drivers
is removed with this patch.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The internal statistic counters for the total number of
requests processed per card and per queue used integers. So they do
wrap after a rather huge amount of crypto requests processed. This
patch introduces uint64 counters which should hold much longer but
still may wrap. The sysfs attributes request_count for card and queue
also used only %ld and now display the counter value with %llu.
This is not a security relevant fix. The int overflow which happened
is not in any way exploitable as a security breach.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This patch moves the reset invocation of an ap device when
fresh detected from the ap bus to the probe() function of
the driver responsible for this device.
The virtualisation of ap devices makes it necessary to
remove unconditioned resets on fresh appearing apqn devices.
It may be that such a device is already enabled for guest
usage. So there may be a race condition between host ap bus
and guest ap bus doing the reset. This patch moves the
reset from the ap bus to the zcrypt drivers. So if there
is no zcrypt driver bound to an ap device - for example
the ap device is bound to the vfio device driver - the
ap device is untouched passed to the vfio device driver.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This patch adds CEX7 exploitation support for the AP bus code,
the zcrypt device driver zoo and the vfio device driver.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The zcrypt device driver does not handle CPRBs which address
a control domain correctly. This fix introduces a workaround:
The domain field of the request CPRB is checked if there is
a valid domain value in there. If this is true and the value
is a control only domain (a domain which is enabled in the
crypto config ADM mask but disabled in the AQM mask) the
CPRB is forwarded to the default usage domain. If there is
no default domain, the request is rejected with an ENODEV.
This fix is important for maintaining crypto adapters. For
example one LPAR can use a crypto adapter domain ('Control
and Usage') but another LPAR needs to be able to maintain
this adapter domain ('Control'). Scenarios like this did
not work properly and the patch enables this.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Working with the vfio-ap driver let to some revisit of the way
how an ap (queue) device is removed from the driver.
With the current implementation all the cleanup was done before
the driver even got notified about the removal. Now the ap
queue removal is done in 3 steps:
1) A preparation step, all ap messages within the queue
are flushed and so the driver does 'receive' them.
Also a new state AP_STATE_REMOVE assigned to the queue
makes sure there are no new messages queued in.
2) Now the driver's remove function is invoked and the
driver should do the job of cleaning up it's internal
administration lists or whatever. After 2) is done
it is guaranteed, that the driver is not invoked any
more. On the other hand the driver has to make sure
that the APQN is not accessed any more after step 2
is complete.
3) Now the ap bus code does the job of total cleanup of the
APQN. A reset with zero is triggered and the state of
the queue goes to AP_STATE_UNBOUND.
After step 3) is complete, the ap queue has no pending
messages and the APQN is cleared and so there are no
requests and replies lingering around in the firmware
queue for this APQN. Also the interrupts are disabled.
After these remove steps the ap queue device may be assigned
to another driver.
Stress testing this remove/probe procedure showed a problem with the
correct module reference counting. The actual receive of an reply in
the driver is done asynchronous with completions. So with a driver
change on an ap queue the message flush triggers completions but the
threads waiting for the completions may run at a time where the queue
already has the new driver assigned. So the module_put() at receive
time needs to be done on the driver module which queued the ap
message. This change is also part of this patch.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
When an alternate driver (vfio-ap) has bound an ap queue and this
binding is revised the ap queue device is in an intermittent
state not bound to any driver. The internal state variable
covered this with the state AP_STATE_BORKED which is also used to
reflect broken devices. When now an ap bus scan runs such a
device is destroyed and on the next scan reconstructed.
So a stress test with high frequency switching the queue driver
between the default and the vfio-ap driver hit this gap and the
queue was removed until the next ap bus scan. This fix now
introduces another state for the in-between condition for a queue
momentary not bound to a driver and so the ap bus scan function
skips this device instead of removing it.
Also some very slight but maybe helpful debug feature messages
come with this patch - in particular a message showing that a
broken card/queue device will get removed.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Until the vfio-ap driver came into live there was a well known
agreement about the way how ap devices are initialized and their
states when the driver's probe function is called.
However, the vfio device driver when receiving an ap queue device does
additional resets thereby removing the registration for interrupts for
the ap device done by the ap bus core code. So when later the vfio
driver releases the device and one of the default zcrypt drivers takes
care of the device the interrupt registration needs to get
renewed. The current code does no renew and result is that requests
send into such a queue will never see a reply processed - the
application hangs.
This patch adds a function which resets the aq queue state machine for
the ap queue device and triggers the walk through the initial states
(which are reset and registration for interrupts). This function is
now called before the driver's probe function is invoked.
When the association between driver and device is released, the
driver's remove function is called. The current implementation calls a
ap queue function ap_queue_remove(). This invokation has been moved to
the ap bus function to make the probe / remove pair for ap bus and
drivers more symmetric.
Fixes: 7e0bdbe5c2 ("s390/zcrypt: AP bus support for alternate driver(s)")
Cc: stable@vger.kernel.org # 4.19+
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewd-by: Tony Krowiak <akrowiak@linux.ibm.com>
Reviewd-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
This patch is an extension to the zcrypt device driver to provide,
support and maintain multiple zcrypt device nodes. The individual
zcrypt device nodes can be restricted in terms of crypto cards,
domains and available ioctls. Such a device node can be used as a
base for container solutions like docker to control and restrict
the access to crypto resources.
The handling is done with a new sysfs subdir /sys/class/zcrypt.
Echoing a name (or an empty sting) into the attribute "create" creates
a new zcrypt device node. In /sys/class/zcrypt a new link will appear
which points to the sysfs device tree of this new device. The
attribute files "ioctlmask", "apmask" and "aqmask" in this directory
are used to customize this new zcrypt device node instance. Finally
the zcrypt device node can be destroyed by echoing the name into
/sys/class/zcrypt/destroy. The internal structs holding the device
info are reference counted - so a destroy will not hard remove a
device but only marks it as removable when the reference counter drops
to zero.
The mask values are bitmaps in big endian order starting with bit 0.
So adapter number 0 is the leftmost bit, mask is 0x8000... The sysfs
attributes accept 2 different formats:
* Absolute hex string starting with 0x like "0x12345678" does set
the mask starting from left to right. If the given string is shorter
than the mask it is padded with 0s on the right. If the string is
longer than the mask an error comes back (EINVAL).
* Relative format - a concatenation (done with ',') of the
terms +<bitnr>[-<bitnr>] or -<bitnr>[-<bitnr>]. <bitnr> may be any
valid number (hex, decimal or octal) in the range 0...255. Here are
some examples:
"+0-15,+32,-128,-0xFF"
"-0-255,+1-16,+0x128"
"+1,+2,+3,+4,-5,-7-10"
A simple usage examples:
# create new zcrypt device 'my_zcrypt':
echo "my_zcrypt" >/sys/class/zcrypt/create
# go into the device dir of this new device
echo "my_zcrypt" >create
cd my_zcrypt/
ls -l
total 0
-rw-r--r-- 1 root root 4096 Jul 20 15:23 apmask
-rw-r--r-- 1 root root 4096 Jul 20 15:23 aqmask
-r--r--r-- 1 root root 4096 Jul 20 15:23 dev
-rw-r--r-- 1 root root 4096 Jul 20 15:23 ioctlmask
lrwxrwxrwx 1 root root 0 Jul 20 15:23 subsystem -> ../../../../class/zcrypt
...
# customize this zcrypt node clone
# enable only adapter 0 and 2
echo "0xa0" >apmask
# enable only domain 6
echo "+6" >aqmask
# enable all 256 ioctls
echo "+0-255" >ioctls
# now the /dev/my_zcrypt may be used
# finally destroy it
echo "my_zcrypt" >/sys/class/zcrypt/destroy
Please note that a very similar 'filtering behavior' also applies to
the parent z90crypt device. The two mask attributes apmask and aqmask
in /sys/bus/ap act the very same for the z90crypt device node. However
the implementation here is totally different as the ap bus acts on
bind/unbind of queue devices and associated drivers but the effect is
still the same. So there are two filters active for each additional
zcrypt device node: The adapter/domain needs to be enabled on the ap
bus level and it needs to be active on the zcrypt device node level.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
The current AP bus, AP devices and AP device drivers implementation
uses a clearly defined mapping for binding AP devices to AP device
drivers. So for example a CEX6C queue will always be bound to the
cex4queue device driver.
The Linux Device Driver model has no sensitivity for more than one
device driver eligible for one device type. If there exist more than
one drivers matching to the device type, simple all drivers are tried
consecutively. There is no way to determine and influence the probing
order of the drivers.
With KVM there is a need to provide additional device drivers matching
to the very same type of AP devices. With a simple implementation the
KVM drivers run in competition to the regular drivers. Whichever
'wins' a device depends on build order and implementation details
within the common Linux Device Driver Model and is not
deterministic. However, a userspace process could figure out which
device should be bound to which driver and sort out the correct
binding by manipulating attributes in the sysfs.
If for security reasons a AP device must not get bound to the 'wrong'
device driver the sorting out has to be done within the Linux kernel
by the AP bus code. This patch modifies the behavior of the AP bus
for probing drivers for devices in a way that two sets of drivers are
usable. Two new bitmasks 'apmask' and 'aqmask' are used to mark a
subset of the APQN range for 'usable by the ap bus and the default
drivers' or 'not usable by the default drivers and thus available for
alternate drivers like vfio-xxx'. So an APQN which is addressed by
this masking only the default drivers will be probed. In contrary an
APQN which is not addressed by the masks will never be probed and
bound to default drivers but onny to alternate drivers.
Eventually the two masks give a way to divide the range of APQNs into
two pools: one pool of APQNs used by the AP bus and the default
drivers and thus via zcrypt drivers available to the userspace of the
system. And another pool where no zcrypt drivers are bound to and
which can be used by alternate drivers (like vfio-xxx) for their
needs. This division is hot-plug save and makes sure a APQN assigned
to an alternate driver is at no time somehow exploitable by the wrong
party.
The two masks are located in sysfs at /sys/bus/ap/apmask and
/sys/bus/ap/aqmask. The mask syntax is exactly the same as the
already existing mask attributes in the /sys/bus/ap directory (for
example ap_usage_domain_mask and ap_control_domain_mask).
By default all APQNs belong to the ap bus and the default drivers:
cat /sys/bus/ap/apmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
cat /sys/bus/ap/aqmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
The masks can be changed at boot time with the kernel command line
like this:
... ap.apmask=0xffff ap.aqmask=0x40
This would give these two pools:
default drivers pool: adapter 0 - 15, domain 1
alternate drivers pool: adapter 0 - 15, all but domain 1
adapter 16-255, all domains
The sysfs attributes for this two masks are writeable and an
administrator is able to reconfigure the assignements on the fly by
writing new mask values into. With changing the mask(s) a revision of
the existing queue to driver bindings is done. So all APQNs which are
bound to the 'wrong' driver are reprobed via kernel function
device_reprobe() and thus the new correct driver will be assigned with
respect of the changed apmask and aqmask bits.
The mask values are bitmaps in big endian order starting with bit 0.
So adapter number 0 is the leftmost bit, mask is 0x8000... The sysfs
attributes accept 2 different formats:
- Absolute hex string starting with 0x like "0x12345678" does set
the mask starting from left to right. If the given string is shorter
than the mask it is padded with 0s on the right. If the string is
longer than the mask an error comes back (EINVAL).
- '+' or '-' followed by a numerical value. Valid examples are "+1",
"-13", "+0x41", "-0xff" and even "+0" and "-0". Only the addressed
bit in the mask is switched on ('+') or off ('-').
This patch will also be the base for an upcoming extension to the
zcrypt drivers to be able to provide additional zcrypt device nodes
with filtering based on ap and aq masks.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Code beautify by following most of the checkpatch suggestions:
- SPDX license identifier line complains by checkpatch
- missing space or newline complains by checkpatch
- octal numbers for permssions complains by checkpatch
- renaming of static sysfs functions complains by checkpatch
- fix of block comment complains by checkpatch
- fix printf like calls where function name instead of %s __func__
was used
- __packed instead of __attribute__((packed))
- init to zero for static variables removed
- use of DEVICE_ATTR_RO and DEVICE_ATTR_RW macros
No functional code changes or API changes!
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Move all the inline functions from the ap bus header
file ap_asm.h into the in-kernel api header file
arch/s390/include/asm/ap.h so that KVM can make use
of all the low level AP functions.
Signed-off-by: Harald Freudenberger <freude@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Tests showed, that the zcrypt device driver produces memory
leaks when a valid CCA or EP11 CPRB can't get delivered or has
a failure during processing within the zcrypt device driver.
This happens when a invalid domain or adapter number is used
or the lower level software or hardware layers produce any
kind of failure during processing of the request.
Only CPRBs send to CCA or EP11 cards can produce this memory
leak. The accelerator and the CPRBs processed by this type
of crypto card is not affected.
The two fields message and private within the ap_message struct
are allocated with pulling the function code for the CPRB but
only freed when processing of the CPRB succeeds. So for example
an invalid domain or adapter field causes the processing to
fail, leaving these two memory areas allocated forever.
Signed-off-by: Harald Freudenberger <freude@de.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
There was an artificial restriction on the card/adapter id
to only 6 bits but all the AP commands do support adapter
ids with 8 bit. This patch removes this restriction to 64
adapters and now up to 256 adapter can get addressed.
Some of the ioctl calls work on the max number of cards
possible (which was 64). These ioctls are now deprecated
but still supported. All the defines, structs and ioctl
interface declarations have been kept for compabibility.
There are now new ioctls (and defines for these) with an
additional '2' appended which provide the extended versions
with 256 cards supported.
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
The AP bus code is not available as kernel module any more.
There was some leftover code dealing with kernel module
exit which has been removed with this patch.
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Now that the SPDX tag is in all drivers/s390/crypto/ files, that
identifies the license in a specific and legally-defined manner. So the
extra GPL text wording can be removed as it is no longer needed at all.
This is done on a quest to remove the 700+ different ways that files in
the kernel describe the GPL license text. And there's unneeded stuff
like the address (sometimes incorrect) for the FSF which is never
needed.
No copyright headers or other non-license-description text was removed.
Cc: Harald Freudenberger <freude@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
It's good to have SPDX identifiers in all files to make it easier to
audit the kernel tree for correct licenses.
Update the drivers/s390/crypto/ files with the correct SPDX license
identifier based on the license text in the file itself. The SPDX
identifier is a legally binding shorthand, which can be used instead of
the full boiler plate text.
This work is based on a script and data from Thomas Gleixner, Philippe
Ombredanne, and Kate Stewart.
Cc: Harald Freudenberger <freude@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Kate Stewart <kstewart@linuxfoundation.org>
Cc: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
In preparation for unconditionally passing the struct timer_list pointer to
all timer callbacks, switch to using the new timer_setup() and from_timer()
to pass the timer pointer explicitly.
Cc: Harald Freudenberger <freude@de.ibm.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
This patch introduces a new ap_qact() function which
exploits the PQAP(QACT) subfunction. QACT is a new
interface to Query the Ap Compatilibity Type based
on a given AP qid, type, mode and version.
Based on this new function the AP bus scan code is
slightly reworked to use this new interface for
querying the compatible type for each new AP queue
device detected. So new and unknown devices can
get automatically mapped to a compatible type and
handled without the need for toleration patches
for every new hardware.
The currently highest known hardware is CEX6S.
With this patch a possible successor can get
queried for a combatible type known by the device
driver without the need for an toleration patch.
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
KVM has a need to fetch the crypto configuration information
as it is returned by the PQAP(QCI) instruction. This patch
introduces a new API ap_query_configuration() which provides
this info in a handy way for the caller.
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Under certain specified conditions, the Test AP Queue (TAPQ)
subfunction of the Process Adjunct Processor Queue (PQAP) instruction
will be intercepted by a guest VM. The guest VM must have a means for
executing the intercepted instruction.
The vfio_ap driver will provide an interface to execute the
PQAP(TAPQ) instruction subfunction on behalf of a guest VM.
The code for executing the AP instructions currently resides in the
AP bus. This patch refactors the AP bus code to externalize access
to the PQAP(TAPQ) instruction subfunction to make it available to
the vfio_ap driver.
Signed-off-by: Tony Krowiak <akrowiak@linux.vnet.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Currently the first eligible AP adapter respectively domain will be
selected to service requests. In case of sequential workload, the
very same adapter/domain will be used.
The adapter/domain selection algorithm now considers the completed
transactions per adaper/domain and therefore ensures a homogeneous
utilization.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Currently the ap infrastructure only supports one domain at a time.
This feature extends the generic cryptographic device driver to
support multiple cryptographic domains simultaneously.
There are now card and queue devices on the AP bus with independent
card and queue drivers. The new /sys layout is as follows:
/sys/bus/ap
devices
<xx>.<yyyy> -> ../../../devices/ap/card<xx>/<xx>.<yyyy>
...
card<xx> -> ../../../devices/ap/card<xx>
...
drivers
<drv>card
card<xx> -> ../../../../devices/ap/card<xx>
<drv>queue
<xx>.<yyyy> -> ../../../../devices/ap/card<xx>/<xx>.<yyyy>
...
/sys/devices/ap
card<xx>
<xx>.<yyyy>
driver -> ../../../../bus/ap/drivers/<zzz>queue
...
driver -> ../../../bus/ap/drivers/<drv>card
...
The two digit <xx> field is the card number, the four digit <yyyy>
field is the queue number and <drv> is the name of the device driver,
e.g. "cex4".
For compatability /sys/bus/ap/card<xx> for the old layout has to exist,
including the attributes that used to reside there.
With additional contributions from Harald Freudenberger and
Martin Schwidefsky.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
The device suspend call triggers all ap devices to fetch potentially
available response messages from the queues. Therefore the
corresponding zcrypt device, that is allocated asynchronously after
ap device probing, needs to be fully prepared. This race condition
could lead to uninitialized response buffers while trying to read
from the queues.
Introduce a new callback within the ap layer to get noticed when a
zcrypt device is fully prepared. Additional checks prevent reading
from devices that are not fully prepared.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Replace the two fields 'unregistered' and 'reset' with a device
state with 5 possible values. Introduce two events for the AP devices,
device poll and device timeout. With the state machine it is easier
to deal with device initialization and suspend/resume. Device polling
is simpler as well, the arkane 'flags' passing is gone.
Reviewd-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
If a AP device is removed while messages are still pending, the requests
are cancelled by calling the message receive function with an error pointer
for the reply. The message type receive handler recognize this and create
a fake hardware error TYPE82_RSP_CODE / REP82_ERROR_MACHINE_FAILURE.
The message with the hardware error then causes a printk and a return
code of -EAGAIN.
Replace the intricate scheme with an explicit return code for this sitation
and avoid the error message.
Reviewd-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
The ap_query_configuration function allocates the ap_config_info
structure, but there is no code to free the structure.
Allocate the structure in the module_init function and free it
again in module_exit.
While we are at it simplify a few functions in regard to the
ap configuration data.
Reviewed-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
ap_test_queue, ap_query_facilities, __ap_query_functions all use
the same PQAP(TAPQ) command. Consolidate the three into a single
ap_test_queue function that returns the AP status and the 64-bit
result. The exception table entry for PQAP(TAPQ) can be avoided
if the T bit for the APFT facility is set only if test_facility(15)
indicated that the facility is present.
Integrate ap_query_function into ap_query queue to avoid calling
PQAP(TAPQ) twice.
Reviewed-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
In case of request timeouts an AP queue reset will be triggered to
recover and reinitialize the AP queue. The previous behavior was an
immediate reset execution regardless of current/pending requests.
Due to newly changed firmware behavior the reset may be delayed, based
on the priority of pending request. The device driver's waiting time
frame was limited, hence it did not received the reset response. As a
consequence interrupts would not be enabled afterwards.
The RAPQ (queue reset) and AQIC (interrupt control) commands will be
treated fully asynchronous now. The device driver will check the reset and
interrupt states periodically, thus it can handle the reinitialization
properly.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Extends the generic cryptographic device driver (zcrypt)
to support the Crypto Express 5S adapter.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
The zcrypt device driver will accept the new crypto adapter
in toleration mode. A new sysfs attribute 'raw_hwtype' will
expose the raw hardware type.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@de.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.vnet.ibm.com>
Extends the number of ap domains within the zcrypt device driver up to 256.
AP domains in the range 00..255 will be detected.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
This feature extends the generic cryptographic device driver (zcrypt)
with a new capability to service EP11 requests for the Crypto Express4S
card in EP11 (Enterprise PKCS#11 mode) coprocessor mode.
Signed-off-by: Ingo Tuchscherer <ingo.tuchscherer@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Detect external AP bus configuration changes and request
an AP device rescan.
Signed-off-by: Holger Dengler <hd@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Harald Freudenberger