2019-05-27 14:55:01 +08:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0-or-later */
|
2016-08-31 20:02:57 +08:00
|
|
|
/*
|
|
|
|
|
* Crypto engine API
|
|
|
|
|
*
|
|
|
|
|
* Copyright (c) 2016 Baolin Wang <baolin.wang@linaro.org>
|
|
|
|
|
*/
|
|
|
|
|
#ifndef _CRYPTO_ENGINE_H
|
|
|
|
|
#define _CRYPTO_ENGINE_H
|
|
|
|
|
|
|
|
|
|
#include <linux/crypto.h>
|
|
|
|
|
#include <linux/list.h>
|
|
|
|
|
#include <linux/kthread.h>
|
2021-12-10 22:30:09 +08:00
|
|
|
#include <linux/spinlock.h>
|
|
|
|
|
#include <linux/types.h>
|
|
|
|
|
|
2016-08-31 20:02:57 +08:00
|
|
|
#include <crypto/algapi.h>
|
2018-01-27 03:15:30 +08:00
|
|
|
#include <crypto/aead.h>
|
|
|
|
|
#include <crypto/akcipher.h>
|
2016-08-31 20:02:58 +08:00
|
|
|
#include <crypto/hash.h>
|
2018-01-27 03:15:30 +08:00
|
|
|
#include <crypto/skcipher.h>
|
2021-10-20 18:35:34 +08:00
|
|
|
#include <crypto/kpp.h>
|
2016-08-31 20:02:57 +08:00
|
|
|
|
2021-12-10 22:30:09 +08:00
|
|
|
struct device;
|
|
|
|
|
|
2016-08-31 20:02:57 +08:00
|
|
|
#define ENGINE_NAME_LEN 30
|
|
|
|
|
/*
|
|
|
|
|
* struct crypto_engine - crypto hardware engine
|
|
|
|
|
* @name: the engine name
|
|
|
|
|
* @idling: the engine is entering idle state
|
|
|
|
|
* @busy: request pump is busy
|
|
|
|
|
* @running: the engine is on working
|
crypto: engine - support for parallel requests based on retry mechanism
Added support for executing multiple requests, in parallel,
for crypto engine based on a retry mechanism.
If hardware was unable to execute a backlog request, enqueue it
back in front of crypto-engine queue, to keep the order
of requests.
A new variable is added, retry_support (this is to keep the
backward compatibility of crypto-engine) , which keeps track
whether the hardware has support for retry mechanism and,
also, if can run multiple requests.
If do_one_request() returns:
>= 0: hardware executed the request successfully;
< 0: this is the old error path. If hardware has support for retry
mechanism, the request is put back in front of crypto-engine queue.
For backwards compatibility, if the retry support is not available,
the crypto-engine will work as before.
If hardware queue is full (-ENOSPC), requeue request regardless
of MAY_BACKLOG flag.
If hardware throws any other error code (like -EIO, -EINVAL,
-ENOMEM, etc.) only MAY_BACKLOG requests are enqueued back into
crypto-engine's queue, since the others can be dropped.
The new crypto_engine_alloc_init_and_set function, initializes
crypto-engine, sets the maximum size for crypto-engine software
queue (not hardcoded anymore) and the retry_support variable
is set, by default, to false.
On crypto_pump_requests(), if do_one_request() returns >= 0,
a new request is send to hardware, until there is no space in
hardware and do_one_request() returns < 0.
By default, retry_support is false and crypto-engine will
work as before - will send requests to hardware,
one-by-one, on crypto_pump_requests(), and complete it, on
crypto_finalize_request(), and so on.
To support multiple requests, in each driver, retry_support
must be set on true, and if do_one_request() returns an error
the request must not be freed, since it will be enqueued back
into crypto-engine's queue.
When all drivers, that use crypto-engine now, will be updated for
retry mechanism, the retry_support variable can be removed.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-04-28 23:49:04 +08:00
|
|
|
* @retry_support: indication that the hardware allows re-execution
|
|
|
|
|
* of a failed backlog request
|
|
|
|
|
* crypto-engine, in head position to keep order
|
2016-08-31 20:02:57 +08:00
|
|
|
* @list: link with the global crypto engine list
|
2021-05-17 19:02:34 +08:00
|
|
|
* @queue_lock: spinlock to synchronise access to request queue
|
2016-08-31 20:02:57 +08:00
|
|
|
* @queue: the crypto queue of the engine
|
|
|
|
|
* @rt: whether this queue is set to run as a realtime task
|
|
|
|
|
* @prepare_crypt_hardware: a request will soon arrive from the queue
|
|
|
|
|
* so the subsystem requests the driver to prepare the hardware
|
|
|
|
|
* by issuing this call
|
|
|
|
|
* @unprepare_crypt_hardware: there are currently no more requests on the
|
|
|
|
|
* queue so the subsystem notifies the driver that it may relax the
|
|
|
|
|
* hardware by issuing this call
|
2020-04-28 23:49:05 +08:00
|
|
|
* @do_batch_requests: execute a batch of requests. Depends on multiple
|
|
|
|
|
* requests support.
|
2016-10-19 19:54:30 +08:00
|
|
|
* @kworker: kthread worker struct for request pump
|
2016-08-31 20:02:57 +08:00
|
|
|
* @pump_requests: work struct for scheduling work to the request pump
|
|
|
|
|
* @priv_data: the engine private data
|
|
|
|
|
* @cur_req: the current request which is on processing
|
|
|
|
|
*/
|
|
|
|
|
struct crypto_engine {
|
|
|
|
|
char name[ENGINE_NAME_LEN];
|
|
|
|
|
bool idling;
|
|
|
|
|
bool busy;
|
|
|
|
|
bool running;
|
crypto: engine - support for parallel requests based on retry mechanism
Added support for executing multiple requests, in parallel,
for crypto engine based on a retry mechanism.
If hardware was unable to execute a backlog request, enqueue it
back in front of crypto-engine queue, to keep the order
of requests.
A new variable is added, retry_support (this is to keep the
backward compatibility of crypto-engine) , which keeps track
whether the hardware has support for retry mechanism and,
also, if can run multiple requests.
If do_one_request() returns:
>= 0: hardware executed the request successfully;
< 0: this is the old error path. If hardware has support for retry
mechanism, the request is put back in front of crypto-engine queue.
For backwards compatibility, if the retry support is not available,
the crypto-engine will work as before.
If hardware queue is full (-ENOSPC), requeue request regardless
of MAY_BACKLOG flag.
If hardware throws any other error code (like -EIO, -EINVAL,
-ENOMEM, etc.) only MAY_BACKLOG requests are enqueued back into
crypto-engine's queue, since the others can be dropped.
The new crypto_engine_alloc_init_and_set function, initializes
crypto-engine, sets the maximum size for crypto-engine software
queue (not hardcoded anymore) and the retry_support variable
is set, by default, to false.
On crypto_pump_requests(), if do_one_request() returns >= 0,
a new request is send to hardware, until there is no space in
hardware and do_one_request() returns < 0.
By default, retry_support is false and crypto-engine will
work as before - will send requests to hardware,
one-by-one, on crypto_pump_requests(), and complete it, on
crypto_finalize_request(), and so on.
To support multiple requests, in each driver, retry_support
must be set on true, and if do_one_request() returns an error
the request must not be freed, since it will be enqueued back
into crypto-engine's queue.
When all drivers, that use crypto-engine now, will be updated for
retry mechanism, the retry_support variable can be removed.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-04-28 23:49:04 +08:00
|
|
|
|
|
|
|
|
bool retry_support;
|
2016-08-31 20:02:57 +08:00
|
|
|
|
|
|
|
|
struct list_head list;
|
|
|
|
|
spinlock_t queue_lock;
|
|
|
|
|
struct crypto_queue queue;
|
2017-06-06 21:44:16 +08:00
|
|
|
struct device *dev;
|
2016-08-31 20:02:57 +08:00
|
|
|
|
|
|
|
|
bool rt;
|
|
|
|
|
|
|
|
|
|
int (*prepare_crypt_hardware)(struct crypto_engine *engine);
|
|
|
|
|
int (*unprepare_crypt_hardware)(struct crypto_engine *engine);
|
2020-04-28 23:49:05 +08:00
|
|
|
int (*do_batch_requests)(struct crypto_engine *engine);
|
|
|
|
|
|
2016-08-31 20:02:57 +08:00
|
|
|
|
2016-10-19 19:54:30 +08:00
|
|
|
struct kthread_worker *kworker;
|
2016-08-31 20:02:57 +08:00
|
|
|
struct kthread_work pump_requests;
|
|
|
|
|
|
|
|
|
|
void *priv_data;
|
2016-08-31 20:02:58 +08:00
|
|
|
struct crypto_async_request *cur_req;
|
2016-08-31 20:02:57 +08:00
|
|
|
};
|
|
|
|
|
|
2018-01-27 03:15:30 +08:00
|
|
|
/*
|
|
|
|
|
* struct crypto_engine_op - crypto hardware engine operations
|
|
|
|
|
* @prepare__request: do some prepare if need before handle the current request
|
|
|
|
|
* @unprepare_request: undo any work done by prepare_request()
|
|
|
|
|
* @do_one_request: do encryption for current request
|
|
|
|
|
*/
|
|
|
|
|
struct crypto_engine_op {
|
|
|
|
|
int (*prepare_request)(struct crypto_engine *engine,
|
|
|
|
|
void *areq);
|
|
|
|
|
int (*unprepare_request)(struct crypto_engine *engine,
|
|
|
|
|
void *areq);
|
|
|
|
|
int (*do_one_request)(struct crypto_engine *engine,
|
|
|
|
|
void *areq);
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
struct crypto_engine_ctx {
|
|
|
|
|
struct crypto_engine_op op;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
int crypto_transfer_aead_request_to_engine(struct crypto_engine *engine,
|
|
|
|
|
struct aead_request *req);
|
|
|
|
|
int crypto_transfer_akcipher_request_to_engine(struct crypto_engine *engine,
|
|
|
|
|
struct akcipher_request *req);
|
2016-08-31 20:02:58 +08:00
|
|
|
int crypto_transfer_hash_request_to_engine(struct crypto_engine *engine,
|
2018-01-27 03:15:30 +08:00
|
|
|
struct ahash_request *req);
|
2021-10-20 18:35:34 +08:00
|
|
|
int crypto_transfer_kpp_request_to_engine(struct crypto_engine *engine,
|
|
|
|
|
struct kpp_request *req);
|
2018-01-27 03:15:30 +08:00
|
|
|
int crypto_transfer_skcipher_request_to_engine(struct crypto_engine *engine,
|
|
|
|
|
struct skcipher_request *req);
|
|
|
|
|
void crypto_finalize_aead_request(struct crypto_engine *engine,
|
|
|
|
|
struct aead_request *req, int err);
|
|
|
|
|
void crypto_finalize_akcipher_request(struct crypto_engine *engine,
|
|
|
|
|
struct akcipher_request *req, int err);
|
2016-08-31 20:02:58 +08:00
|
|
|
void crypto_finalize_hash_request(struct crypto_engine *engine,
|
|
|
|
|
struct ahash_request *req, int err);
|
2021-10-20 18:35:34 +08:00
|
|
|
void crypto_finalize_kpp_request(struct crypto_engine *engine,
|
|
|
|
|
struct kpp_request *req, int err);
|
2018-01-27 03:15:30 +08:00
|
|
|
void crypto_finalize_skcipher_request(struct crypto_engine *engine,
|
|
|
|
|
struct skcipher_request *req, int err);
|
2016-08-31 20:02:57 +08:00
|
|
|
int crypto_engine_start(struct crypto_engine *engine);
|
|
|
|
|
int crypto_engine_stop(struct crypto_engine *engine);
|
|
|
|
|
struct crypto_engine *crypto_engine_alloc_init(struct device *dev, bool rt);
|
crypto: engine - support for parallel requests based on retry mechanism
Added support for executing multiple requests, in parallel,
for crypto engine based on a retry mechanism.
If hardware was unable to execute a backlog request, enqueue it
back in front of crypto-engine queue, to keep the order
of requests.
A new variable is added, retry_support (this is to keep the
backward compatibility of crypto-engine) , which keeps track
whether the hardware has support for retry mechanism and,
also, if can run multiple requests.
If do_one_request() returns:
>= 0: hardware executed the request successfully;
< 0: this is the old error path. If hardware has support for retry
mechanism, the request is put back in front of crypto-engine queue.
For backwards compatibility, if the retry support is not available,
the crypto-engine will work as before.
If hardware queue is full (-ENOSPC), requeue request regardless
of MAY_BACKLOG flag.
If hardware throws any other error code (like -EIO, -EINVAL,
-ENOMEM, etc.) only MAY_BACKLOG requests are enqueued back into
crypto-engine's queue, since the others can be dropped.
The new crypto_engine_alloc_init_and_set function, initializes
crypto-engine, sets the maximum size for crypto-engine software
queue (not hardcoded anymore) and the retry_support variable
is set, by default, to false.
On crypto_pump_requests(), if do_one_request() returns >= 0,
a new request is send to hardware, until there is no space in
hardware and do_one_request() returns < 0.
By default, retry_support is false and crypto-engine will
work as before - will send requests to hardware,
one-by-one, on crypto_pump_requests(), and complete it, on
crypto_finalize_request(), and so on.
To support multiple requests, in each driver, retry_support
must be set on true, and if do_one_request() returns an error
the request must not be freed, since it will be enqueued back
into crypto-engine's queue.
When all drivers, that use crypto-engine now, will be updated for
retry mechanism, the retry_support variable can be removed.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-04-28 23:49:04 +08:00
|
|
|
struct crypto_engine *crypto_engine_alloc_init_and_set(struct device *dev,
|
|
|
|
|
bool retry_support,
|
2020-04-28 23:49:05 +08:00
|
|
|
int (*cbk_do_batch)(struct crypto_engine *engine),
|
crypto: engine - support for parallel requests based on retry mechanism
Added support for executing multiple requests, in parallel,
for crypto engine based on a retry mechanism.
If hardware was unable to execute a backlog request, enqueue it
back in front of crypto-engine queue, to keep the order
of requests.
A new variable is added, retry_support (this is to keep the
backward compatibility of crypto-engine) , which keeps track
whether the hardware has support for retry mechanism and,
also, if can run multiple requests.
If do_one_request() returns:
>= 0: hardware executed the request successfully;
< 0: this is the old error path. If hardware has support for retry
mechanism, the request is put back in front of crypto-engine queue.
For backwards compatibility, if the retry support is not available,
the crypto-engine will work as before.
If hardware queue is full (-ENOSPC), requeue request regardless
of MAY_BACKLOG flag.
If hardware throws any other error code (like -EIO, -EINVAL,
-ENOMEM, etc.) only MAY_BACKLOG requests are enqueued back into
crypto-engine's queue, since the others can be dropped.
The new crypto_engine_alloc_init_and_set function, initializes
crypto-engine, sets the maximum size for crypto-engine software
queue (not hardcoded anymore) and the retry_support variable
is set, by default, to false.
On crypto_pump_requests(), if do_one_request() returns >= 0,
a new request is send to hardware, until there is no space in
hardware and do_one_request() returns < 0.
By default, retry_support is false and crypto-engine will
work as before - will send requests to hardware,
one-by-one, on crypto_pump_requests(), and complete it, on
crypto_finalize_request(), and so on.
To support multiple requests, in each driver, retry_support
must be set on true, and if do_one_request() returns an error
the request must not be freed, since it will be enqueued back
into crypto-engine's queue.
When all drivers, that use crypto-engine now, will be updated for
retry mechanism, the retry_support variable can be removed.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-04-28 23:49:04 +08:00
|
|
|
bool rt, int qlen);
|
2016-08-31 20:02:57 +08:00
|
|
|
int crypto_engine_exit(struct crypto_engine *engine);
|
|
|
|
|
|
|
|
|
|
#endif /* _CRYPTO_ENGINE_H */
|