mtd: nand: Introduce the ECC engine framework

Create a generic ECC engine framework. This is a base to instantiate ECC
engine objects.

If we really want to be generic, bindings must evolve, so here is the
new logic. The following three properties are mutually exclusive:
- The nand-no-ecc-engine boolean property is set and there is no
  ECC engine to retrieve.
- The nand-use-soft-ecc-engine boolean property is set and the core
  will force using the use of software correction.
- There is a nand-ecc-engine property pointing at a node which will
  act as ECC engine.

It the later case, the property may reference:
- The NAND chip node itself (for the on-die ECC case).
- The parent node if the NAND controller embeds an ECC engine.
- Any other node being an external ECC controller as well.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Link: https://lore.kernel.org/linux-mtd/20200827085208.16276-9-miquel.raynal@bootlin.com
This commit is contained in:
Miquel Raynal 2020-08-27 10:51:56 +02:00
parent b440915af2
commit a8c7ffdb5f
4 changed files with 608 additions and 0 deletions

View File

@ -9,4 +9,12 @@ source "drivers/mtd/nand/onenand/Kconfig"
source "drivers/mtd/nand/raw/Kconfig"
source "drivers/mtd/nand/spi/Kconfig"
menu "ECC engine support"
config MTD_NAND_ECC
bool
depends on MTD_NAND_CORE
endmenu
endmenu

View File

@ -6,3 +6,5 @@ obj-$(CONFIG_MTD_NAND_CORE) += nandcore.o
obj-y += onenand/
obj-y += raw/
obj-y += spi/
nandcore-$(CONFIG_MTD_NAND_ECC) += ecc.o

484
drivers/mtd/nand/ecc.c Normal file
View File

@ -0,0 +1,484 @@
// SPDX-License-Identifier: GPL-2.0+
/*
* Generic Error-Correcting Code (ECC) engine
*
* Copyright (C) 2019 Macronix
* Author:
* Miquèl RAYNAL <miquel.raynal@bootlin.com>
*
*
* This file describes the abstraction of any NAND ECC engine. It has been
* designed to fit most cases, including parallel NANDs and SPI-NANDs.
*
* There are three main situations where instantiating this ECC engine makes
* sense:
* - external: The ECC engine is outside the NAND pipeline, typically this
* is a software ECC engine, or an hardware engine that is
* outside the NAND controller pipeline.
* - pipelined: The ECC engine is inside the NAND pipeline, ie. on the
* controller's side. This is the case of most of the raw NAND
* controllers. In the pipeline case, the ECC bytes are
* generated/data corrected on the fly when a page is
* written/read.
* - ondie: The ECC engine is inside the NAND pipeline, on the chip's side.
* Some NAND chips can correct themselves the data.
*
* Besides the initial setup and final cleanups, the interfaces are rather
* simple:
* - prepare: Prepare an I/O request. Enable/disable the ECC engine based on
* the I/O request type. In case of software correction or external
* engine, this step may involve to derive the ECC bytes and place
* them in the OOB area before a write.
* - finish: Finish an I/O request. Correct the data in case of a read
* request and report the number of corrected bits/uncorrectable
* errors. Most likely empty for write operations, unless you have
* hardware specific stuff to do, like shutting down the engine to
* save power.
*
* The I/O request should be enclosed in a prepare()/finish() pair of calls
* and will behave differently depending on the requested I/O type:
* - raw: Correction disabled
* - ecc: Correction enabled
*
* The request direction is impacting the logic as well:
* - read: Load data from the NAND chip
* - write: Store data in the NAND chip
*
* Mixing all this combinations together gives the following behavior.
* Those are just examples, drivers are free to add custom steps in their
* prepare/finish hook.
*
* [external ECC engine]
* - external + prepare + raw + read: do nothing
* - external + finish + raw + read: do nothing
* - external + prepare + raw + write: do nothing
* - external + finish + raw + write: do nothing
* - external + prepare + ecc + read: do nothing
* - external + finish + ecc + read: calculate expected ECC bytes, extract
* ECC bytes from OOB buffer, correct
* and report any bitflip/error
* - external + prepare + ecc + write: calculate ECC bytes and store them at
* the right place in the OOB buffer based
* on the OOB layout
* - external + finish + ecc + write: do nothing
*
* [pipelined ECC engine]
* - pipelined + prepare + raw + read: disable the controller's ECC engine if
* activated
* - pipelined + finish + raw + read: do nothing
* - pipelined + prepare + raw + write: disable the controller's ECC engine if
* activated
* - pipelined + finish + raw + write: do nothing
* - pipelined + prepare + ecc + read: enable the controller's ECC engine if
* deactivated
* - pipelined + finish + ecc + read: check the status, report any
* error/bitflip
* - pipelined + prepare + ecc + write: enable the controller's ECC engine if
* deactivated
* - pipelined + finish + ecc + write: do nothing
*
* [ondie ECC engine]
* - ondie + prepare + raw + read: send commands to disable the on-chip ECC
* engine if activated
* - ondie + finish + raw + read: do nothing
* - ondie + prepare + raw + write: send commands to disable the on-chip ECC
* engine if activated
* - ondie + finish + raw + write: do nothing
* - ondie + prepare + ecc + read: send commands to enable the on-chip ECC
* engine if deactivated
* - ondie + finish + ecc + read: send commands to check the status, report
* any error/bitflip
* - ondie + prepare + ecc + write: send commands to enable the on-chip ECC
* engine if deactivated
* - ondie + finish + ecc + write: do nothing
*/
#include <linux/module.h>
#include <linux/mtd/nand.h>
/**
* nand_ecc_init_ctx - Init the ECC engine context
* @nand: the NAND device
*
* On success, the caller is responsible of calling @nand_ecc_cleanup_ctx().
*/
int nand_ecc_init_ctx(struct nand_device *nand)
{
if (!nand->ecc.engine->ops->init_ctx)
return 0;
return nand->ecc.engine->ops->init_ctx(nand);
}
EXPORT_SYMBOL(nand_ecc_init_ctx);
/**
* nand_ecc_cleanup_ctx - Cleanup the ECC engine context
* @nand: the NAND device
*/
void nand_ecc_cleanup_ctx(struct nand_device *nand)
{
if (nand->ecc.engine->ops->cleanup_ctx)
nand->ecc.engine->ops->cleanup_ctx(nand);
}
EXPORT_SYMBOL(nand_ecc_cleanup_ctx);
/**
* nand_ecc_prepare_io_req - Prepare an I/O request
* @nand: the NAND device
* @req: the I/O request
*/
int nand_ecc_prepare_io_req(struct nand_device *nand,
struct nand_page_io_req *req)
{
if (!nand->ecc.engine->ops->prepare_io_req)
return 0;
return nand->ecc.engine->ops->prepare_io_req(nand, req);
}
EXPORT_SYMBOL(nand_ecc_prepare_io_req);
/**
* nand_ecc_finish_io_req - Finish an I/O request
* @nand: the NAND device
* @req: the I/O request
*/
int nand_ecc_finish_io_req(struct nand_device *nand,
struct nand_page_io_req *req)
{
if (!nand->ecc.engine->ops->finish_io_req)
return 0;
return nand->ecc.engine->ops->finish_io_req(nand, req);
}
EXPORT_SYMBOL(nand_ecc_finish_io_req);
/* Define default OOB placement schemes for large and small page devices */
static int nand_ooblayout_ecc_sp(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
struct nand_device *nand = mtd_to_nanddev(mtd);
unsigned int total_ecc_bytes = nand->ecc.ctx.total;
if (section > 1)
return -ERANGE;
if (!section) {
oobregion->offset = 0;
if (mtd->oobsize == 16)
oobregion->length = 4;
else
oobregion->length = 3;
} else {
if (mtd->oobsize == 8)
return -ERANGE;
oobregion->offset = 6;
oobregion->length = total_ecc_bytes - 4;
}
return 0;
}
static int nand_ooblayout_free_sp(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
if (section > 1)
return -ERANGE;
if (mtd->oobsize == 16) {
if (section)
return -ERANGE;
oobregion->length = 8;
oobregion->offset = 8;
} else {
oobregion->length = 2;
if (!section)
oobregion->offset = 3;
else
oobregion->offset = 6;
}
return 0;
}
static const struct mtd_ooblayout_ops nand_ooblayout_sp_ops = {
.ecc = nand_ooblayout_ecc_sp,
.free = nand_ooblayout_free_sp,
};
const struct mtd_ooblayout_ops *nand_get_small_page_ooblayout(void)
{
return &nand_ooblayout_sp_ops;
}
EXPORT_SYMBOL_GPL(nand_get_small_page_ooblayout);
static int nand_ooblayout_ecc_lp(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
struct nand_device *nand = mtd_to_nanddev(mtd);
unsigned int total_ecc_bytes = nand->ecc.ctx.total;
if (section || !total_ecc_bytes)
return -ERANGE;
oobregion->length = total_ecc_bytes;
oobregion->offset = mtd->oobsize - oobregion->length;
return 0;
}
static int nand_ooblayout_free_lp(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
struct nand_device *nand = mtd_to_nanddev(mtd);
unsigned int total_ecc_bytes = nand->ecc.ctx.total;
if (section)
return -ERANGE;
oobregion->length = mtd->oobsize - total_ecc_bytes - 2;
oobregion->offset = 2;
return 0;
}
static const struct mtd_ooblayout_ops nand_ooblayout_lp_ops = {
.ecc = nand_ooblayout_ecc_lp,
.free = nand_ooblayout_free_lp,
};
const struct mtd_ooblayout_ops *nand_get_large_page_ooblayout(void)
{
return &nand_ooblayout_lp_ops;
}
EXPORT_SYMBOL_GPL(nand_get_large_page_ooblayout);
/*
* Support the old "large page" layout used for 1-bit Hamming ECC where ECC
* are placed at a fixed offset.
*/
static int nand_ooblayout_ecc_lp_hamming(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
struct nand_device *nand = mtd_to_nanddev(mtd);
unsigned int total_ecc_bytes = nand->ecc.ctx.total;
if (section)
return -ERANGE;
switch (mtd->oobsize) {
case 64:
oobregion->offset = 40;
break;
case 128:
oobregion->offset = 80;
break;
default:
return -EINVAL;
}
oobregion->length = total_ecc_bytes;
if (oobregion->offset + oobregion->length > mtd->oobsize)
return -ERANGE;
return 0;
}
static int nand_ooblayout_free_lp_hamming(struct mtd_info *mtd, int section,
struct mtd_oob_region *oobregion)
{
struct nand_device *nand = mtd_to_nanddev(mtd);
unsigned int total_ecc_bytes = nand->ecc.ctx.total;
int ecc_offset = 0;
if (section < 0 || section > 1)
return -ERANGE;
switch (mtd->oobsize) {
case 64:
ecc_offset = 40;
break;
case 128:
ecc_offset = 80;
break;
default:
return -EINVAL;
}
if (section == 0) {
oobregion->offset = 2;
oobregion->length = ecc_offset - 2;
} else {
oobregion->offset = ecc_offset + total_ecc_bytes;
oobregion->length = mtd->oobsize - oobregion->offset;
}
return 0;
}
static const struct mtd_ooblayout_ops nand_ooblayout_lp_hamming_ops = {
.ecc = nand_ooblayout_ecc_lp_hamming,
.free = nand_ooblayout_free_lp_hamming,
};
const struct mtd_ooblayout_ops *nand_get_large_page_hamming_ooblayout(void)
{
return &nand_ooblayout_lp_hamming_ops;
}
EXPORT_SYMBOL_GPL(nand_get_large_page_hamming_ooblayout);
static enum nand_ecc_engine_type
of_get_nand_ecc_engine_type(struct device_node *np)
{
struct device_node *eng_np;
if (of_property_read_bool(np, "nand-no-ecc-engine"))
return NAND_ECC_ENGINE_TYPE_NONE;
if (of_property_read_bool(np, "nand-use-soft-ecc-engine"))
return NAND_ECC_ENGINE_TYPE_SOFT;
eng_np = of_parse_phandle(np, "nand-ecc-engine", 0);
of_node_put(eng_np);
if (eng_np) {
if (eng_np == np)
return NAND_ECC_ENGINE_TYPE_ON_DIE;
else
return NAND_ECC_ENGINE_TYPE_ON_HOST;
}
return NAND_ECC_ENGINE_TYPE_INVALID;
}
static const char * const nand_ecc_placement[] = {
[NAND_ECC_PLACEMENT_OOB] = "oob",
[NAND_ECC_PLACEMENT_INTERLEAVED] = "interleaved",
};
static enum nand_ecc_placement of_get_nand_ecc_placement(struct device_node *np)
{
enum nand_ecc_placement placement;
const char *pm;
int err;
err = of_property_read_string(np, "nand-ecc-placement", &pm);
if (!err) {
for (placement = NAND_ECC_PLACEMENT_OOB;
placement < ARRAY_SIZE(nand_ecc_placement); placement++) {
if (!strcasecmp(pm, nand_ecc_placement[placement]))
return placement;
}
}
return NAND_ECC_PLACEMENT_UNKNOWN;
}
static const char * const nand_ecc_algos[] = {
[NAND_ECC_ALGO_HAMMING] = "hamming",
[NAND_ECC_ALGO_BCH] = "bch",
[NAND_ECC_ALGO_RS] = "rs",
};
static enum nand_ecc_algo of_get_nand_ecc_algo(struct device_node *np)
{
enum nand_ecc_algo ecc_algo;
const char *pm;
int err;
err = of_property_read_string(np, "nand-ecc-algo", &pm);
if (!err) {
for (ecc_algo = NAND_ECC_ALGO_HAMMING;
ecc_algo < ARRAY_SIZE(nand_ecc_algos);
ecc_algo++) {
if (!strcasecmp(pm, nand_ecc_algos[ecc_algo]))
return ecc_algo;
}
}
return NAND_ECC_ALGO_UNKNOWN;
}
static int of_get_nand_ecc_step_size(struct device_node *np)
{
int ret;
u32 val;
ret = of_property_read_u32(np, "nand-ecc-step-size", &val);
return ret ? ret : val;
}
static int of_get_nand_ecc_strength(struct device_node *np)
{
int ret;
u32 val;
ret = of_property_read_u32(np, "nand-ecc-strength", &val);
return ret ? ret : val;
}
void of_get_nand_ecc_user_config(struct nand_device *nand)
{
struct device_node *dn = nanddev_get_of_node(nand);
int strength, size;
nand->ecc.user_conf.engine_type = of_get_nand_ecc_engine_type(dn);
nand->ecc.user_conf.algo = of_get_nand_ecc_algo(dn);
nand->ecc.user_conf.placement = of_get_nand_ecc_placement(dn);
strength = of_get_nand_ecc_strength(dn);
if (strength >= 0)
nand->ecc.user_conf.strength = strength;
size = of_get_nand_ecc_step_size(dn);
if (size >= 0)
nand->ecc.user_conf.step_size = size;
if (of_property_read_bool(dn, "nand-ecc-maximize"))
nand->ecc.user_conf.flags |= NAND_ECC_MAXIMIZE_STRENGTH;
}
EXPORT_SYMBOL(of_get_nand_ecc_user_config);
/**
* nand_ecc_is_strong_enough - Check if the chip configuration meets the
* datasheet requirements.
*
* @nand: Device to check
*
* If our configuration corrects A bits per B bytes and the minimum
* required correction level is X bits per Y bytes, then we must ensure
* both of the following are true:
*
* (1) A / B >= X / Y
* (2) A >= X
*
* Requirement (1) ensures we can correct for the required bitflip density.
* Requirement (2) ensures we can correct even when all bitflips are clumped
* in the same sector.
*/
bool nand_ecc_is_strong_enough(struct nand_device *nand)
{
const struct nand_ecc_props *reqs = nanddev_get_ecc_requirements(nand);
const struct nand_ecc_props *conf = nanddev_get_ecc_conf(nand);
struct mtd_info *mtd = nanddev_to_mtd(nand);
int corr, ds_corr;
if (conf->step_size == 0 || reqs->step_size == 0)
/* Not enough information */
return true;
/*
* We get the number of corrected bits per page to compare
* the correction density.
*/
corr = (mtd->writesize * conf->strength) / conf->step_size;
ds_corr = (mtd->writesize * reqs->strength) / reqs->step_size;
return corr >= ds_corr && conf->strength >= reqs->strength;
}
EXPORT_SYMBOL(nand_ecc_is_strong_enough);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Miquel Raynal <miquel.raynal@bootlin.com>");
MODULE_DESCRIPTION("Generic ECC engine");

View File

@ -127,6 +127,40 @@ struct nand_page_io_req {
int mode;
};
const struct mtd_ooblayout_ops *nand_get_small_page_ooblayout(void);
const struct mtd_ooblayout_ops *nand_get_large_page_ooblayout(void);
const struct mtd_ooblayout_ops *nand_get_large_page_hamming_ooblayout(void);
/**
* enum nand_ecc_engine_type - NAND ECC engine type
* @NAND_ECC_ENGINE_TYPE_INVALID: Invalid value
* @NAND_ECC_ENGINE_TYPE_NONE: No ECC correction
* @NAND_ECC_ENGINE_TYPE_SOFT: Software ECC correction
* @NAND_ECC_ENGINE_TYPE_ON_HOST: On host hardware ECC correction
* @NAND_ECC_ENGINE_TYPE_ON_DIE: On chip hardware ECC correction
*/
enum nand_ecc_engine_type {
NAND_ECC_ENGINE_TYPE_INVALID,
NAND_ECC_ENGINE_TYPE_NONE,
NAND_ECC_ENGINE_TYPE_SOFT,
NAND_ECC_ENGINE_TYPE_ON_HOST,
NAND_ECC_ENGINE_TYPE_ON_DIE,
};
/**
* enum nand_ecc_placement - NAND ECC bytes placement
* @NAND_ECC_PLACEMENT_UNKNOWN: The actual position of the ECC bytes is unknown
* @NAND_ECC_PLACEMENT_OOB: The ECC bytes are located in the OOB area
* @NAND_ECC_PLACEMENT_INTERLEAVED: Syndrome layout, there are ECC bytes
* interleaved with regular data in the main
* area
*/
enum nand_ecc_placement {
NAND_ECC_PLACEMENT_UNKNOWN,
NAND_ECC_PLACEMENT_OOB,
NAND_ECC_PLACEMENT_INTERLEAVED,
};
/**
* enum nand_ecc_algo - NAND ECC algorithm
* @NAND_ECC_ALGO_UNKNOWN: Unknown algorithm
@ -143,16 +177,27 @@ enum nand_ecc_algo {
/**
* struct nand_ecc_props - NAND ECC properties
* @engine_type: ECC engine type
* @placement: OOB placement (if relevant)
* @algo: ECC algorithm (if relevant)
* @strength: ECC strength
* @step_size: Number of bytes per step
* @flags: Misc properties
*/
struct nand_ecc_props {
enum nand_ecc_engine_type engine_type;
enum nand_ecc_placement placement;
enum nand_ecc_algo algo;
unsigned int strength;
unsigned int step_size;
unsigned int flags;
};
#define NAND_ECCREQ(str, stp) { .strength = (str), .step_size = (stp) }
/* NAND ECC misc flags */
#define NAND_ECC_MAXIMIZE_STRENGTH BIT(0)
/**
* struct nand_bbt - bad block table object
* @cache: in memory BBT cache
@ -183,6 +228,75 @@ struct nand_ops {
bool (*isbad)(struct nand_device *nand, const struct nand_pos *pos);
};
/**
* struct nand_ecc_context - Context for the ECC engine
* @conf: basic ECC engine parameters
* @total: total number of bytes used for storing ECC codes, this is used by
* generic OOB layouts
* @priv: ECC engine driver private data
*/
struct nand_ecc_context {
struct nand_ecc_props conf;
unsigned int total;
void *priv;
};
/**
* struct nand_ecc_engine_ops - ECC engine operations
* @init_ctx: given a desired user configuration for the pointed NAND device,
* requests the ECC engine driver to setup a configuration with
* values it supports.
* @cleanup_ctx: clean the context initialized by @init_ctx.
* @prepare_io_req: is called before reading/writing a page to prepare the I/O
* request to be performed with ECC correction.
* @finish_io_req: is called after reading/writing a page to terminate the I/O
* request and ensure proper ECC correction.
*/
struct nand_ecc_engine_ops {
int (*init_ctx)(struct nand_device *nand);
void (*cleanup_ctx)(struct nand_device *nand);
int (*prepare_io_req)(struct nand_device *nand,
struct nand_page_io_req *req);
int (*finish_io_req)(struct nand_device *nand,
struct nand_page_io_req *req);
};
/**
* struct nand_ecc_engine - ECC engine abstraction for NAND devices
* @ops: ECC engine operations
*/
struct nand_ecc_engine {
struct nand_ecc_engine_ops *ops;
};
void of_get_nand_ecc_user_config(struct nand_device *nand);
int nand_ecc_init_ctx(struct nand_device *nand);
void nand_ecc_cleanup_ctx(struct nand_device *nand);
int nand_ecc_prepare_io_req(struct nand_device *nand,
struct nand_page_io_req *req);
int nand_ecc_finish_io_req(struct nand_device *nand,
struct nand_page_io_req *req);
bool nand_ecc_is_strong_enough(struct nand_device *nand);
/**
* struct nand_ecc - Information relative to the ECC
* @defaults: Default values, depend on the underlying subsystem
* @requirements: ECC requirements from the NAND chip perspective
* @user_conf: User desires in terms of ECC parameters
* @ctx: ECC context for the ECC engine, derived from the device @requirements
* the @user_conf and the @defaults
* @ondie_engine: On-die ECC engine reference, if any
* @engine: ECC engine actually bound
*/
struct nand_ecc {
struct nand_ecc_props defaults;
struct nand_ecc_props requirements;
struct nand_ecc_props user_conf;
struct nand_ecc_context ctx;
struct nand_ecc_engine *ondie_engine;
struct nand_ecc_engine *engine;
};
/**
* struct nand_device - NAND device
* @mtd: MTD instance attached to the NAND device