UbiquitousOS
/
OpenCloudOS-Kernel
mirror of https://gitee.com/OpenCloudOS/OpenCloudOS-Kernel.git
11
0
Fork 0
Code Issues Projects Releases Wiki Activity
OpenCloudOS-Kernel/drivers/lightnvm/core.c

1126 lines
24 KiB
C
Raw Normal View History

lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
/*
* Copyright (C) 2015 IT University of Copenhagen. All rights reserved.
* Initial release: Matias Bjorling <m@bjorling.me>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License version
* 2 as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; see the file COPYING. If not, write to
* the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139,
* USA.
*
*/
#include <linux/blkdev.h>
#include <linux/blk-mq.h>
#include <linux/list.h>
#include <linux/types.h>
#include <linux/sem.h>
#include <linux/bitmap.h>
#include <linux/module.h>
#include <linux/miscdevice.h>
#include <linux/lightnvm.h>
lightnvm: refactor end_io functions for sync To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:21 +08:00
#include <linux/sched/sysctl.h>
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
#include <uapi/linux/lightnvm.h>
static LIST_HEAD(nvm_targets);
static LIST_HEAD(nvm_mgrs);
static LIST_HEAD(nvm_devices);
static DECLARE_RWSEM(nvm_lock);
static struct nvm_tgt_type *nvm_find_target_type(const char *name)
{
struct nvm_tgt_type *tt;
list_for_each_entry(tt, &nvm_targets, list)
if (!strcmp(name, tt->name))
return tt;
return NULL;
}
int nvm_register_target(struct nvm_tgt_type *tt)
{
int ret = 0;
down_write(&nvm_lock);
if (nvm_find_target_type(tt->name))
ret = -EEXIST;
else
list_add(&tt->list, &nvm_targets);
up_write(&nvm_lock);
return ret;
}
EXPORT_SYMBOL(nvm_register_target);
void nvm_unregister_target(struct nvm_tgt_type *tt)
{
if (!tt)
return;
down_write(&nvm_lock);
list_del(&tt->list);
up_write(&nvm_lock);
}
EXPORT_SYMBOL(nvm_unregister_target);
void *nvm_dev_dma_alloc(struct nvm_dev *dev, gfp_t mem_flags,
dma_addr_t *dma_handler)
{
lightnvm: replace req queue with nvmdev for lld In the case where a request queue is passed to the low lever lightnvm device drive integration, the device driver might pass its admin commands through another queue. Instead pass nvm_dev, and let the low level drive the appropriate queue. Reported-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:48 +08:00
return dev->ops->dev_dma_alloc(dev, dev->ppalist_pool, mem_flags,
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
dma_handler);
}
EXPORT_SYMBOL(nvm_dev_dma_alloc);
void nvm_dev_dma_free(struct nvm_dev *dev, void *ppa_list,
dma_addr_t dma_handler)
{
dev->ops->dev_dma_free(dev->ppalist_pool, ppa_list, dma_handler);
}
EXPORT_SYMBOL(nvm_dev_dma_free);
static struct nvmm_type *nvm_find_mgr_type(const char *name)
{
struct nvmm_type *mt;
list_for_each_entry(mt, &nvm_mgrs, list)
if (!strcmp(name, mt->name))
return mt;
return NULL;
}
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
struct nvmm_type *nvm_init_mgr(struct nvm_dev *dev)
{
struct nvmm_type *mt;
int ret;
lockdep_assert_held(&nvm_lock);
list_for_each_entry(mt, &nvm_mgrs, list) {
ret = mt->register_mgr(dev);
if (ret < 0) {
pr_err("nvm: media mgr failed to init (%d) on dev %s\n",
ret, dev->name);
return NULL; /* initialization failed */
} else if (ret > 0)
return mt;
}
return NULL;
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
int nvm_register_mgr(struct nvmm_type *mt)
{
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
struct nvm_dev *dev;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
int ret = 0;
down_write(&nvm_lock);
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
if (nvm_find_mgr_type(mt->name)) {
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
ret = -EEXIST;
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
goto finish;
} else {
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
list_add(&mt->list, &nvm_mgrs);
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
}
/* try to register media mgr if any device have none configured */
list_for_each_entry(dev, &nvm_devices, devices) {
if (dev->mt)
continue;
dev->mt = nvm_init_mgr(dev);
}
finish:
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
up_write(&nvm_lock);
return ret;
}
EXPORT_SYMBOL(nvm_register_mgr);
void nvm_unregister_mgr(struct nvmm_type *mt)
{
if (!mt)
return;
down_write(&nvm_lock);
list_del(&mt->list);
up_write(&nvm_lock);
}
EXPORT_SYMBOL(nvm_unregister_mgr);
static struct nvm_dev *nvm_find_nvm_dev(const char *name)
{
struct nvm_dev *dev;
list_for_each_entry(dev, &nvm_devices, devices)
if (!strcmp(name, dev->name))
return dev;
return NULL;
}
lightnvm: manage open and closed blocks separately LightNVM targets need to know the state of the flash block when doing flash optimizations. An example is implementing a write buffer to respect the flash page size. Currently, block state is not accounted for; the media manager only differentiates among free, bad and in-use blocks. This patch adds the logic in the generic media manager to enable targets manage blocks into open and close separately, and it implements such management in rrpc. It also adds a set of flags to describe the state of the block (open, closed, free, bad). In order to avoid taking two locks (nvm_lun and rrpc_lun) consecutively, we introduce lockless get_/put_block primitives so that the open and close list locks and future common logic is handled within the nvm_lun lock. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:33 +08:00
struct nvm_block *nvm_get_blk_unlocked(struct nvm_dev *dev, struct nvm_lun *lun,
unsigned long flags)
{
return dev->mt->get_blk_unlocked(dev, lun, flags);
}
EXPORT_SYMBOL(nvm_get_blk_unlocked);
/* Assumes that all valid pages have already been moved on release to bm */
void nvm_put_blk_unlocked(struct nvm_dev *dev, struct nvm_block *blk)
{
return dev->mt->put_blk_unlocked(dev, blk);
}
EXPORT_SYMBOL(nvm_put_blk_unlocked);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
struct nvm_block *nvm_get_blk(struct nvm_dev *dev, struct nvm_lun *lun,
unsigned long flags)
{
return dev->mt->get_blk(dev, lun, flags);
}
EXPORT_SYMBOL(nvm_get_blk);
/* Assumes that all valid pages have already been moved on release to bm */
void nvm_put_blk(struct nvm_dev *dev, struct nvm_block *blk)
{
return dev->mt->put_blk(dev, blk);
}
EXPORT_SYMBOL(nvm_put_blk);
int nvm_submit_io(struct nvm_dev *dev, struct nvm_rq *rqd)
{
return dev->mt->submit_io(dev, rqd);
}
EXPORT_SYMBOL(nvm_submit_io);
int nvm_erase_blk(struct nvm_dev *dev, struct nvm_block *blk)
{
return dev->mt->erase_blk(dev, blk, 0);
}
EXPORT_SYMBOL(nvm_erase_blk);
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
void nvm_addr_to_generic_mode(struct nvm_dev *dev, struct nvm_rq *rqd)
{
int i;
if (rqd->nr_pages > 1) {
for (i = 0; i < rqd->nr_pages; i++)
rqd->ppa_list[i] = dev_to_generic_addr(dev,
rqd->ppa_list[i]);
} else {
rqd->ppa_addr = dev_to_generic_addr(dev, rqd->ppa_addr);
}
}
EXPORT_SYMBOL(nvm_addr_to_generic_mode);
void nvm_generic_to_addr_mode(struct nvm_dev *dev, struct nvm_rq *rqd)
{
int i;
if (rqd->nr_pages > 1) {
for (i = 0; i < rqd->nr_pages; i++)
rqd->ppa_list[i] = generic_to_dev_addr(dev,
rqd->ppa_list[i]);
} else {
rqd->ppa_addr = generic_to_dev_addr(dev, rqd->ppa_addr);
}
}
EXPORT_SYMBOL(nvm_generic_to_addr_mode);
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
int nvm_set_rqd_ppalist(struct nvm_dev *dev, struct nvm_rq *rqd,
struct ppa_addr *ppas, int nr_ppas)
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
{
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
int i, plane_cnt, pl_idx;
if (dev->plane_mode == NVM_PLANE_SINGLE && nr_ppas == 1) {
rqd->nr_pages = 1;
rqd->ppa_addr = ppas[0];
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
return 0;
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
}
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
plane_cnt = (1 << dev->plane_mode);
rqd->nr_pages = plane_cnt * nr_ppas;
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
if (dev->ops->max_phys_sect < rqd->nr_pages)
return -EINVAL;
rqd->ppa_list = nvm_dev_dma_alloc(dev, GFP_KERNEL, &rqd->dma_ppa_list);
if (!rqd->ppa_list) {
pr_err("nvm: failed to allocate dma memory\n");
return -ENOMEM;
}
lightnvm: sectors first in ppa list The Westlake controller requires that the PPA list has sectors defined sequentially. Currently, the PPA list is created with planes first, then sectors. Change this to sectors first, then planes. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:26 +08:00
for (pl_idx = 0; pl_idx < plane_cnt; pl_idx++) {
for (i = 0; i < nr_ppas; i++) {
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
ppas[i].g.pl = pl_idx;
lightnvm: sectors first in ppa list The Westlake controller requires that the PPA list has sectors defined sequentially. Currently, the PPA list is created with planes first, then sectors. Change this to sectors first, then planes. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:26 +08:00
rqd->ppa_list[(pl_idx * nr_ppas) + i] = ppas[i];
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
}
}
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
return 0;
}
EXPORT_SYMBOL(nvm_set_rqd_ppalist);
void nvm_free_rqd_ppalist(struct nvm_dev *dev, struct nvm_rq *rqd)
{
if (!rqd->ppa_list)
return;
nvm_dev_dma_free(dev, rqd->ppa_list, rqd->dma_ppa_list);
}
EXPORT_SYMBOL(nvm_free_rqd_ppalist);
lightnvm: support multiple ppas in nvm_erase_ppa Sometimes a user want to erase multiple PPAs at the same time. Extend nvm_erase_ppa to take multiple ppas and number of ppas to be erased. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:28 +08:00
int nvm_erase_ppa(struct nvm_dev *dev, struct ppa_addr *ppas, int nr_ppas)
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
{
struct nvm_rq rqd;
int ret;
if (!dev->ops->erase_block)
return 0;
memset(&rqd, 0, sizeof(struct nvm_rq));
lightnvm: support multiple ppas in nvm_erase_ppa Sometimes a user want to erase multiple PPAs at the same time. Extend nvm_erase_ppa to take multiple ppas and number of ppas to be erased. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:28 +08:00
ret = nvm_set_rqd_ppalist(dev, &rqd, ppas, nr_ppas);
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
if (ret)
return ret;
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
nvm_generic_to_addr_mode(dev, &rqd);
ret = dev->ops->erase_block(dev, &rqd);
lightnvm: refactor rqd ppa list into set/free A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:20 +08:00
nvm_free_rqd_ppalist(dev, &rqd);
lightnvm: move ppa erase logic to core A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:19 +08:00
return ret;
}
EXPORT_SYMBOL(nvm_erase_ppa);
lightnvm: refactor end_io functions for sync To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:21 +08:00
void nvm_end_io(struct nvm_rq *rqd, int error)
{
lightnvm: move rq->error to nvm_rq->error Instead of passing request error into the LightNVM modules, incorporate it into the nvm_rq. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:29 +08:00
rqd->error = error;
rqd->end_io(rqd);
lightnvm: refactor end_io functions for sync To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:21 +08:00
}
EXPORT_SYMBOL(nvm_end_io);
lightnvm: introduce nvm_submit_ppa Internal logic for both core and media managers, does not have a backing bio for issuing I/Os. Introduce nvm_submit_ppa to allow raw I/Os to be submitted to the underlying device driver. The function request the device, ppa, data buffer and its length and will submit the I/O synchronously to the device. The return value may therefore be used to detect any errors regarding the issued I/O. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:30 +08:00
static void nvm_end_io_sync(struct nvm_rq *rqd)
lightnvm: refactor end_io functions for sync To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:21 +08:00
{
struct completion *waiting = rqd->wait;
rqd->wait = NULL;
complete(waiting);
}
lightnvm: introduce nvm_submit_ppa Internal logic for both core and media managers, does not have a backing bio for issuing I/Os. Introduce nvm_submit_ppa to allow raw I/Os to be submitted to the underlying device driver. The function request the device, ppa, data buffer and its length and will submit the I/O synchronously to the device. The return value may therefore be used to detect any errors regarding the issued I/O. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:30 +08:00
int nvm_submit_ppa(struct nvm_dev *dev, struct ppa_addr *ppa, int nr_ppas,
int opcode, int flags, void *buf, int len)
{
DECLARE_COMPLETION_ONSTACK(wait);
struct nvm_rq rqd;
struct bio *bio;
int ret;
unsigned long hang_check;
bio = bio_map_kern(dev->q, buf, len, GFP_KERNEL);
if (IS_ERR_OR_NULL(bio))
return -ENOMEM;
memset(&rqd, 0, sizeof(struct nvm_rq));
ret = nvm_set_rqd_ppalist(dev, &rqd, ppa, nr_ppas);
if (ret) {
bio_put(bio);
return ret;
}
rqd.opcode = opcode;
rqd.bio = bio;
rqd.wait = &wait;
rqd.dev = dev;
rqd.end_io = nvm_end_io_sync;
rqd.flags = flags;
nvm_generic_to_addr_mode(dev, &rqd);
ret = dev->ops->submit_io(dev, &rqd);
/* Prevent hang_check timer from firing at us during very long I/O */
hang_check = sysctl_hung_task_timeout_secs;
if (hang_check)
while (!wait_for_completion_io_timeout(&wait, hang_check * (HZ/2)));
else
wait_for_completion_io(&wait);
nvm_free_rqd_ppalist(dev, &rqd);
return rqd.error;
}
EXPORT_SYMBOL(nvm_submit_ppa);
lightnvm: introduce mlc lower page table mappings NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:35 +08:00
static int nvm_init_slc_tbl(struct nvm_dev *dev, struct nvm_id_group *grp)
{
int i;
dev->lps_per_blk = dev->pgs_per_blk;
dev->lptbl = kcalloc(dev->lps_per_blk, sizeof(int), GFP_KERNEL);
if (!dev->lptbl)
return -ENOMEM;
/* Just a linear array */
for (i = 0; i < dev->lps_per_blk; i++)
dev->lptbl[i] = i;
return 0;
}
static int nvm_init_mlc_tbl(struct nvm_dev *dev, struct nvm_id_group *grp)
{
int i, p;
struct nvm_id_lp_mlc *mlc = &grp->lptbl.mlc;
if (!mlc->num_pairs)
return 0;
dev->lps_per_blk = mlc->num_pairs;
dev->lptbl = kcalloc(dev->lps_per_blk, sizeof(int), GFP_KERNEL);
if (!dev->lptbl)
return -ENOMEM;
/* The lower page table encoding consists of a list of bytes, where each
* has a lower and an upper half. The first half byte maintains the
* increment value and every value after is an offset added to the
* previous incrementation value */
dev->lptbl[0] = mlc->pairs[0] & 0xF;
for (i = 1; i < dev->lps_per_blk; i++) {
p = mlc->pairs[i >> 1];
if (i & 0x1) /* upper */
dev->lptbl[i] = dev->lptbl[i - 1] + ((p & 0xF0) >> 4);
else /* lower */
dev->lptbl[i] = dev->lptbl[i - 1] + (p & 0xF);
}
return 0;
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
static int nvm_core_init(struct nvm_dev *dev)
{
struct nvm_id *id = &dev->identity;
struct nvm_id_group *grp = &id->groups[0];
/* device values */
dev->nr_chnls = grp->num_ch;
dev->luns_per_chnl = grp->num_lun;
dev->pgs_per_blk = grp->num_pg;
dev->blks_per_lun = grp->num_blk;
dev->nr_planes = grp->num_pln;
dev->sec_size = grp->csecs;
dev->oob_size = grp->sos;
dev->sec_per_pg = grp->fpg_sz / grp->csecs;
lightnvm: add mccap support Some flash media has extended capabilities, such as programming SLC pages on MLC/TLC flash, erase/program suspend, scramble and encryption. MCCAP is introduced to detect support for these capabilities in the command set. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:34 +08:00
dev->mccap = grp->mccap;
lightnvm: remove linear and device addr modes The linear and device specific address modes can be replaced with a simple offset and bit length conversion that is generic across all devices. This both simplifies the specification and removes the special case for qemu nvme, that previously relied on the linear address mapping. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:44 +08:00
memcpy(&dev->ppaf, &id->ppaf, sizeof(struct nvm_addr_format));
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
dev->plane_mode = NVM_PLANE_SINGLE;
dev->max_rq_size = dev->ops->max_phys_sect * dev->sec_size;
lightnvm: check for NAND flash and its type Only NAND flash with SLC and MLC is supported. Make sure to not try to initialize TLC memory or other non-volatile memory types. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:41 +08:00
if (grp->mtype != 0) {
pr_err("nvm: memory type not supported\n");
return -EINVAL;
}
lightnvm: introduce mlc lower page table mappings NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:35 +08:00
switch (grp->fmtype) {
case NVM_ID_FMTYPE_SLC:
if (nvm_init_slc_tbl(dev, grp))
return -ENOMEM;
break;
case NVM_ID_FMTYPE_MLC:
if (nvm_init_mlc_tbl(dev, grp))
return -ENOMEM;
break;
default:
lightnvm: check for NAND flash and its type Only NAND flash with SLC and MLC is supported. Make sure to not try to initialize TLC memory or other non-volatile memory types. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:41 +08:00
pr_err("nvm: flash type not supported\n");
return -EINVAL;
}
lightnvm: introduce mlc lower page table mappings NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:35 +08:00
if (!dev->lps_per_blk)
pr_info("nvm: lower page programming table missing\n");
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (grp->mpos & 0x020202)
dev->plane_mode = NVM_PLANE_DOUBLE;
if (grp->mpos & 0x040404)
dev->plane_mode = NVM_PLANE_QUAD;
/* calculated values */
dev->sec_per_pl = dev->sec_per_pg * dev->nr_planes;
dev->sec_per_blk = dev->sec_per_pl * dev->pgs_per_blk;
dev->sec_per_lun = dev->sec_per_blk * dev->blks_per_lun;
dev->nr_luns = dev->luns_per_chnl * dev->nr_chnls;
dev->total_blocks = dev->nr_planes *
dev->blks_per_lun *
dev->luns_per_chnl *
dev->nr_chnls;
dev->total_pages = dev->total_blocks * dev->pgs_per_blk;
INIT_LIST_HEAD(&dev->online_targets);
lightnvm: core on-disk initialization An Open-Channel SSD shall be initialized before use. To initialize, we define an on-disk format, that keeps a small set of metadata to bring up the media manager on top of the device. The initial step is introduced to allow a user to format the disks for a given media manager. During format, a system block is stored on one to three separate luns on the device. Each lun has the system block duplicated. During initialization, the system block can be retrieved and the appropriate media manager can initialized. The on-disk format currently covers (struct nvm_system_block): - Magic value "NVMS". - Monotonic increasing sequence number. - The physical block erase count. - Version of the system block format. - Media manager type. - Media manager superblock physical address. The interface provides three functions to manage the system block: int nvm_init_sysblock(struct nvm_dev *, struct nvm_sb_info *) int nvm_get_sysblock(struct nvm *dev, struct nvm_sb_info *) int nvm_update_sysblock(struct nvm *dev, struct nvm_sb_info *) Each implement a part of the logic to manage the system block. The initialization creates the first system blocks and mark them on the device. Get retrieves the latest system block by scanning all pages in the associated system blocks. The update sysblock writes new metadata and allocates new block if necessary. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:36 +08:00
mutex_init(&dev->mlock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return 0;
}
static void nvm_free(struct nvm_dev *dev)
{
if (!dev)
return;
if (dev->mt)
dev->mt->unregister_mgr(dev);
lightnvm: introduce mlc lower page table mappings NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:35 +08:00
kfree(dev->lptbl);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
}
static int nvm_init(struct nvm_dev *dev)
{
lightnvm: wrong return value and redundant free The return value should be non-zero under error conditions. Remove nvme_free(dev) to avoid free dev more than once. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20 20:47:53 +08:00
int ret = -EINVAL;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (!dev->q || !dev->ops)
lightnvm: wrong return value and redundant free The return value should be non-zero under error conditions. Remove nvme_free(dev) to avoid free dev more than once. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20 20:47:53 +08:00
return ret;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
lightnvm: replace req queue with nvmdev for lld In the case where a request queue is passed to the low lever lightnvm device drive integration, the device driver might pass its admin commands through another queue. Instead pass nvm_dev, and let the low level drive the appropriate queue. Reported-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:48 +08:00
if (dev->ops->identity(dev, &dev->identity)) {
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
pr_err("nvm: device could not be identified\n");
goto err;
}
pr_debug("nvm: ver:%x nvm_vendor:%x groups:%u\n",
dev->identity.ver_id, dev->identity.vmnt,
dev->identity.cgrps);
if (dev->identity.ver_id != 1) {
pr_err("nvm: device not supported by kernel.");
goto err;
}
if (dev->identity.cgrps != 1) {
pr_err("nvm: only one group configuration supported.");
goto err;
}
ret = nvm_core_init(dev);
if (ret) {
pr_err("nvm: could not initialize core structures.\n");
goto err;
}
pr_info("nvm: registered %s [%u/%u/%u/%u/%u/%u]\n",
dev->name, dev->sec_per_pg, dev->nr_planes,
dev->pgs_per_blk, dev->blks_per_lun, dev->nr_luns,
dev->nr_chnls);
return 0;
err:
pr_err("nvm: failed to initialize nvm\n");
return ret;
}
static void nvm_exit(struct nvm_dev *dev)
{
if (dev->ppalist_pool)
dev->ops->destroy_dma_pool(dev->ppalist_pool);
nvm_free(dev);
pr_info("nvm: successfully unloaded\n");
}
int nvm_register(struct request_queue *q, char *disk_name,
struct nvm_dev_ops *ops)
{
struct nvm_dev *dev;
int ret;
if (!ops->identity)
return -EINVAL;
dev = kzalloc(sizeof(struct nvm_dev), GFP_KERNEL);
if (!dev)
return -ENOMEM;
dev->q = q;
dev->ops = ops;
strncpy(dev->name, disk_name, DISK_NAME_LEN);
ret = nvm_init(dev);
if (ret)
goto err_init;
lightnvm: do device max sectors boundary check first do device max_phys_sect boundary check first, otherwise we will allocate dma_pools for devices whose max sectors are beyond lightnvm support and register them. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:25 +08:00
if (dev->ops->max_phys_sect > 256) {
pr_info("nvm: max sectors supported is 256.\n");
ret = -EINVAL;
goto err_init;
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (dev->ops->max_phys_sect > 1) {
lightnvm: replace req queue with nvmdev for lld In the case where a request queue is passed to the low lever lightnvm device drive integration, the device driver might pass its admin commands through another queue. Instead pass nvm_dev, and let the low level drive the appropriate queue. Reported-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:48 +08:00
dev->ppalist_pool = dev->ops->create_dma_pool(dev, "ppalist");
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (!dev->ppalist_pool) {
pr_err("nvm: could not create ppa pool\n");
lightnvm: missing free on init error If either max_phys_sect is out of bound, the nvm_dev structure is not freed. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20 20:47:54 +08:00
ret = -ENOMEM;
goto err_init;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
}
}
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
/* register device with a supported media manager */
lightnvm: prematurely activate nvm_dev We register with nvm_devices when there registration can still fail. Move the final registration at the end of the nvm_register function to make sure we are fully registered when added to the nvm_devices list. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:42 +08:00
down_write(&nvm_lock);
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
dev->mt = nvm_init_mgr(dev);
lightnvm: prematurely activate nvm_dev We register with nvm_devices when there registration can still fail. Move the final registration at the end of the nvm_register function to make sure we are fully registered when added to the nvm_devices list. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:42 +08:00
list_add(&dev->devices, &nvm_devices);
up_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return 0;
err_init:
kfree(dev);
return ret;
}
EXPORT_SYMBOL(nvm_register);
void nvm_unregister(char *disk_name)
{
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
struct nvm_dev *dev;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
down_write(&nvm_lock);
dev = nvm_find_nvm_dev(disk_name);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (!dev) {
pr_err("nvm: could not find device %s to unregister\n",
disk_name);
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
up_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return;
}
list_del(&dev->devices);
up_write(&nvm_lock);
lightnvm: prevent double free on init error Both the nvm_register and nvm_init does a kfree(dev) on error. Make sure to only free it once. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:43 +08:00
nvm_exit(dev);
kfree(dev);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
}
EXPORT_SYMBOL(nvm_unregister);
static const struct block_device_operations nvm_fops = {
.owner = THIS_MODULE,
};
static int nvm_create_target(struct nvm_dev *dev,
struct nvm_ioctl_create *create)
{
struct nvm_ioctl_create_simple *s = &create->conf.s;
struct request_queue *tqueue;
struct gendisk *tdisk;
struct nvm_tgt_type *tt;
struct nvm_target *t;
void *targetdata;
if (!dev->mt) {
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
pr_info("nvm: device has no media manager registered.\n");
return -ENODEV;
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
}
lightnvm: fix media mgr registration This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-06 18:25:49 +08:00
down_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
tt = nvm_find_target_type(create->tgttype);
if (!tt) {
pr_err("nvm: target type %s not found\n", create->tgttype);
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
up_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return -EINVAL;
}
list_for_each_entry(t, &dev->online_targets, list) {
if (!strcmp(create->tgtname, t->disk->disk_name)) {
pr_err("nvm: target name already exists.\n");
up_write(&nvm_lock);
return -EINVAL;
}
}
up_write(&nvm_lock);
t = kmalloc(sizeof(struct nvm_target), GFP_KERNEL);
if (!t)
return -ENOMEM;
tqueue = blk_alloc_queue_node(GFP_KERNEL, dev->q->node);
if (!tqueue)
goto err_t;
blk_queue_make_request(tqueue, tt->make_rq);
tdisk = alloc_disk(0);
if (!tdisk)
goto err_queue;
sprintf(tdisk->disk_name, "%s", create->tgtname);
tdisk->flags = GENHD_FL_EXT_DEVT;
tdisk->major = 0;
tdisk->first_minor = 0;
tdisk->fops = &nvm_fops;
tdisk->queue = tqueue;
targetdata = tt->init(dev, tdisk, s->lun_begin, s->lun_end);
if (IS_ERR(targetdata))
goto err_init;
tdisk->private_data = targetdata;
tqueue->queuedata = targetdata;
blk_queue_max_hw_sectors(tqueue, 8 * dev->ops->max_phys_sect);
set_capacity(tdisk, tt->capacity(targetdata));
add_disk(tdisk);
t->type = tt;
t->disk = tdisk;
down_write(&nvm_lock);
list_add_tail(&t->list, &dev->online_targets);
up_write(&nvm_lock);
return 0;
err_init:
put_disk(tdisk);
err_queue:
blk_cleanup_queue(tqueue);
err_t:
kfree(t);
return -ENOMEM;
}
static void nvm_remove_target(struct nvm_target *t)
{
struct nvm_tgt_type *tt = t->type;
struct gendisk *tdisk = t->disk;
struct request_queue *q = tdisk->queue;
lockdep_assert_held(&nvm_lock);
del_gendisk(tdisk);
lightnvm: cleanup queue before target removal This prevents outstanding IOs to be sent for completion to target after the target has been removed. The flow is now: stop new IOs > cleanup queue > remove target. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-16 22:34:47 +08:00
blk_cleanup_queue(q);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (tt->exit)
tt->exit(tdisk->private_data);
put_disk(tdisk);
list_del(&t->list);
kfree(t);
}
static int __nvm_configure_create(struct nvm_ioctl_create *create)
{
struct nvm_dev *dev;
struct nvm_ioctl_create_simple *s;
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
down_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
dev = nvm_find_nvm_dev(create->dev);
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
up_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (!dev) {
pr_err("nvm: device not found\n");
return -EINVAL;
}
if (create->conf.type != NVM_CONFIG_TYPE_SIMPLE) {
pr_err("nvm: config type not valid\n");
return -EINVAL;
}
s = &create->conf.s;
if (s->lun_begin > s->lun_end || s->lun_end > dev->nr_luns) {
pr_err("nvm: lun out of bound (%u:%u > %u)\n",
s->lun_begin, s->lun_end, dev->nr_luns);
return -EINVAL;
}
return nvm_create_target(dev, create);
}
static int __nvm_configure_remove(struct nvm_ioctl_remove *remove)
{
struct nvm_target *t = NULL;
struct nvm_dev *dev;
int ret = -1;
down_write(&nvm_lock);
list_for_each_entry(dev, &nvm_devices, devices)
list_for_each_entry(t, &dev->online_targets, list) {
if (!strcmp(remove->tgtname, t->disk->disk_name)) {
nvm_remove_target(t);
ret = 0;
break;
}
}
up_write(&nvm_lock);
if (ret) {
pr_err("nvm: target \"%s\" doesn't exist.\n", remove->tgtname);
return -EINVAL;
}
return 0;
}
#ifdef CONFIG_NVM_DEBUG
static int nvm_configure_show(const char *val)
{
struct nvm_dev *dev;
char opcode, devname[DISK_NAME_LEN];
int ret;
ret = sscanf(val, "%c %32s", &opcode, devname);
if (ret != 2) {
pr_err("nvm: invalid command. Use \"opcode devicename\".\n");
return -EINVAL;
}
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
down_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
dev = nvm_find_nvm_dev(devname);
lightnvm: missing nvm_lock acquire To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:28 +08:00
up_write(&nvm_lock);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
if (!dev) {
pr_err("nvm: device not found\n");
return -EINVAL;
}
if (!dev->mt)
return 0;
lightnvm: add free and bad lun info to show luns Add free block, used block, and bad block information to the show debug interface. This information is used to debug how targets track blocks. Also, change debug function name to make it more generic. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20 20:47:57 +08:00
dev->mt->lun_info_print(dev);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return 0;
}
static int nvm_configure_remove(const char *val)
{
struct nvm_ioctl_remove remove;
char opcode;
int ret;
ret = sscanf(val, "%c %256s", &opcode, remove.tgtname);
if (ret != 2) {
pr_err("nvm: invalid command. Use \"d targetname\".\n");
return -EINVAL;
}
remove.flags = 0;
return __nvm_configure_remove(&remove);
}
static int nvm_configure_create(const char *val)
{
struct nvm_ioctl_create create;
char opcode;
int lun_begin, lun_end, ret;
ret = sscanf(val, "%c %256s %256s %48s %u:%u", &opcode, create.dev,
create.tgtname, create.tgttype,
&lun_begin, &lun_end);
if (ret != 6) {
pr_err("nvm: invalid command. Use \"opcode device name tgttype lun_begin:lun_end\".\n");
return -EINVAL;
}
create.flags = 0;
create.conf.type = NVM_CONFIG_TYPE_SIMPLE;
create.conf.s.lun_begin = lun_begin;
create.conf.s.lun_end = lun_end;
return __nvm_configure_create(&create);
}
/* Exposes administrative interface through /sys/module/lnvm/configure_by_str */
static int nvm_configure_by_str_event(const char *val,
const struct kernel_param *kp)
{
char opcode;
int ret;
ret = sscanf(val, "%c", &opcode);
if (ret != 1) {
pr_err("nvm: string must have the format of \"cmd ...\"\n");
return -EINVAL;
}
switch (opcode) {
case 'a':
return nvm_configure_create(val);
case 'd':
return nvm_configure_remove(val);
case 's':
return nvm_configure_show(val);
default:
pr_err("nvm: invalid command\n");
return -EINVAL;
}
return 0;
}
static int nvm_configure_get(char *buf, const struct kernel_param *kp)
{
int sz = 0;
char *buf_start = buf;
struct nvm_dev *dev;
buf += sprintf(buf, "available devices:\n");
down_write(&nvm_lock);
list_for_each_entry(dev, &nvm_devices, devices) {
if (sz > 4095 - DISK_NAME_LEN)
break;
buf += sprintf(buf, " %32s\n", dev->name);
}
up_write(&nvm_lock);
return buf - buf_start - 1;
}
static const struct kernel_param_ops nvm_configure_by_str_event_param_ops = {
.set = nvm_configure_by_str_event,
.get = nvm_configure_get,
};
#undef MODULE_PARAM_PREFIX
#define MODULE_PARAM_PREFIX "lnvm."
module_param_cb(configure_debug, &nvm_configure_by_str_event_param_ops, NULL,
0644);
#endif /* CONFIG_NVM_DEBUG */
static long nvm_ioctl_info(struct file *file, void __user *arg)
{
struct nvm_ioctl_info *info;
struct nvm_tgt_type *tt;
int tgt_iter = 0;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
info = memdup_user(arg, sizeof(struct nvm_ioctl_info));
if (IS_ERR(info))
return -EFAULT;
info->version[0] = NVM_VERSION_MAJOR;
info->version[1] = NVM_VERSION_MINOR;
info->version[2] = NVM_VERSION_PATCH;
down_write(&nvm_lock);
list_for_each_entry(tt, &nvm_targets, list) {
struct nvm_ioctl_info_tgt *tgt = &info->tgts[tgt_iter];
tgt->version[0] = tt->version[0];
tgt->version[1] = tt->version[1];
tgt->version[2] = tt->version[2];
strncpy(tgt->tgtname, tt->name, NVM_TTYPE_NAME_MAX);
tgt_iter++;
}
info->tgtsize = tgt_iter;
up_write(&nvm_lock);
lightnvm: fix ioctl memory leaks If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:24 +08:00
if (copy_to_user(arg, info, sizeof(struct nvm_ioctl_info))) {
kfree(info);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return -EFAULT;
lightnvm: fix ioctl memory leaks If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:24 +08:00
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
kfree(info);
return 0;
}
static long nvm_ioctl_get_devices(struct file *file, void __user *arg)
{
struct nvm_ioctl_get_devices *devices;
struct nvm_dev *dev;
int i = 0;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
devices = kzalloc(sizeof(struct nvm_ioctl_get_devices), GFP_KERNEL);
if (!devices)
return -ENOMEM;
down_write(&nvm_lock);
list_for_each_entry(dev, &nvm_devices, devices) {
struct nvm_ioctl_device_info *info = &devices->info[i];
sprintf(info->devname, "%s", dev->name);
if (dev->mt) {
info->bmversion[0] = dev->mt->version[0];
info->bmversion[1] = dev->mt->version[1];
info->bmversion[2] = dev->mt->version[2];
sprintf(info->bmname, "%s", dev->mt->name);
} else {
sprintf(info->bmname, "none");
}
i++;
if (i > 31) {
pr_err("nvm: max 31 devices can be reported.\n");
break;
}
}
up_write(&nvm_lock);
devices->nr_devices = i;
lightnvm: fix ioctl memory leaks If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:24 +08:00
if (copy_to_user(arg, devices,
sizeof(struct nvm_ioctl_get_devices))) {
kfree(devices);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
return -EFAULT;
lightnvm: fix ioctl memory leaks If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-28 23:49:24 +08:00
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
kfree(devices);
return 0;
}
static long nvm_ioctl_dev_create(struct file *file, void __user *arg)
{
struct nvm_ioctl_create create;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
if (copy_from_user(&create, arg, sizeof(struct nvm_ioctl_create)))
return -EFAULT;
create.dev[DISK_NAME_LEN - 1] = '\0';
create.tgttype[NVM_TTYPE_NAME_MAX - 1] = '\0';
create.tgtname[DISK_NAME_LEN - 1] = '\0';
if (create.flags != 0) {
pr_err("nvm: no flags supported\n");
return -EINVAL;
}
return __nvm_configure_create(&create);
}
static long nvm_ioctl_dev_remove(struct file *file, void __user *arg)
{
struct nvm_ioctl_remove remove;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
if (copy_from_user(&remove, arg, sizeof(struct nvm_ioctl_remove)))
return -EFAULT;
remove.tgtname[DISK_NAME_LEN - 1] = '\0';
if (remove.flags != 0) {
pr_err("nvm: no flags supported\n");
return -EINVAL;
}
return __nvm_configure_remove(&remove);
}
lightnvm: introduce ioctl to initialize device Based on the previous patch, we now introduce an ioctl to initialize the device using nvm_init_sysblock and create the necessary system blocks. The user may specify the media manager that they wish to instantiate on top. Default from user-space will be "gennvm". Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:37 +08:00
static void nvm_setup_nvm_sb_info(struct nvm_sb_info *info)
{
info->seqnr = 1;
info->erase_cnt = 0;
info->version = 1;
}
static long __nvm_ioctl_dev_init(struct nvm_ioctl_dev_init *init)
{
struct nvm_dev *dev;
struct nvm_sb_info info;
down_write(&nvm_lock);
dev = nvm_find_nvm_dev(init->dev);
up_write(&nvm_lock);
if (!dev) {
pr_err("nvm: device not found\n");
return -EINVAL;
}
nvm_setup_nvm_sb_info(&info);
strncpy(info.mmtype, init->mmtype, NVM_MMTYPE_LEN);
info.fs_ppa.ppa = -1;
return nvm_init_sysblock(dev, &info);
}
static long nvm_ioctl_dev_init(struct file *file, void __user *arg)
{
struct nvm_ioctl_dev_init init;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
if (copy_from_user(&init, arg, sizeof(struct nvm_ioctl_dev_init)))
return -EFAULT;
if (init.flags != 0) {
pr_err("nvm: no flags supported\n");
return -EINVAL;
}
init.dev[DISK_NAME_LEN - 1] = '\0';
return __nvm_ioctl_dev_init(&init);
}
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
static long nvm_ctl_ioctl(struct file *file, uint cmd, unsigned long arg)
{
void __user *argp = (void __user *)arg;
switch (cmd) {
case NVM_INFO:
return nvm_ioctl_info(file, argp);
case NVM_GET_DEVICES:
return nvm_ioctl_get_devices(file, argp);
case NVM_DEV_CREATE:
return nvm_ioctl_dev_create(file, argp);
case NVM_DEV_REMOVE:
return nvm_ioctl_dev_remove(file, argp);
lightnvm: introduce ioctl to initialize device Based on the previous patch, we now introduce an ioctl to initialize the device using nvm_init_sysblock and create the necessary system blocks. The user may specify the media manager that they wish to instantiate on top. Default from user-space will be "gennvm". Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12 14:49:37 +08:00
case NVM_DEV_INIT:
return nvm_ioctl_dev_init(file, argp);
lightnvm: Support for Open-Channel SSDs Open-channel SSDs are devices that share responsibilities with the host in order to implement and maintain features that typical SSDs keep strictly in firmware. These include (i) the Flash Translation Layer (FTL), (ii) bad block management, and (iii) hardware units such as the flash controller, the interface controller, and large amounts of flash chips. In this way, Open-channels SSDs exposes direct access to their physical flash storage, while keeping a subset of the internal features of SSDs. LightNVM is a specification that gives support to Open-channel SSDs LightNVM allows the host to manage data placement, garbage collection, and parallelism. Device specific responsibilities such as bad block management, FTL extensions to support atomic IOs, or metadata persistence are still handled by the device. The implementation of LightNVM consists of two parts: core and (multiple) targets. The core implements functionality shared across targets. This is initialization, teardown and statistics. The targets implement the interface that exposes physical flash to user-space applications. Examples of such targets include key-value store, object-store, as well as traditional block devices, which can be application-specific. Contributions in this patch from: Javier Gonzalez <jg@lightnvm.io> Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Jesper Madsen <jmad@itu.dk> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29 02:54:55 +08:00
}
return 0;
}
static const struct file_operations _ctl_fops = {
.open = nonseekable_open,
.unlocked_ioctl = nvm_ctl_ioctl,
.owner = THIS_MODULE,
.llseek = noop_llseek,
};
static struct miscdevice _nvm_misc = {
.minor = MISC_DYNAMIC_MINOR,
.name = "lightnvm",
.nodename = "lightnvm/control",
.fops = &_ctl_fops,
};
MODULE_ALIAS_MISCDEV(MISC_DYNAMIC_MINOR);
static int __init nvm_mod_init(void)
{
int ret;
ret = misc_register(&_nvm_misc);
if (ret)
pr_err("nvm: misc_register failed for control device");
return ret;
}
static void __exit nvm_mod_exit(void)
{
misc_deregister(&_nvm_misc);
}
MODULE_AUTHOR("Matias Bjorling <m@bjorling.me>");
MODULE_LICENSE("GPL v2");
MODULE_VERSION("0.1");
module_init(nvm_mod_init);
module_exit(nvm_mod_exit);