sg_miter_start() is currently unaware of the direction of the copy
process (to or from the scatter list). It is important to know the
direction because the page has to be flushed in case the data written
is seen on a different mapping in user land on cache incoherent
architectures.
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Pierre Ossman <pierre@ossman.eu>
This is alternative implementation of sg content iterator introduced
by commit 83e7d317... from Pierre Ossman in next-20080716. As there's
already an sg iterator which iterates over sg entries themselves, name
this sg_mapping_iterator.
Slightly edited description from the original implementation follows.
Iteration over a sg list is not that trivial when you take into
account that memory pages might have to be mapped before being used.
Unfortunately, that means that some parts of the kernel restrict
themselves to directly accesible memory just to not have to deal with
the mess.
This patch adds a simple iterator system that allows any code to
easily traverse an sg list and not have to deal with all the details.
The user can decide to consume part of the iteration. Also, iteration
can be stopped and resumed later if releasing the kmap between
iteration steps is necessary. These features are useful to implement
piecemeal sg copying for interrupt drive PIO for example.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Pierre Ossman <drzeus@drzeus.cx>
This patch adds new three helper functions to copy data between an SG
list and a linear buffer.
- sg_copy_from_buffer copies data from linear buffer to an SG list
- sg_copy_to_buffer copies data from an SG list to a linear buffer
When the APIs copy data from a linear buffer to an SG list,
flush_kernel_dcache_page is called. It's not necessary for everyone
but it's a no-op on most architectures and in general the API is not
used in performance critical path.
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
SCSI sg table allocation has a maximum size (of SCSI_MAX_SG_SEGMENTS,
currently 128) and this will cause a BUG_ON() in SCSI if something
tries an allocation over it. This patch adds a size limit to the
chaining allocator to allow the specification of the maximum
allocation size for chaining, so we always chain in units of the
maximum SCSI allocation size.
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Manually doing chained sg lists is not trivial, so add some helpers
to make sure that drivers get it right.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
I realize that sg chaining is a ploy to make the rest of the kernel
devs feel the pain of the SCSI subsystem. But this was a little
unsubtle.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Acked-by: Tejun Heo <htejun@gmail.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Add more safeguards to protect against misinterpreting a chain entry
as a normal scatterlist and vice-versa.
* Make sure the entry isn't a chain when assigning and reading a
normal sg.
* Clear offset and length when chaining.
Signed-off-by: Tejun Heo <htejun@gmail.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
sg_mark_end() overwrites the page_link information, but all users want
__sg_mark_end() behaviour where we just set the end bit. That is the most
natural way to use the sg list, since you'll fill it in and then mark the
end point.
So change sg_mark_end() to only set the termination bit. Add a sg_magic
debug check as well, and clear a chain pointer if it is set.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Clean up: fix a mixed sign comparison in sg_init_table() accidentally
introduced by commit d6ec0842. The sign of the loop index variable
should match the sign of the "nents" argument.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Signed-off-by: Jens Axboe <axboe@carl.home.kernel.dk>
Clean up: fix a mixed sign comparison in sg_last() accidentally
introduced by commit 70eb8040. The sign of the loop index variable
should match the sign of the "nents" argument.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jens Axboe <axboe@carl.home.kernel.dk>
Since we are using the last entry in the list, clear any possible
termination bit that may have already been set. Pointed out by Rusty.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
x86_32 CONFIG_HIGHMEM64G with 5GB RAM hung when booting, after issuing
some "request_module: runaway loop modprobe binfmt-0000" messages in
trying to exec /sbin/init.
The binprm buf doesn't see the right ".ELF" header because sg_phys()
is providing the wrong physical addresses for high pages: a 32-bit
unsigned long is too small in this case, we need to use dma_addr_t.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Most drivers need to set length and offset as well, so may as well fold
those three lines into one.
Add sg_assign_page() for those two locations that only needed to set
the page, where the offset/length is set outside of the function context.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Change the page member of the scatterlist structure to be an unsigned
long, and encode more stuff in the lower bits:
- Bits 0 and 1 zero: this is a normal sg entry. Next sg entry is located
at sg + 1.
- Bit 0 set: this is a chain entry, the next real entry is at ->page_link
with the two low bits masked off.
- Bit 1 set: this is the final entry in the sg entry. sg_next() will return
NULL when passed such an entry.
It's thus important that sg table users use the proper accessors to get
and set the page member.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
The core of the patch - allow the last sg element in a scatterlist
table to point to the start of a new table. We overload the LSB of
the page pointer to indicate whether this is a valid sg entry, or
merely a link to the next list.
Includes a fix from Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
correcting the ifdef ARCH_HAS_SG_CHAIN guarding sg_last().
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
This patch adds a const modifier to the buf argument of sg_set_buf and
sg_init_one. This lets people call it with pointers that are const.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
sg_init_one is a nice tool for the block layer. However, users
of struct scatterlist in other subsystems don't usually need the
DMA attributes. For them it's a waste of time and space to
initialise the whole struct scatterlist structure.
Therefore this patch adds a new function sg_set_buf to initialise
a scatterlist without zeroing the DMA attributes.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
Sebastian Andrzej Siewior