175 lines
5.4 KiB
Plaintext
175 lines
5.4 KiB
Plaintext
====================================
|
|
SLOW WORK ITEM EXECUTION THREAD POOL
|
|
====================================
|
|
|
|
By: David Howells <dhowells@redhat.com>
|
|
|
|
The slow work item execution thread pool is a pool of threads for performing
|
|
things that take a relatively long time, such as making mkdir calls.
|
|
Typically, when processing something, these items will spend a lot of time
|
|
blocking a thread on I/O, thus making that thread unavailable for doing other
|
|
work.
|
|
|
|
The standard workqueue model is unsuitable for this class of work item as that
|
|
limits the owner to a single thread or a single thread per CPU. For some
|
|
tasks, however, more threads - or fewer - are required.
|
|
|
|
There is just one pool per system. It contains no threads unless something
|
|
wants to use it - and that something must register its interest first. When
|
|
the pool is active, the number of threads it contains is dynamic, varying
|
|
between a maximum and minimum setting, depending on the load.
|
|
|
|
|
|
====================
|
|
CLASSES OF WORK ITEM
|
|
====================
|
|
|
|
This pool support two classes of work items:
|
|
|
|
(*) Slow work items.
|
|
|
|
(*) Very slow work items.
|
|
|
|
The former are expected to finish much quicker than the latter.
|
|
|
|
An operation of the very slow class may do a batch combination of several
|
|
lookups, mkdirs, and a create for instance.
|
|
|
|
An operation of the ordinarily slow class may, for example, write stuff or
|
|
expand files, provided the time taken to do so isn't too long.
|
|
|
|
Operations of both types may sleep during execution, thus tying up the thread
|
|
loaned to it.
|
|
|
|
|
|
THREAD-TO-CLASS ALLOCATION
|
|
--------------------------
|
|
|
|
Not all the threads in the pool are available to work on very slow work items.
|
|
The number will be between one and one fewer than the number of active threads.
|
|
This is configurable (see the "Pool Configuration" section).
|
|
|
|
All the threads are available to work on ordinarily slow work items, but a
|
|
percentage of the threads will prefer to work on very slow work items.
|
|
|
|
The configuration ensures that at least one thread will be available to work on
|
|
very slow work items, and at least one thread will be available that won't work
|
|
on very slow work items at all.
|
|
|
|
|
|
=====================
|
|
USING SLOW WORK ITEMS
|
|
=====================
|
|
|
|
Firstly, a module or subsystem wanting to make use of slow work items must
|
|
register its interest:
|
|
|
|
int ret = slow_work_register_user();
|
|
|
|
This will return 0 if successful, or a -ve error upon failure.
|
|
|
|
|
|
Slow work items may then be set up by:
|
|
|
|
(1) Declaring a slow_work struct type variable:
|
|
|
|
#include <linux/slow-work.h>
|
|
|
|
struct slow_work myitem;
|
|
|
|
(2) Declaring the operations to be used for this item:
|
|
|
|
struct slow_work_ops myitem_ops = {
|
|
.get_ref = myitem_get_ref,
|
|
.put_ref = myitem_put_ref,
|
|
.execute = myitem_execute,
|
|
};
|
|
|
|
[*] For a description of the ops, see section "Item Operations".
|
|
|
|
(3) Initialising the item:
|
|
|
|
slow_work_init(&myitem, &myitem_ops);
|
|
|
|
or:
|
|
|
|
vslow_work_init(&myitem, &myitem_ops);
|
|
|
|
depending on its class.
|
|
|
|
A suitably set up work item can then be enqueued for processing:
|
|
|
|
int ret = slow_work_enqueue(&myitem);
|
|
|
|
This will return a -ve error if the thread pool is unable to gain a reference
|
|
on the item, 0 otherwise.
|
|
|
|
|
|
The items are reference counted, so there ought to be no need for a flush
|
|
operation. When all a module's slow work items have been processed, and the
|
|
module has no further interest in the facility, it should unregister its
|
|
interest:
|
|
|
|
slow_work_unregister_user();
|
|
|
|
|
|
===============
|
|
ITEM OPERATIONS
|
|
===============
|
|
|
|
Each work item requires a table of operations of type struct slow_work_ops.
|
|
All members are required:
|
|
|
|
(*) Get a reference on an item:
|
|
|
|
int (*get_ref)(struct slow_work *work);
|
|
|
|
This allows the thread pool to attempt to pin an item by getting a
|
|
reference on it. This function should return 0 if the reference was
|
|
granted, or a -ve error otherwise. If an error is returned,
|
|
slow_work_enqueue() will fail.
|
|
|
|
The reference is held whilst the item is queued and whilst it is being
|
|
executed. The item may then be requeued with the same reference held, or
|
|
the reference will be released.
|
|
|
|
(*) Release a reference on an item:
|
|
|
|
void (*put_ref)(struct slow_work *work);
|
|
|
|
This allows the thread pool to unpin an item by releasing the reference on
|
|
it. The thread pool will not touch the item again once this has been
|
|
called.
|
|
|
|
(*) Execute an item:
|
|
|
|
void (*execute)(struct slow_work *work);
|
|
|
|
This should perform the work required of the item. It may sleep, it may
|
|
perform disk I/O and it may wait for locks.
|
|
|
|
|
|
==================
|
|
POOL CONFIGURATION
|
|
==================
|
|
|
|
The slow-work thread pool has a number of configurables:
|
|
|
|
(*) /proc/sys/kernel/slow-work/min-threads
|
|
|
|
The minimum number of threads that should be in the pool whilst it is in
|
|
use. This may be anywhere between 2 and max-threads.
|
|
|
|
(*) /proc/sys/kernel/slow-work/max-threads
|
|
|
|
The maximum number of threads that should in the pool. This may be
|
|
anywhere between min-threads and 255 or NR_CPUS * 2, whichever is greater.
|
|
|
|
(*) /proc/sys/kernel/slow-work/vslow-percentage
|
|
|
|
The percentage of active threads in the pool that may be used to execute
|
|
very slow work items. This may be between 1 and 99. The resultant number
|
|
is bounded to between 1 and one fewer than the number of active threads.
|
|
This ensures there is always at least one thread that can process very
|
|
slow work items, and always at least one thread that won't.
|