2011-06-01 07:08:32 +08:00
|
|
|
<HTML>
|
|
|
|
<CENTER><A HREF = "http://lammps.sandia.gov">LAMMPS WWW Site</A> - <A HREF = "Manual.html">LAMMPS Documentation</A> - <A HREF = "Section_commands.html#comm">LAMMPS Commands</A>
|
|
|
|
</CENTER>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<HR>
|
|
|
|
|
|
|
|
<H3>package command
|
|
|
|
</H3>
|
|
|
|
<P><B>Syntax:</B>
|
|
|
|
</P>
|
|
|
|
<PRE>package style args
|
|
|
|
</PRE>
|
2011-08-17 22:20:30 +08:00
|
|
|
<UL><LI>style = <I>gpu</I> or <I>cuda</I> or <I>omp</I>
|
2011-06-01 07:08:32 +08:00
|
|
|
|
2011-08-17 22:20:30 +08:00
|
|
|
<LI>args = arguments specific to the style
|
2011-06-01 07:08:32 +08:00
|
|
|
|
2011-12-03 02:38:12 +08:00
|
|
|
<PRE> <I>gpu</I> args = mode first last split keyword value ...
|
2011-08-17 22:22:48 +08:00
|
|
|
mode = force or force/neigh
|
|
|
|
first = ID of first GPU to be used on each node
|
|
|
|
last = ID of last GPU to be used on each node
|
|
|
|
split = fraction of particles assigned to the GPU
|
2011-12-03 02:38:12 +08:00
|
|
|
zero or more keyword/value pairs may be appended
|
2012-09-22 00:04:48 +08:00
|
|
|
keywords = <I>threads_per_atom</I> or <I>cellsize</I>
|
|
|
|
<I>threads_per_atom</I> value = Nthreads
|
|
|
|
Nthreads = # of GPU threads used per atom
|
|
|
|
<I>cellsize</I> value = dist
|
|
|
|
dist = length (distance units) in each dimension for neighbor bins
|
2011-12-03 02:38:12 +08:00
|
|
|
<I>cuda</I> args = keyword value ...
|
|
|
|
one or more keyword/value pairs may be appended
|
2011-12-02 23:47:30 +08:00
|
|
|
keywords = <I>gpu/node</I> or <I>gpu/node/special</I> or <I>timing</I> or <I>test</I> or <I>override/bpa</I>
|
2012-09-22 00:04:48 +08:00
|
|
|
<I>gpu/node</I> value = N
|
|
|
|
N = number of GPUs to be used per node
|
|
|
|
<I>gpu/node/special</I> values = N gpu1 .. gpuN
|
|
|
|
N = number of GPUs to be used per node
|
|
|
|
gpu1 .. gpuN = N IDs of the GPUs to use
|
|
|
|
<I>timing</I> values = none
|
|
|
|
<I>test</I> values = id
|
|
|
|
id = atom-ID of a test particle
|
|
|
|
<I>override/bpa</I> values = flag
|
|
|
|
flag = 0 for TpA algorithm, 1 for BpA algorithm
|
2011-12-01 23:35:32 +08:00
|
|
|
<I>omp</I> args = Nthreads mode
|
|
|
|
Nthreads = # of OpenMP threads to associate with each MPI process
|
|
|
|
mode = force or force/neigh (optional)
|
2011-06-01 07:08:32 +08:00
|
|
|
</PRE>
|
|
|
|
|
|
|
|
</UL>
|
|
|
|
<P><B>Examples:</B>
|
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<PRE>package gpu force 0 0 1.0
|
|
|
|
package gpu force 0 0 0.75
|
|
|
|
package gpu force/neigh 0 0 1.0
|
|
|
|
package gpu force/neigh 0 1 -1.0
|
2011-12-03 02:38:12 +08:00
|
|
|
package cuda gpu/node/special 2 0 2
|
|
|
|
package cuda test 3948
|
2011-12-01 23:35:32 +08:00
|
|
|
package omp * force/neigh
|
|
|
|
package omp 4 force
|
2011-06-01 07:08:32 +08:00
|
|
|
</PRE>
|
|
|
|
<P><B>Description:</B>
|
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<P>This command invokes package-specific settings. Currently the
|
|
|
|
following packages use it: GPU, USER-CUDA, and USER-OMP.
|
|
|
|
</P>
|
2011-12-02 23:47:30 +08:00
|
|
|
<P>To use the accelerated GPU and USER-OMP styles, the use of the package
|
|
|
|
command is required. However, as described in the "Defaults" section
|
2012-01-28 07:39:14 +08:00
|
|
|
below, if you use the "-sf gpu" or "-sf omp" <A HREF = "Section_start.html#start_7">command-line
|
2011-12-02 23:47:30 +08:00
|
|
|
options</A> to enable use of these styles,
|
|
|
|
then default package settings are enabled. In that case you only need
|
|
|
|
to use the package command if you want to change the defaults.
|
|
|
|
</P>
|
|
|
|
<P>To use the accelerate USER-CUDA styles, the package command is not
|
|
|
|
required as defaults are assigned internally. You only need to use
|
|
|
|
the package command if you want to change the defaults.
|
|
|
|
</P>
|
2011-12-14 04:35:35 +08:00
|
|
|
<P>See <A HREF = "Section_accelerate.html">Section_accelerate</A> of the manual for
|
|
|
|
more details about using these various packages for accelerating
|
|
|
|
LAMMPS calculations.
|
2011-08-17 22:20:30 +08:00
|
|
|
</P>
|
|
|
|
<HR>
|
|
|
|
|
|
|
|
<P>The <I>gpu</I> style invokes options associated with the use of the GPU
|
2011-12-02 23:47:30 +08:00
|
|
|
package.
|
2011-08-17 22:20:30 +08:00
|
|
|
</P>
|
|
|
|
<P>The <I>mode</I> setting specifies where neighbor list calculations will be
|
|
|
|
performed. If <I>mode</I> is force, neighbor list calculation is performed
|
|
|
|
on the CPU. If <I>mode</I> is force/neigh, neighbor list calculation is
|
|
|
|
performed on the GPU. GPU neighbor list calculation currently cannot
|
|
|
|
be used with a triclinic box. GPU neighbor list calculation currently
|
|
|
|
cannot be used with <A HREF = "pair_hybrid.html">hybrid</A> pair styles. GPU
|
|
|
|
neighbor lists are not compatible with styles that are not
|
|
|
|
GPU-enabled. When a non-GPU enabled style requires a neighbor list,
|
|
|
|
it will also be built using CPU routines. In these cases, it will
|
|
|
|
typically be more efficient to only use CPU neighbor list builds.
|
|
|
|
</P>
|
|
|
|
<P>The <I>first</I> and <I>last</I> settings specify the GPUs that will be used for
|
|
|
|
simulation. On each node, the GPU IDs in the inclusive range from
|
|
|
|
<I>first</I> to <I>last</I> will be used.
|
|
|
|
</P>
|
|
|
|
<P>The <I>split</I> setting can be used for load balancing force calculation
|
|
|
|
work between CPU and GPU cores in GPU-enabled pair styles. If 0 <
|
|
|
|
<I>split</I> < 1.0, a fixed fraction of particles is offloaded to the GPU
|
|
|
|
while force calculation for the other particles occurs simulataneously
|
|
|
|
on the CPU. If <I>split</I><0, the optimal fraction (based on CPU and GPU
|
|
|
|
timings) is calculated every 25 timesteps. If <I>split</I> = 1.0, all force
|
|
|
|
calculations for GPU accelerated pair styles are performed on the
|
|
|
|
GPU. In this case, <A HREF = "pair_hybrid.html">hybrid</A>, <A HREF = "bond_style.html">bond</A>,
|
|
|
|
<A HREF = "angle_style.html">angle</A>, <A HREF = "dihedral_style.html">dihedral</A>,
|
|
|
|
<A HREF = "improper_style.html">improper</A>, and <A HREF = "kspace_style.html">long-range</A>
|
|
|
|
calculations can be performed on the CPU while the GPU is performing
|
|
|
|
force calculations for the GPU-enabled pair style. If all CPU force
|
|
|
|
computations complete before the GPU, LAMMPS will block until the GPU
|
|
|
|
has finished before continuing the timestep.
|
|
|
|
</P>
|
|
|
|
<P>As an example, if you have two GPUs per node and 8 CPU cores per node,
|
|
|
|
and would like to run on 4 nodes (32 cores) with dynamic balancing of
|
|
|
|
force calculation across CPU and GPU cores, you could specify
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<PRE>package gpu force/neigh 0 1 -1
|
|
|
|
</PRE>
|
|
|
|
<P>In this case, all CPU cores and GPU devices on the nodes would be
|
|
|
|
utilized. Each GPU device would be shared by 4 CPU cores. The CPU
|
|
|
|
cores would perform force calculations for some fraction of the
|
|
|
|
particles at the same time the GPUs performed force calculation for
|
|
|
|
the other particles.
|
|
|
|
</P>
|
2011-12-03 02:38:12 +08:00
|
|
|
<P>The <I>threads_per_atom</I> keyword allows control of the number of GPU
|
|
|
|
threads used per-atom to perform the short range force calculation.
|
|
|
|
By default, the value will be chosen based on the pair style, however,
|
|
|
|
the value can be set with this keyword to fine-tune performance. For
|
|
|
|
large cutoffs or with a small number of particles per GPU, increasing
|
|
|
|
the value can improve performance. The number of threads per atom must
|
|
|
|
be a power of 2 and currently cannot be greater than 32.
|
|
|
|
</P>
|
2012-09-22 00:04:48 +08:00
|
|
|
<P>The <I>cellsize</I> keyword can be used to control the size of the cells used
|
|
|
|
for binning atoms in neighbor list calculations. Setting this value is
|
|
|
|
normally not needed; the optimal value is close to the default
|
|
|
|
(equal to the cutoff distance for the short range interactions
|
|
|
|
plus the neighbor skin). GPUs can perform efficiently with much larger cutoffs
|
|
|
|
than CPUs and this can be used to reduce the time required for long-range
|
|
|
|
calculations or in some cases to eliminate them with models such as
|
|
|
|
<A HREF = "pair_coul.html">coul/wolf</A> or <A HREF = "pair_coul.html">coul/dsf</A>. For very large cutoffs,
|
|
|
|
it can be more efficient to use smaller values for cellsize in parallel
|
|
|
|
simulations. For example, with a cutoff of 20*sigma and a neighbor skin of
|
|
|
|
sigma, a cellsize of 5.25*sigma can be efficient for parallel simulations.
|
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<HR>
|
|
|
|
|
2011-06-01 07:08:32 +08:00
|
|
|
<P>The <I>cuda</I> style invokes options associated with the use of the
|
2011-12-02 23:47:30 +08:00
|
|
|
USER-CUDA package.
|
|
|
|
</P>
|
2011-12-03 02:38:12 +08:00
|
|
|
<P>The <I>gpu/node</I> keyword specifies the number <I>N</I> of GPUs to be used on
|
2011-12-02 23:47:30 +08:00
|
|
|
each node. An MPI process with rank <I>K</I> will use the GPU (K mod N).
|
|
|
|
This implies that processes should be assigned with successive ranks
|
|
|
|
on each node, which is the default with most (or even all) MPI
|
|
|
|
implementations. The default value for <I>N</I> is 2.
|
|
|
|
</P>
|
2011-12-03 02:38:12 +08:00
|
|
|
<P>The <I>gpu/node/special</I> keyword also specifies the number (N) of GPUs
|
2011-12-02 23:47:30 +08:00
|
|
|
to be used on each node, but allows more control over their
|
|
|
|
specification. An MPI process with rank <I>K</I> will use the GPU <I>gpuI</I>
|
|
|
|
with l = (K mod N) + 1. This implies that processes should be assigned
|
|
|
|
with successive ranks on each node, which is the default with most (or
|
|
|
|
even all) MPI implementations. For example if you have three GPUs on
|
|
|
|
a machine, one of which is used for the X-Server (the GPU with the ID
|
|
|
|
1) while the others (with IDs 0 and 2) are used for computations you
|
|
|
|
would specify:
|
|
|
|
</P>
|
|
|
|
<PRE>package cuda gpu/node/special 2 0 2
|
|
|
|
</PRE>
|
2011-12-03 02:38:12 +08:00
|
|
|
<P>A main purpose of the <I>gpu/node/special</I> optoin is to allow two (or
|
2011-12-02 23:47:30 +08:00
|
|
|
more) simulations to be run on one workstation. In that case one
|
|
|
|
would set the first simulation to use GPU 0 and the second to use GPU
|
|
|
|
1. This is not necessary though, if the GPUs are in what is called
|
|
|
|
<I>compute exclusive</I> mode. Using that setting, every process will get
|
|
|
|
its own GPU automatically. This <I>compute exclusive</I> mode can be set
|
|
|
|
as root using the <I>nvidia-smi</I> tool which is part of the CUDA
|
|
|
|
installation.
|
|
|
|
</P>
|
|
|
|
<P>Note that if the <I>gpu/node/special</I> keyword is not used, the USER-CUDA
|
|
|
|
package sorts existing GPUs on each node according to their number of
|
|
|
|
multiprocessors. This way, compute GPUs will be priorized over
|
|
|
|
X-Server GPUs.
|
|
|
|
</P>
|
|
|
|
<P>Use of the <I>timing</I> keyword will output detailed timing information
|
|
|
|
for various subroutines.
|
|
|
|
</P>
|
|
|
|
<P>The <I>test</I> keyword will output info for the the specified atom at
|
|
|
|
several points during each time step. This is mainly usefull for
|
|
|
|
debugging purposes. Note that the simulation will be severly slowed
|
|
|
|
down if this option is used.
|
|
|
|
</P>
|
|
|
|
<P>The <I>override/bpa</I> keyword can be used to specify which mode is used
|
|
|
|
for pair-force evaluation. TpA = one thread per atom; BpA = one block
|
|
|
|
per atom. If this keyword is not used, a short test at the begin of
|
|
|
|
each run will determine which method is more effective (the result of
|
|
|
|
this test is part of the LAMMPS output). Therefore it is usually not
|
2011-12-03 02:38:12 +08:00
|
|
|
necessary to use this keyword.
|
2011-08-17 22:20:30 +08:00
|
|
|
</P>
|
|
|
|
<HR>
|
|
|
|
|
|
|
|
<P>The <I>omp</I> style invokes options associated with the use of the
|
|
|
|
USER-OMP package.
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-12-01 23:35:32 +08:00
|
|
|
<P>The first argument allows to explicitly set the number of OpenMP
|
|
|
|
threads to be allocated for each MPI process. For example, if your
|
|
|
|
system has nodes with dual quad-core processors, it has a total of 8
|
|
|
|
cores per node. You could run MPI on 2 cores on each node (e.g. using
|
|
|
|
options for the mpirun command), and set the <I>Nthreads</I> setting to 4.
|
|
|
|
This would effectively use all 8 cores on each node. Since each MPI
|
|
|
|
process would spawn 4 threads (one of which runs as part of the MPI
|
|
|
|
process itself).
|
2011-08-17 22:20:30 +08:00
|
|
|
</P>
|
|
|
|
<P>For performance reasons, you should not set <I>Nthreads</I> to more threads
|
2011-12-01 23:35:32 +08:00
|
|
|
than there are physical cores (per MPI task), but LAMMPS cannot check
|
|
|
|
for this.
|
|
|
|
</P>
|
|
|
|
<P>An <I>Nthreads</I> value of '*' instructs LAMMPS to use whatever is the
|
|
|
|
default for the given OpenMP environment. This is usually determined
|
|
|
|
via the <I>OMP_NUM_THREADS</I> environment variable or the compiler
|
|
|
|
runtime. Please note that in most cases the default for OpenMP
|
|
|
|
capable compilers is to use one thread for each available CPU core
|
|
|
|
when <I>OMP_NUM_THREADS</I> is not set, which can lead to extremely bad
|
|
|
|
performance.
|
|
|
|
</P>
|
|
|
|
<P>Which combination of threads and MPI tasks gives the best performance
|
|
|
|
is difficult to predict and can depend on many components of your input.
|
|
|
|
Not all features of LAMMPS support OpenMP and the parallel efficiency
|
|
|
|
can be very different, too.
|
|
|
|
</P>
|
|
|
|
<P>The <I>mode</I> setting specifies where neighbor list calculations will be
|
|
|
|
multi-threaded as well. If <I>mode</I> is force, neighbor list calculation
|
|
|
|
is performed in serial. If <I>mode</I> is force/neigh, a multi-threaded
|
|
|
|
neighbor list build is used. Using the force/neigh setting is almost
|
|
|
|
always faster and should produce idential neighbor lists at the
|
|
|
|
expense of using some more memory (neighbor list pages are always
|
|
|
|
allocated for all threads at the same time and each thread works on
|
|
|
|
its own pages).
|
2011-08-17 22:20:30 +08:00
|
|
|
</P>
|
|
|
|
<HR>
|
|
|
|
|
2011-06-01 07:08:32 +08:00
|
|
|
<P><B>Restrictions:</B>
|
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<P>This command cannot be used after the simulation box is defined by a
|
|
|
|
<A HREF = "read_data.html">read_data</A> or <A HREF = "create_box.html">create_box</A> command.
|
|
|
|
</P>
|
2011-06-01 07:08:32 +08:00
|
|
|
<P>The cuda style of this command can only be invoked if LAMMPS was built
|
2011-08-26 00:46:23 +08:00
|
|
|
with the USER-CUDA package. See the <A HREF = "Section_start.html#start_3">Making
|
2011-12-02 23:47:30 +08:00
|
|
|
LAMMPS</A> section for more info.
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<P>The gpu style of this command can only be invoked if LAMMPS was built
|
2011-08-26 00:46:23 +08:00
|
|
|
with the GPU package. See the <A HREF = "Section_start.html#start_3">Making
|
2011-12-02 23:47:30 +08:00
|
|
|
LAMMPS</A> section for more info.
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-08-17 22:20:30 +08:00
|
|
|
<P>The omp style of this command can only be invoked if LAMMPS was built
|
2011-08-26 00:46:23 +08:00
|
|
|
with the USER-OMP package. See the <A HREF = "Section_start.html#start_3">Making
|
2011-12-02 23:47:30 +08:00
|
|
|
LAMMPS</A> section for more info.
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-12-02 01:51:50 +08:00
|
|
|
<P><B>Related commands:</B>
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2011-12-02 01:51:50 +08:00
|
|
|
<P><A HREF = "suffix.html">suffix</A>
|
|
|
|
</P>
|
|
|
|
<P><B>Default:</B>
|
|
|
|
</P>
|
2012-01-28 07:39:14 +08:00
|
|
|
<P>If the "-sf gpu" <A HREF = "Section_start.html#start_7">command-line switch</A> is
|
2011-12-02 23:47:30 +08:00
|
|
|
used then it is as if the command "package gpu force/neigh 0 0 1" were
|
|
|
|
invoked, to specify default settings for the GPU package. If the
|
|
|
|
command-line switch is not used, then no defaults are set, and you
|
|
|
|
must specify the appropriate package command in your input script.
|
|
|
|
</P>
|
|
|
|
<P>The default settings for the USER CUDA package are "package cuda gpu
|
2012-01-28 07:39:14 +08:00
|
|
|
2". This is the case whether the "-sf cuda" <A HREF = "Section_start.html#start_7">command-line
|
2011-12-02 23:47:30 +08:00
|
|
|
switch</A> is used or not.
|
2011-06-01 07:08:32 +08:00
|
|
|
</P>
|
2012-01-28 07:39:14 +08:00
|
|
|
<P>If the "-sf omp" <A HREF = "Section_start.html#start_7">command-line switch</A> is
|
2011-12-02 23:47:30 +08:00
|
|
|
used then it is as if the command "package omp *" were invoked, to
|
|
|
|
specify default settings for the USER-OMP package. If the
|
|
|
|
command-line switch is not used, then no defaults are set, and you
|
|
|
|
must specify the appropriate package command in your input script.
|
2011-12-02 01:57:44 +08:00
|
|
|
</P>
|
2011-06-01 07:08:32 +08:00
|
|
|
</HTML>
|