2006-09-22 00:22:34 +08:00
|
|
|
<HTML>
|
|
|
|
|
<CENTER><A HREF = "http://lammps.sandia.gov">LAMMPS WWW Site</A> - <A HREF = "Manual.html">LAMMPS Documentation</A> - <A HREF = "Section_commands.html#comm">LAMMPS Commands</A>
|
|
|
|
|
</CENTER>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<HR>
|
|
|
|
|
|
|
|
|
|
<H3>processors command
|
|
|
|
|
</H3>
|
|
|
|
|
<P><B>Syntax:</B>
|
|
|
|
|
</P>
|
2011-12-08 07:43:51 +08:00
|
|
|
<PRE>processors Px Py Pz keyword args ...
|
2006-09-22 00:22:34 +08:00
|
|
|
</PRE>
|
2011-12-14 01:03:43 +08:00
|
|
|
<UL><LI>Px,Py,Pz = # of processors in each dimension of 3d grid overlaying the simulation domain
|
2011-12-08 07:43:51 +08:00
|
|
|
|
|
|
|
|
<LI>zero or more keyword/arg pairs may be appended
|
|
|
|
|
|
2011-12-14 01:03:43 +08:00
|
|
|
<LI>keyword = <I>grid</I> or <I>map</I> or <I>part</I> or <I>file</I>
|
2011-12-08 07:43:51 +08:00
|
|
|
|
2011-12-14 01:03:43 +08:00
|
|
|
<PRE> <I>grid</I> arg = gstyle params ...
|
|
|
|
|
gstyle = <I>onelevel</I> or <I>twolevel</I> or <I>numa</I> or <I>custom</I>
|
|
|
|
|
onelevel params = none
|
|
|
|
|
twolevel params = Cx Cy Cz
|
|
|
|
|
Cx,Cy,Cz = # of cores in each dimension of 3d sub-grid assigned to each node
|
|
|
|
|
numa params = none
|
|
|
|
|
custom params = fname
|
|
|
|
|
fname = file containing grid layout
|
|
|
|
|
<I>map</I> arg = <I>cart</I> or <I>cart/reorder</I> or <I>xyz</I> or <I>xzy</I> or <I>yxz</I> or <I>yzx</I> or <I>zxy</I> or <I>zyx</I>
|
|
|
|
|
cart = use MPI_Cart() methods to map processors to 3d grid with reorder = 0
|
|
|
|
|
cart/reorder = use MPI_Cart() methods to map processors to 3d grid with reorder = 1
|
|
|
|
|
xyz,xzy,yxz,yzx,zxy,zyx = map procesors to 3d grid in IJK ordering
|
2011-12-09 06:50:07 +08:00
|
|
|
<I>numa</I> arg = none
|
|
|
|
|
<I>part</I> args = Psend Precv cstyle
|
2011-12-08 07:43:51 +08:00
|
|
|
Psend = partition # (1 to Np) which will send its processor layout
|
|
|
|
|
Precv = partition # (1 to Np) which will recv the processor layout
|
|
|
|
|
cstyle = <I>multiple</I>
|
2011-12-14 01:03:43 +08:00
|
|
|
<I>multiple</I> = Psend grid will be multiple of Precv grid in each dimension
|
|
|
|
|
<I>file</I> arg = outname
|
|
|
|
|
outname = name of file to write 3d grid of processors to
|
2011-12-08 07:43:51 +08:00
|
|
|
</PRE>
|
|
|
|
|
|
2006-09-22 00:22:34 +08:00
|
|
|
</UL>
|
|
|
|
|
<P><B>Examples:</B>
|
|
|
|
|
</P>
|
2011-12-13 23:58:47 +08:00
|
|
|
<PRE>processors * * 5
|
|
|
|
|
processors 2 4 4
|
2011-12-14 01:03:43 +08:00
|
|
|
processors * * 8 map xyz
|
|
|
|
|
processors * * * grid numa
|
|
|
|
|
processors 4 8 16 grid custom myfile
|
2011-12-08 07:43:51 +08:00
|
|
|
processors * * * part 1 2 multiple
|
2006-09-22 00:22:34 +08:00
|
|
|
</PRE>
|
|
|
|
|
<P><B>Description:</B>
|
|
|
|
|
</P>
|
|
|
|
|
<P>Specify how processors are mapped as a 3d logical grid to the global
|
2011-12-09 06:50:07 +08:00
|
|
|
simulation box. This involves 2 steps. First if there are P
|
|
|
|
|
processors it means choosing a factorization P = Px by Py by Pz so
|
|
|
|
|
that there are Px processors in the x dimension, and similarly for the
|
|
|
|
|
y and z dimensions. Second, the P processors (with MPI ranks 0 to
|
2011-12-13 23:58:47 +08:00
|
|
|
P-1) are mapped to the logical 3d grid. The arguments to this command
|
|
|
|
|
control each of these 2 steps.
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
|
|
|
|
<P>The Px, Py, Pz parameters affect the factorization. Any of the 3
|
|
|
|
|
parameters can be specified with an asterisk "*", which means LAMMPS
|
2011-12-13 23:58:47 +08:00
|
|
|
will choose the number of processors in that dimension of the grid.
|
|
|
|
|
It will do this based on the size and shape of the global simulation
|
|
|
|
|
box so as to minimize the surface-to-volume ratio of each processor's
|
|
|
|
|
sub-domain.
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
|
|
|
|
<P>Since LAMMPS does not load-balance by changing the grid of 3d
|
2011-12-13 23:58:47 +08:00
|
|
|
processors on-the-fly, choosing explicit values for Px or Py or Pz can
|
|
|
|
|
be used to override the LAMMPS default if it is known to be
|
|
|
|
|
sub-optimal for a particular problem. E.g. a problem where the extent
|
|
|
|
|
of atoms will change dramatically in a particular dimension over the
|
|
|
|
|
course of the simulation.
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
|
|
|
|
<P>The product of Px, Py, Pz must equal P, the total # of processors
|
2010-05-21 00:03:29 +08:00
|
|
|
LAMMPS is running on. For a <A HREF = "dimension.html">2d simulation</A>, Pz must
|
2011-12-13 23:58:47 +08:00
|
|
|
equal 1.
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
2010-05-21 00:03:29 +08:00
|
|
|
<P>Note that if you run on a large, prime number of processors P, then a
|
|
|
|
|
grid such as 1 x P x 1 will be required, which may incur extra
|
2011-12-09 06:50:07 +08:00
|
|
|
communication costs due to the high surface area of each processor's
|
|
|
|
|
sub-domain.
|
|
|
|
|
</P>
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>Also note that if multiple partitions are being used then P is the
|
|
|
|
|
number of processors in this partition; see <A HREF = "Section_start.html#start_6">this
|
|
|
|
|
section</A> for an explanation of the
|
|
|
|
|
-partition command-line switch. Also note that you can prefix the
|
|
|
|
|
processors command with the <A HREF = "partition.html">partition</A> command to
|
|
|
|
|
easily specify different Px,Py,Pz values for different partitions.
|
|
|
|
|
</P>
|
|
|
|
|
<P>You can use the <A HREF = "partition.html">partition</A> command to specify
|
|
|
|
|
different processor grids for different partitions, e.g.
|
|
|
|
|
</P>
|
|
|
|
|
<PRE>partition yes 1 processors 4 4 4
|
|
|
|
|
partition yes 2 processors 2 3 2
|
|
|
|
|
</PRE>
|
2011-12-09 06:50:07 +08:00
|
|
|
<HR>
|
|
|
|
|
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>The <I>grid</I> keyword affects how the P processor IDs (from 0 to P-1) are
|
|
|
|
|
mapped to the 3d grid of processors.
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>The <I>cart</I> style uses the family of MPI Cartesian functions to perform
|
|
|
|
|
the mapping, namely MPI_Cart_create(), MPI_Cart_get(),
|
|
|
|
|
MPI_Cart_shift(), and MPI_Cart_rank(). It invokes the
|
|
|
|
|
MPI_Cart_create() function with its reorder flag = 0, so that MPI is
|
|
|
|
|
not free to reorder the processors.
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
|
|
|
|
<P>The <I>cart/reorder</I> style does the same thing as the <I>cart</I> style
|
2011-12-13 23:58:47 +08:00
|
|
|
except it sets the reorder flag to 1, so that MPI can reorder
|
2011-12-09 06:50:07 +08:00
|
|
|
processors if it desires.
|
|
|
|
|
</P>
|
|
|
|
|
<P>The <I>xyz</I>, <I>xzy</I>, <I>yxz</I>, <I>yzx</I>, <I>zxy</I>, and <I>zyx</I> styles are all
|
2011-12-13 23:58:47 +08:00
|
|
|
similar. If the style is IJK, then it maps the P processors to the
|
|
|
|
|
grid so that the processor ID in the I direction varies fastest, the
|
|
|
|
|
processor ID in the J direction varies next fastest, and the processor
|
|
|
|
|
ID in the K direction varies slowest. For example, if you select
|
|
|
|
|
style <I>xyz</I> and you have a 2x2x2 grid of 8 processors, the assignments
|
|
|
|
|
of the 8 octants of the simulation domain will be:
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
|
|
|
|
<PRE>proc 0 = lo x, lo y, lo z octant
|
|
|
|
|
proc 1 = hi x, lo y, lo z octant
|
|
|
|
|
proc 2 = lo x, hi y, lo z octant
|
|
|
|
|
proc 3 = hi x, hi y, lo z octant
|
|
|
|
|
proc 4 = lo x, lo y, hi z octant
|
|
|
|
|
proc 5 = hi x, lo y, hi z octant
|
|
|
|
|
proc 6 = lo x, hi y, hi z octant
|
|
|
|
|
proc 7 = hi x, hi y, hi z octant
|
|
|
|
|
</PRE>
|
|
|
|
|
<P>Note that, in principle, an MPI implementation on a particular machine
|
|
|
|
|
should be aware of both the machine's network topology and the
|
|
|
|
|
specific subset of processors and nodes that were assigned to your
|
|
|
|
|
simulation. Thus its MPI_Cart calls can optimize the assignment of
|
2011-12-13 23:58:47 +08:00
|
|
|
MPI processes to the 3d grid to minimize communication costs. In
|
|
|
|
|
practice, however, few if any MPI implementations actually do this.
|
|
|
|
|
So it is likely that the <I>cart</I> and <I>cart/reorder</I> styles simply give
|
|
|
|
|
the same result as one of the IJK styles.
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
|
|
|
|
<HR>
|
|
|
|
|
|
|
|
|
|
<P>The <I>numa</I> keyword affects both the factorization of P into Px,Py,Pz
|
|
|
|
|
and the mapping of processors to the 3d grid.
|
|
|
|
|
</P>
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>It operates similar to the <I>level2</I> and <I>level3</I> keywords except that
|
|
|
|
|
it tries to auto-detect the count and topology of the processors and
|
|
|
|
|
cores within a node. Currently, it does this in only 2 levels
|
|
|
|
|
(assumes the proces/node = 1), but it may be extended in the future.
|
|
|
|
|
</P>
|
|
|
|
|
<P>It also uses a different algorithm (iterative) than the <I>level2</I>
|
|
|
|
|
keyword for doing the two-level factorization of the simulation box
|
|
|
|
|
into a 3d processor grid to minimize off-node communication. Thus it
|
|
|
|
|
may give a differnet or improved mapping of processors to the 3d grid.
|
|
|
|
|
</P>
|
|
|
|
|
<P>The numa setting will give an error if the number of MPI processes
|
|
|
|
|
is not evenly divisible by the number of cores used per node.
|
2011-12-09 06:50:07 +08:00
|
|
|
</P>
|
|
|
|
|
<P>The numa setting will be ignored if (a) there are less than 4 cores
|
|
|
|
|
per node, or (b) the number of MPI processes is not divisible by the
|
|
|
|
|
number of cores used per node, or (c) only 1 node is allocated, or (d)
|
|
|
|
|
any of the Px or Py of Pz values is greater than 1.
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
2011-12-08 07:43:51 +08:00
|
|
|
<HR>
|
|
|
|
|
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>The <I>part</I> keyword affects the factorization of P into Px,Py,Pz.
|
|
|
|
|
</P>
|
|
|
|
|
<P>It can be useful when running in multi-partition mode, e.g. with the
|
|
|
|
|
<A HREF = "run_style.html">run_style verlet/split</A> command. It specifies a
|
|
|
|
|
dependency bewteen a sending partition <I>Psend</I> and a receiving
|
|
|
|
|
partition <I>Precv</I> which is enforced when each is setting up their own
|
|
|
|
|
mapping of their processors to the simulation box. Each of <I>Psend</I>
|
|
|
|
|
and <I>Precv</I> must be integers from 1 to Np, where Np is the number of
|
|
|
|
|
partitions you have defined via the <A HREF = "Section_start.html#start_6">-partition command-line
|
|
|
|
|
switch</A>.
|
2011-12-08 07:43:51 +08:00
|
|
|
</P>
|
|
|
|
|
<P>A "dependency" means that the sending partition will create its 3d
|
|
|
|
|
logical grid as Px by Py by Pz and after it has done this, it will
|
|
|
|
|
send the Px,Py,Pz values to the receiving partition. The receiving
|
|
|
|
|
partition will wait to receive these values before creating its own 3d
|
|
|
|
|
logical grid and will use the sender's Px,Py,Pz values as a
|
|
|
|
|
constraint. The nature of the constraint is determined by the
|
|
|
|
|
<I>cstyle</I> argument.
|
|
|
|
|
</P>
|
|
|
|
|
<P>For a <I>cstyle</I> of <I>multiple</I>, each dimension of the sender's processor
|
|
|
|
|
grid is required to be an integer multiple of the corresponding
|
|
|
|
|
dimension in the receiver's processor grid. This is a requirement of
|
|
|
|
|
the <A HREF = "run_style.html">run_style verlet/split</A> command.
|
|
|
|
|
</P>
|
|
|
|
|
<P>For example, assume the sending partition creates a 4x6x10 grid = 240
|
|
|
|
|
processor grid. If the receiving partition is running on 80
|
|
|
|
|
processors, it could create a 4x2x10 grid, but it will not create a
|
|
|
|
|
2x4x10 grid, since in the y-dimension, 6 is not an integer multiple of
|
|
|
|
|
4.
|
|
|
|
|
</P>
|
|
|
|
|
<P>IMPORTANT NOTE: If you use the <A HREF = "partition.html">partition</A> command to
|
|
|
|
|
invoke different "processsors" commands on different partitions, and
|
|
|
|
|
you also use the <I>part</I> keyword, then you must insure that both the
|
|
|
|
|
sending and receiving partitions invoke the "processors" command that
|
|
|
|
|
connects the 2 partitions via the <I>part</I> keyword. LAMMPS cannot
|
|
|
|
|
easily check for this, but your simulation will likely hang in its
|
|
|
|
|
setup phase if this error has been made.
|
|
|
|
|
</P>
|
2011-12-08 08:00:59 +08:00
|
|
|
<HR>
|
|
|
|
|
|
2011-12-13 23:58:47 +08:00
|
|
|
<P>The <I>out</I> keyword writes the mapping of the factorization of P
|
|
|
|
|
processors and their mapping to the 3d grid to the specified file
|
|
|
|
|
<I>fname</I>. This is useful to check that you assigned physical
|
|
|
|
|
processors in the manner you desired, which can be tricky to figure
|
|
|
|
|
out, especially when running on multiple partitions or on a multicore
|
|
|
|
|
machine or when the processor ranks were reordered by use of the
|
|
|
|
|
<A HREF = "Section_start.html#start_6">-reorder command-line switch</A> or due to
|
|
|
|
|
use of MPI-specific launch options such as a config file.
|
|
|
|
|
</P>
|
|
|
|
|
<P>If you have multiple partitions you should insure that each one writes
|
|
|
|
|
to a different file, e.g. using a <A HREF = "variable.html">world-style variable</A>
|
|
|
|
|
for the filename. The file will have a self-explanatory header,
|
|
|
|
|
followed by one-line per processor in this format:
|
|
|
|
|
</P>
|
|
|
|
|
<P>I J K: world-ID universe-ID original-ID: name
|
|
|
|
|
</P>
|
|
|
|
|
<P>I,J,K are the indices of the processor in the 3d logical grid. The
|
|
|
|
|
IDs are the processor's rank in this simulation (the world), the
|
|
|
|
|
universe (of multiple simulations), and the original MPI communicator
|
|
|
|
|
used to instantiate LAMMPS, respectively. The world and universe IDs
|
|
|
|
|
will only be different if you are running on more than one partition;
|
|
|
|
|
see the <A HREF = "Section_start.html#start_6">-partition command-line switch</A>.
|
|
|
|
|
The universe and original IDs will only be different if you used the
|
|
|
|
|
<A HREF = "Section_start.html#start_6">-reorder command-line switch</A> to reorder
|
|
|
|
|
the processors differently than their rank in the original
|
|
|
|
|
communicator LAMMPS was instantiated with. The <I>name</I> is what is
|
|
|
|
|
returned by a call to MPI_Get_processor_name() and should represent an
|
|
|
|
|
identifier relevant to the physical processors in your machine. Note
|
|
|
|
|
that depending on the MPI implementation, multiple cores can have the
|
|
|
|
|
same <I>name</I>.
|
|
|
|
|
</P>
|
|
|
|
|
<HR>
|
|
|
|
|
|
2006-09-22 00:22:34 +08:00
|
|
|
<P><B>Restrictions:</B>
|
|
|
|
|
</P>
|
|
|
|
|
<P>This command cannot be used after the simulation box is defined by a
|
|
|
|
|
<A HREF = "read_data.html">read_data</A> or <A HREF = "create_box.html">create_box</A> command.
|
|
|
|
|
It can be used before a restart file is read to change the 3d
|
|
|
|
|
processor grid from what is specified in the restart file.
|
|
|
|
|
</P>
|
2011-12-14 01:03:43 +08:00
|
|
|
<P>The <I>grid numa</I> keyword only currently works with the <I>map cart</I> option.
|
2011-12-13 23:58:47 +08:00
|
|
|
</P>
|
|
|
|
|
<P><B>Related commands:</B>
|
2011-12-10 00:47:47 +08:00
|
|
|
</P>
|
2011-12-13 23:58:47 +08:00
|
|
|
<P><A HREF = "partition.html">partition</A>, <A HREF = "Section_start.html#start_6">-reorder command-line
|
|
|
|
|
switch</A>
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
|
|
|
|
<P><B>Default:</B>
|
|
|
|
|
</P>
|
2011-12-14 01:03:43 +08:00
|
|
|
<P>The option defaults are Px Py Pz = * * *, grid = level1, and map = cart.
|
2006-09-22 00:22:34 +08:00
|
|
|
</P>
|
|
|
|
|
</HTML>
|
