forked from OSchip/llvm-project
[Clang][Docs] Document the clang-offload-packager better
Summary: This patch adds more in-depth documentation to the clang-offload-packacker's binary format. This format is used to create fat binaries and link them.
This commit is contained in:
Joseph Huber
parent
c797952d4f
commit
e22b02d9b4
|
|
@ -16,21 +16,130 @@ together. The image format is a small header wrapping around a string map. This
|
|||
tool creates bundled binaries so that they can be embedded into the host to
|
||||
create a fat-binary.
|
||||
|
||||
An embedded binary is marked by the ``0x10FF10AD`` magic bytes, followed by a
|
||||
Binary Format
|
||||
=============
|
||||
|
||||
The binary format is marked by the ``0x10FF10AD`` magic bytes, followed by a
|
||||
version. Each created binary contains its own magic bytes. This allows us to
|
||||
locate all the embedded offloading sections even after they may have been merged
|
||||
by the linker, such as when using relocatable linking. The format used is
|
||||
primarily a binary serialization of the following struct.
|
||||
by the linker, such as when using relocatable linking. Conceptually, this binary
|
||||
format is a serialization of a string map and an image buffer. The binary header
|
||||
is described in the following :ref:`table<table-binary_header>`.
|
||||
|
||||
.. code-block:: c++
|
||||
.. table:: Offloading Binary Header
|
||||
:name: table-binary_header
|
||||
|
||||
struct OffloadingImage {
|
||||
uint16_t TheImageKind;
|
||||
uint16_t TheOffloadKind;
|
||||
uint32_t Flags;
|
||||
StringMap<StringRef> StringData;
|
||||
MemoryBufferRef Image;
|
||||
};
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
| Type | Identifier | Description |
|
||||
+==========+==============+====================================================+
|
||||
| uint8_t | magic | The magic bytes for the binary format (0x10FF10AD) |
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
| uint32_t | version | Version of this format (currently version 1) |
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
| uint64_t | size | Size of this binary in bytes |
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
| uint64_t | entry offset | Absolute offset of the offload entries in bytes |
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
| uint64_t | entry size | Size of the offload entries in bytes |
|
||||
+----------+--------------+----------------------------------------------------+
|
||||
|
||||
Once identified through the magic bytes, we use the size field to take a slice
|
||||
of the binary blob containing the information for a single offloading image. We
|
||||
can then use the offset field to find the actual offloading entries containing
|
||||
the image and metadata. The offload entry contains information about the device
|
||||
image. It contains the fields shown in the following
|
||||
:ref:`table<table-binary_entry>`.
|
||||
|
||||
.. table:: Offloading Entry Table
|
||||
:name: table-binary_entry
|
||||
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| Type | Identifier | Description |
|
||||
+==========+===============+====================================================+
|
||||
| uint16_t | image kind | The kind of the device image (e.g. bc, cubin) |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint16_t | offload kind | The producer of the image (e.g. openmp, cuda) |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint32_t | flags | Generic flags for the image |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint64_t | string offset | Absolute offset of the string metadata table |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint64_t | num strings | Number of string entries in the table |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint64_t | image offset | Absolute offset of the device image in bytes |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
| uint64_t | image size | Size of the device image in bytes |
|
||||
+----------+---------------+----------------------------------------------------+
|
||||
|
||||
This table contains the offsets of the string table and the device image itself
|
||||
along with some other integer information. The image kind lets us easily
|
||||
identify the type of image stored here without needing to inspect the binary.
|
||||
The offloading kind is used to determine which registration code or linking
|
||||
semantics are necessary for this image. These are stored as enumerations with
|
||||
the following values for the :ref:`offload kind<table-offload_kind` and the
|
||||
:ref:`image kind<table-image_kind>`.
|
||||
|
||||
.. table:: Image Kind
|
||||
:name: table-image_kind
|
||||
|
||||
+---------------+-------+---------------------------------------+
|
||||
| Name | Value | Description |
|
||||
+===============+=======+=======================================+
|
||||
| IMG_None | 0x00 | No image information provided |
|
||||
+---------------+-------+---------------------------------------+
|
||||
| IMG_Object | 0x01 | The image is a generic object file |
|
||||
+---------------+-------+---------------------------------------+
|
||||
| IMG_Bitcode | 0x02 | The image is an LLVM-IR bitcode file |
|
||||
+---------------+-------+---------------------------------------+
|
||||
| IMG_Cubin | 0x03 | The image is a CUDA object file |
|
||||
+---------------+-------+---------------------------------------+
|
||||
| IMG_Fatbinary | 0x04 | The image is a CUDA fatbinary file |
|
||||
+---------------+-------+---------------------------------------+
|
||||
| IMG_PTX | 0x05 | The iamge is a CUDA PTX file |
|
||||
+---------------+-------+---------------------------------------+
|
||||
|
||||
.. table:: Offload Kind
|
||||
:name: table-offload_kind
|
||||
|
||||
+------------+-------+---------------------------------------+
|
||||
| Name | Value | Description |
|
||||
+============+=======+=======================================+
|
||||
| OFK_None | 0x00 | No offloading information provided |
|
||||
+------------+-------+---------------------------------------+
|
||||
| OFK_OpenMP | 0x01 | The producer was OpenMP offloading |
|
||||
+------------+-------+---------------------------------------+
|
||||
| OFK_CUDA | 0x02 | The producer was CUDA |
|
||||
+------------+-------+---------------------------------------+
|
||||
| OFK_HIP | 0x03 | The producer was HIP |
|
||||
+------------+-------+---------------------------------------+
|
||||
|
||||
The flags are used to signify certain conditions, such as the presence of
|
||||
debugging information or whether or not LTO was used. The string entry table is
|
||||
used to generically contain any arbitrary key-value pair. This is stored as an
|
||||
array of the :ref:`string entry<table-binary_string>` format.
|
||||
|
||||
.. table:: Offloading String Entry
|
||||
:name: table-binary_string
|
||||
|
||||
+----------+--------------+-------------------------------------------------------+
|
||||
| Type | Identifier | Description |
|
||||
+==========+==============+=======================================================+
|
||||
| uint64_t | key offset | Absolute byte offset of the key in th string table |
|
||||
+----------+--------------+-------------------------------------------------------+
|
||||
| uint64_t | value offset | Absolute byte offset of the value in the string table |
|
||||
+----------+--------------+-------------------------------------------------------+
|
||||
|
||||
The string entries simply provide offsets to a key and value pair in the
|
||||
binary images string table. The string table is simply a collection of null
|
||||
terminated strings with defined offsets in the image. The string entry allows us
|
||||
to create a key-value pair from this string table. This is used for passing
|
||||
arbitrary arguments to the image, such as the triple and architecture.
|
||||
|
||||
All of these structures are combined to form a single binary blob, the order
|
||||
does not matter because of the use of absolute offsets. This makes it easier to
|
||||
extend in the future. As mentioned previously, multiple offloading images are
|
||||
bundled together by simply concatenating them in this format. Because we have
|
||||
the magic bytes and size of each image, we can extract them as-needed.
|
||||
|
||||
Usage
|
||||
=====
|
||||
|
|
|
|||
Loading…
Reference in New Issue