287 lines
12 KiB
Markdown
287 lines
12 KiB
Markdown
|
Go 1.11 release introduces [AVX-512](https://en.wikipedia.org/wiki/AVX-512) support.
|
||
|
|
This page describes how to use new features as well as some important encoder details.
|
||
|
|
|
||
|
|
### Terminology
|
||
|
|
|
||
|
|
Most terminology comes from [Intel Software Developer's manual](https://software.intel.com/en-us/articles/intel-sdm).
|
||
|
|
Suffixes originate from Go assembler syntax, which is close to AT&T, which also uses size suffixes.
|
||
|
|
|
||
|
|
Some terms are listed to avoid ambiguity (for example, opcode can have different meanings).
|
||
|
|
|
||
|
|
<table>
|
||
|
|
<tr>
|
||
|
|
<th>Term</th>
|
||
|
|
<th>Description</th>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Operand</td>
|
||
|
|
<td>
|
||
|
|
Same as "instruction argument".
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Opcode</td>
|
||
|
|
<td>
|
||
|
|
Name that refers to instruction group. For example, <code>VADDPD</code> is an opcode.<br>
|
||
|
|
It refers to both VEX and EVEX encoded forms and all operand combinations.<br>
|
||
|
|
Most Go assembler opcodes for AVX-512 match Intel manual entries, with exceptions for cases<br>
|
||
|
|
where additional size suffix is used (e.g. <code>VCVTTPD2DQY</code> is <code>VCVTTPD2DQ</code>).
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Opcode suffix</td>
|
||
|
|
<td>
|
||
|
|
Suffix that overrides some opcode properties. Listed after "." (dot).<br>
|
||
|
|
For example, <code>VADDPD.Z</code> has "Z" opcode suffix.<br>
|
||
|
|
There can be multiple dot-separated opcode suffixes.
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Size suffix</td>
|
||
|
|
<td>
|
||
|
|
Suffix that specifies instruction operand size if it can't be inferred from operands alone.<br>
|
||
|
|
For example, <code>VCVTSS2USIL</code> has "L" size suffix.
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Opmask</td>
|
||
|
|
<td>
|
||
|
|
Used for both <code>{k1}</code> notation and to describe instructions that have <code>K</code> registers operands.<br>
|
||
|
|
Related to masking support in EVEX prefix.
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>Register block</td>
|
||
|
|
<td>
|
||
|
|
Multi-source operand that encodes register range.<br>
|
||
|
|
Intel manual uses <code>+n</code> notation for register blocks.<br>
|
||
|
|
For example, <code>+3</code> is a register block of 4 registers.
|
||
|
|
</td>
|
||
|
|
</tr>
|
||
|
|
<tr>
|
||
|
|
<td>FP</td>
|
||
|
|
<td>Floating-point</td>
|
||
|
|
</tr>
|
||
|
|
</table>
|
||
|
|
|
||
|
|
### New registers
|
||
|
|
|
||
|
|
EVEX-enabled instructions can access additional 16 `X` (128-bit xmm) and `Y` (256-bit ymm) registers, plus 32 new `Z` (512-bit zmm) registers in 64-bit mode. 32-bit mode only gets `Z0-Z7`.
|
||
|
|
|
||
|
|
New opmask registers are named `K0-K7`.
|
||
|
|
They can be used for both masking and for special opmask instructions (like `KADDB`).
|
||
|
|
|
||
|
|
### Masking support
|
||
|
|
|
||
|
|
Instructions that support masking can omit `K` register operand.
|
||
|
|
In this case, `K0` register is implied ("all ones") and merging-masking is performed.
|
||
|
|
This is effectively "no masking".
|
||
|
|
|
||
|
|
`K1-K7` registers can be used to override default opmask.
|
||
|
|
`K` register should be placed right before destination operand.
|
||
|
|
|
||
|
|
Zeroing-masking can be activated with `Z` opcode suffix. Zeroing-masking requires that a mask register other than K0 be specified.
|
||
|
|
|
||
|
|
For example, `VADDPD.Z (AX), Z30, K3, Z10` uses zeroing-masking and explicit `K` register.
|
||
|
|
- If `Z` opcode suffix is removed, it's merging-masking with `K3` mask.
|
||
|
|
- If `K3` operand is removed, it generates an assembler error.
|
||
|
|
- If both `Z` opcode suffix and `K3` operand are removed, it is merging-masking with `K0` mask.
|
||
|
|
|
||
|
|
It's compile-time error to use `K0` register for `{k1}` operands (consult [manuals](https://software.intel.com/en-us/articles/intel-sdm) for details).
|
||
|
|
|
||
|
|
### EVEX broadcast/rounding/SAE support
|
||
|
|
|
||
|
|
Embedded broadcast, rounding and SAE activated through opcode suffixes.
|
||
|
|
|
||
|
|
For reg-reg FP instructions with `{er}` enabled, rounding opcode suffix can be specified:
|
||
|
|
|
||
|
|
* `RU_SAE` to round towards +Inf
|
||
|
|
* `RD_SAE` to round towards -Inf
|
||
|
|
* `RZ_SAE` to round towards zero
|
||
|
|
* `RN_SAE` to round towards nearest
|
||
|
|
|
||
|
|
> To read more about rounding modes, see [MXCSR.RC info](http://qcd.phys.cmu.edu/QCDcluster/intel/vtune/reference/vc148.htm).
|
||
|
|
|
||
|
|
For reg-reg FP instructions with `{sae}` enabled, exception suppression can be specified with `SAE` opcode suffix.
|
||
|
|
|
||
|
|
For reg-mem instrictons with `m32bcst/m64bcst` operand, broadcasting can be turned on with `BCST` opcode suffix.
|
||
|
|
|
||
|
|
Zeroing opcode suffix can be combined with any of these.
|
||
|
|
For example, `VMAXPD.SAE.Z Z3, Z2, Z1` uses both `Z` and `SAE` opcode suffixes.
|
||
|
|
It is important to put zeroing opcode suffix last, otherwise it is a compilation error.
|
||
|
|
|
||
|
|
### Register block (multi-source) operands
|
||
|
|
|
||
|
|
Register blocks are specified using register range syntax.
|
||
|
|
|
||
|
|
It would be enough to specify just first (low) register, but Go assembler requires
|
||
|
|
explicit range with both ends for readability reasons.
|
||
|
|
|
||
|
|
For example, instructions with `+3` range can be used like `VP4DPWSSD Z25, [Z0-Z3], (AX)`.
|
||
|
|
Range `[Z0-Z3]` reads like "register block of Z0, Z1, Z2, Z3".
|
||
|
|
Invalid ranges result in compilation error.
|
||
|
|
|
||
|
|
### AVX1 and AVX2 instructions with EVEX prefix
|
||
|
|
|
||
|
|
Previously existed opcodes that can be encoded using EVEX prefix now can access AVX-512 features like wider register file, zeroing/merging masking, etc. For example, `VADDPD` can now use 512-bit vector registers.
|
||
|
|
|
||
|
|
See [encoder details](#encoder-details) for more info.
|
||
|
|
|
||
|
|
### Supported extensions
|
||
|
|
|
||
|
|
Best way to get up-to-date list of supported extensions is to do `ls -1` inside [test suite](https://github.com/golang/go/tree/master/src/cmd/asm/internal/asm/testdata/avx512enc) directory.
|
||
|
|
|
||
|
|
Latest list includes:
|
||
|
|
```
|
||
|
|
aes_avx512f
|
||
|
|
avx512_4fmaps
|
||
|
|
avx512_4vnniw
|
||
|
|
avx512_bitalg
|
||
|
|
avx512_ifma
|
||
|
|
avx512_vbmi
|
||
|
|
avx512_vbmi2
|
||
|
|
avx512_vnni
|
||
|
|
avx512_vpopcntdq
|
||
|
|
avx512bw
|
||
|
|
avx512cd
|
||
|
|
avx512dq
|
||
|
|
avx512er
|
||
|
|
avx512f
|
||
|
|
avx512pf
|
||
|
|
gfni_avx512f
|
||
|
|
vpclmulqdq_avx512f
|
||
|
|
```
|
||
|
|
|
||
|
|
128-bit and 256-bit instructions additionally require `avx512vl`.
|
||
|
|
That is, if `VADDPD` is available in `avx512f`, you can't use `X` and `Y` arguments
|
||
|
|
without `avx512vl`.
|
||
|
|
|
||
|
|
Filenames follow `GNU as` (gas) conventions.
|
||
|
|
[avx512extmap.csv](https://gist.github.com/Quasilyte/92321dadcc3f86b05c1aeda2c13c851f) can make naming scheme more apparent.
|
||
|
|
|
||
|
|
### Instructions with size suffix
|
||
|
|
|
||
|
|
Some opcodes do not match Intel manual entries.
|
||
|
|
This section is provided for search convenience.
|
||
|
|
|
||
|
|
| Intel opcode | Go assembler opcodes |
|
||
|
|
|--------------|----------------------|
|
||
|
|
| `VCVTPD2DQ` | `VCVTPD2DQX`, `VCVTPD2DQY` |
|
||
|
|
| `VCVTPD2PS` | `VCVTPD2PSX`, `VCVTPD2PSY` |
|
||
|
|
| `VCVTTPD2DQ` | `VCVTTPD2DQX`, `VCVTTPD2DQY` |
|
||
|
|
| `VCVTQQ2PS` | `VCVTQQ2PSX`, `VCVTQQ2PSY` |
|
||
|
|
| `VCVTUQQ2PS` | `VCVTUQQ2PSX`, `VCVTUQQ2PSY` |
|
||
|
|
| `VCVTPD2UDQ` | `VCVTPD2UDQX`, `VCVTPD2UDQY` |
|
||
|
|
| `VCVTTPD2UDQ` | `VCVTTPD2UDQX`, `VCVTTPD2UDQY` |
|
||
|
|
| `VFPCLASSPD` | `VFPCLASSPDX`, `VFPCLASSPDY`, `VFPCLASSPDZ` |
|
||
|
|
| `VFPCLASSPS` | `VFPCLASSPSX`, `VFPCLASSPSY`, `VFPCLASSPSZ` |
|
||
|
|
| `VCVTSD2SI` | `VCVTSD2SI`, `VCVTSD2SIQ` |
|
||
|
|
| `VCVTTSD2SI` | `VCVTSD2SI`, `VCVTSD2SIQ` |
|
||
|
|
| `VCVTTSS2SI` | `VCVTSD2SI`, `VCVTSD2SIQ` |
|
||
|
|
| `VCVTSS2SI` | `VCVTSD2SI`, `VCVTSD2SIQ` |
|
||
|
|
| `VCVTSD2USI` | `VCVTSD2USIL`, `VCVTSD2USIQ` |
|
||
|
|
| `VCVTSS2USI` | `VCVTSS2USIL`, `VCVTSS2USIQ` |
|
||
|
|
| `VCVTTSD2USI` | `VCVTTSD2USIL`, `VCVTTSD2USIQ` |
|
||
|
|
| `VCVTTSS2USI` | `VCVTTSS2USIL`, `VCVTTSS2USIQ` |
|
||
|
|
| `VCVTUSI2SD` | `VCVTUSI2SDL`, `VCVTUSI2SDQ` |
|
||
|
|
| `VCVTUSI2SS` | `VCVTUSI2SSL`, `VCVTUSI2SSQ` |
|
||
|
|
| `VCVTSI2SD` | `VCVTSI2SDL`, `VCVTSI2SDQ` |
|
||
|
|
| `VCVTSI2SS` | `VCVTSI2SSL`, `VCVTSI2SSQ` |
|
||
|
|
| `ANDN` | `ANDNL`, `ANDNQ` |
|
||
|
|
| `BEXTR` | `BEXTRL`, `BEXTRQ` |
|
||
|
|
| `BLSI` | `BLSIL`, `BLSIQ` |
|
||
|
|
| `BLSMSK` | `BLSMSKL`, `BLSMSKQ` |
|
||
|
|
| `BLSR` | `BLSRL`, `BLSRQ` |
|
||
|
|
| `BZHI` | `BZHIL`, `BZHIQ` |
|
||
|
|
| `MULX` | `MULXL`, `MULXQ` |
|
||
|
|
| `PDEP` | `PDEPL`, `PDEPQ` |
|
||
|
|
| `PEXT` | `PEXTL`, `PEXTQ` |
|
||
|
|
| `RORX` | `RORXL`, `RORXQ` |
|
||
|
|
| `SARX` | `SARXL`, `SARXQ` |
|
||
|
|
| `SHLX` | `SHLXL`, `SHLXQ` |
|
||
|
|
| `SHRX` | `SHRXL`, `SHRXQ` |
|
||
|
|
|
||
|
|
### Encoder details
|
||
|
|
|
||
|
|
Bitwise comparison with older encoder may fail for VEX-encoded instructions due to slightly different encoder tables order.
|
||
|
|
|
||
|
|
This difference may arise for instructions with both `{reg, reg/mem}` and `{reg/mem, reg}` forms for reg-reg case. One of such instructions is `VMOVUPS`.
|
||
|
|
|
||
|
|
This does not affect code behavior, nor makes it bigger/less efficient.
|
||
|
|
New encoding selection scheme is borrowed from [Intel XED](https://github.com/intelxed/xed).
|
||
|
|
|
||
|
|
EVEX encoding is used when any of the following is true:
|
||
|
|
|
||
|
|
* Instruction uses new registers (High 16 `X`/`Y`, `Z` or `K` registers)
|
||
|
|
* Instruction uses EVEX-related opcode suffixes like `BCST`
|
||
|
|
* Instruction uses operands combination that is only available for AVX-512
|
||
|
|
|
||
|
|
In all other cases VEX encoding is used.
|
||
|
|
This means that VEX is used whenever possible, and EVEX whenever required.
|
||
|
|
|
||
|
|
Compressed disp8 is applied whenever possible for EVEX-encoded instructions.
|
||
|
|
This also covers broadcasting disp8 which sometimes has different N multiplier.
|
||
|
|
|
||
|
|
Experienced readers can inspect [avx_optabs.go](https://github.com/golang/go/blob/master/src/cmd/internal/obj/x86/avx_optabs.go) to learn about N multipliers for any instruction.
|
||
|
|
|
||
|
|
For example, `VADDPD` has these:
|
||
|
|
* `N=64` for 512-bit form; `N=8` when broadcasting
|
||
|
|
* `N=32` for 256-bit form; `N=8` when broadcasting
|
||
|
|
* `N=16` for 128-bit form; `N=8` when broadcasting
|
||
|
|
|
||
|
|
### Examples
|
||
|
|
|
||
|
|
Exhaustive amount of examples can be found in Go assembler [test suite](https://github.com/golang/go/tree/master/src/cmd/asm/internal/asm/testdata/avx512enc).
|
||
|
|
|
||
|
|
Each file provides several examples for every supported instruction form in particular AVX-512 extension.
|
||
|
|
Every example also includes generated machine code.
|
||
|
|
|
||
|
|
Here is adopted "Vectorized Histogram Update Using AVX-512CD" from
|
||
|
|
[Intel® Optimization Manual](https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf):
|
||
|
|
|
||
|
|
```go
|
||
|
|
for i := 0; i < 512; i++ {
|
||
|
|
histo[key[i]] += 1
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
```asm
|
||
|
|
top:
|
||
|
|
VMOVUPS 0x40(SP)(DX*4), Z4 //; vmovups zmm4, [rsp+rdx*4+0x40]
|
||
|
|
VPXORD Z1, Z1, Z1 //; vpxord zmm1, zmm1, zmm1
|
||
|
|
KMOVW K1, K2 //; kmovw k2, k1
|
||
|
|
VPCONFLICTD Z4, Z2 //; vpconflictd zmm2, zmm4
|
||
|
|
VPGATHERDD (AX)(Z4*4), K2, Z1 //; vpgatherdd zmm1{k2}, [rax+zmm4*4]
|
||
|
|
VPTESTMD histo<>(SB), Z2, K0 //; vptestmd k0, zmm2, [rip+0x185c]
|
||
|
|
KMOVW K0, CX //; kmovw ecx, k0
|
||
|
|
VPADDD Z0, Z1, Z3 //; vpaddd zmm3, zmm1, zmm0
|
||
|
|
TESTL CX, CX //; test ecx, ecx
|
||
|
|
JZ noConflicts //; jz noConflicts
|
||
|
|
VMOVUPS histo<>(SB), Z1 //; vmovups zmm1, [rip+0x1884]
|
||
|
|
VPTESTMD histo<>(SB), Z2, K0 //; vptestmd k0, zmm2, [rip+0x18ba]
|
||
|
|
VPLZCNTD Z2, Z5 //; vplzcntd zmm5, zmm2
|
||
|
|
XORB BX, BX //; xor bl, bl
|
||
|
|
KMOVW K0, CX //; kmovw ecx, k0
|
||
|
|
VPSUBD Z5, Z1, Z1 //; vpsubd zmm1, zmm1, zmm5
|
||
|
|
VPSUBD Z5, Z1, Z1 //; vpsubd zmm1, zmm1, zmm5
|
||
|
|
|
||
|
|
resolveConflicts:
|
||
|
|
VPBROADCASTD CX, Z5 //; vpbroadcastd zmm5, ecx
|
||
|
|
KMOVW CX, K2 //; kmovw k2, ecx
|
||
|
|
VPERMD Z3, Z1, K2, Z3 //; vpermd zmm3{k2}, zmm1, zmm3
|
||
|
|
VPADDD Z0, Z3, K2, Z3 //; vpaddd zmm3{k2}, zmm3, zmm0
|
||
|
|
VPTESTMD Z2, Z5, K2, K0 //; vptestmd k0{k2}, zmm5, zmm2
|
||
|
|
KMOVW K0, SI //; kmovw esi, k0
|
||
|
|
ANDL SI, CX //; and ecx, esi
|
||
|
|
JZ noConflicts //; jz noConflicts
|
||
|
|
ADDB $1, BX //; add bl, 0x1
|
||
|
|
CMPB BX, $16 //; cmp bl, 0x10
|
||
|
|
JB resolveConflicts //; jb resolveConflicts
|
||
|
|
|
||
|
|
noConflicts:
|
||
|
|
KMOVW K1, K2 //; kmovw k2, k1
|
||
|
|
VPSCATTERDD Z3, K2, (AX)(Z4*4) //; vpscatterdd [rax+zmm4*4]{k2}, zmm3
|
||
|
|
ADDL $16, DX //; add edx, 0x10
|
||
|
|
CMPL DX, $1024 //; cmp edx, 0x400
|
||
|
|
JB top //; jb top
|
||
|
|
```
|