maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

[spirv] Add script to auto-generate SPIR-V op template from spec 

SPIR-V has a JSON grammar file that defines the syntax of SPIR-V
instructions. However, its lacks fine-grained constraints on
instruction operands; those information is only available as
natural language sentences in the SPIR-V spec, which also contains
the detailed documentation for each SPIR-V instruction.

This CL pulls information from both the JSON grammar and HTML
spec. It right now uses the former to deduce the arguments and
results (with coarse-grained constraints) and the latter for
documentation. In the future we can add the functionality to
match certain natural language sentences for more fine-grained
constraints, but right now the developer is expected to update
the generated op definition. This should serve as a nice
bootstrap step to save efforts.

PiperOrigin-RevId: 257821205
This commit is contained in:
Lei Zhang 2019-07-12 10:00:13 -07:00 committed by Mehdi Amini
parent 60a2983779
commit bd484f17a0
3 changed files with 543 additions and 122 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

228
mlir/include/mlir/SPIRV/SPIRVOps.td
View File

@ -20,6 +20,12 @@
//
//===----------------------------------------------------------------------===//
// Note that for each op in this file, we use a tool to automatically generate
// certain sections in its definition: basic structure, summary, description.
// So modifications to these sections will not be respected. Modifications to
// op traits, arguments, results, and sections after the results are retained.
// Besides, ops in this file must be separated via the '// -----' marker.
#ifdef SPIRV_OPS
#else
#define SPIRV_OPS
@ -35,6 +41,52 @@ include "mlir/SPIRV/SPIRVBase.td"
include "mlir/SPIRV/SPIRVStructureOps.td"
#endif // SPIRV_STRUCTURE_OPS
// -----
def SPV_CompositeExtractOp : SPV_Op<"CompositeExtract", [NoSideEffect]> {
let summary = "Extract a part of a composite object.";
let description = [{
Result Type must be the type of object selected by the last provided
index. The instruction result is the extracted object.
Composite is the composite to extract from.
Indexes walk the type hierarchy, potentially down to component
granularity, to select the part to extract. All indexes must be in
bounds. All composite constituents use zero-based numbering, as
described by their OpType… instruction.
### Custom assembly form
``` {.ebnf}
composite-extract-op ::= ssa-id `=` `spv.CompositeExtract` ssa-use
`[` integer-literal (',' integer-literal)* `]`
`:` composite-type
```
For example:
```
%0 = spv.Variable : !spv.ptr<!spv.array<4x!spv.array<4xf32>>, Function>
%1 = spv.Load "Function" %0 ["Volatile"] : !spv.array<4x!spv.array<4xf32>>
%2 = spv.CompositeExtract %1[1 : i32] : !spv.array<4x!spv.array<4xf32>>
```
}];
let arguments = (ins
SPV_Composite:$composite,
I32ArrayAttr:$indices
);
let results = (outs
SPV_Type:$component
);
}
// -----
def SPV_EntryPointOp : SPV_Op<"EntryPoint", [ModuleOnly]> {
let summary = [{
Declare an entry point, its execution model, and its interface.
@ -46,20 +98,23 @@ def SPV_EntryPointOp : SPV_Op<"EntryPoint", [ModuleOnly]> {
Entry Point must be the Result <id> of an OpFunction instruction.
Name is a name string for the entry point. A module cannot have
two OpEntryPoint instructions with the same Execution Model and
the same Name string.
Name is a name string for the entry point. A module cannot have two
OpEntryPoint instructions with the same Execution Model and the same
Name string.
Interface is a list of <id> of global OpVariable
instructions. These declare the set of global variables from a
module that form the interface of this entry point. The set of
Interface <id> must be equal to or a superset of the global
OpVariable Result <id> referenced by the entry point’s static call
tree, within the interface’s storage classes. Before version 1.4,
the interface’s storage classes are limited to the Input and
Output storage classes. Starting with version 1.4, the interface’s
storage classes are all storage classes used in declaring all
global variables referenced by the entry point’s call tree.
Interface is a list of <id> of global OpVariable instructions. These
declare the set of global variables from a module that form the
interface of this entry point. The set of Interface <id> must be equal
to or a superset of the global OpVariable Result <id> referenced by the
entry point’s static call tree, within the interface’s storage classes.
Before version 1.4, the interface’s storage classes are limited to the
Input and Output storage classes. Starting with version 1.4, the
interface’s storage classes are all storage classes used in declaring
all global variables referenced by the entry point’s call tree.
Interface <id> are forward references. Before version 1.4, duplication
of these <id> is tolerated. Starting with version 1.4, an <id> must not
appear more than once.
### Custom assembly form
@ -90,18 +145,20 @@ def SPV_EntryPointOp : SPV_Op<"EntryPoint", [ModuleOnly]> {
let results = (outs SPV_EntryPoint:$id);
}
// -----
def SPV_ExecutionModeOp : SPV_Op<"ExecutionMode", [ModuleOnly]> {
let summary = "Declare an execution mode for an entry point.";
let description = [{
Entry Point must be the Entry Point <id> operand of an
OpEntryPoint instruction.
Entry Point must be the Entry Point <id> operand of an OpEntryPoint
instruction.
Mode is the execution mode. See Execution Mode.
This instruction is only valid when the Mode operand is an
execution mode that takes no Extra Operands, or takes Extra
Operands that are not <id> operands.
This instruction is only valid when the Mode operand is an execution
mode that takes no Extra Operands, or takes Extra Operands that are not
<id> operands.
### Custom assembly form
@ -127,18 +184,39 @@ def SPV_ExecutionModeOp : SPV_Op<"ExecutionMode", [ModuleOnly]> {
OptionalAttr<I32ArrayAttr>:$values
);
let results = (outs);
let verifier = [{ return success(); }];
}
// -----
def SPV_FMulOp : SPV_Op<"FMul", [NoSideEffect, SameOperandsAndResultType]> {
let summary = "Floating-point multiplication of Operand 1 and Operand 2";
let summary = "Floating-point multiplication of Operand 1 and Operand 2.";
let description = [{
Result Type must be a scalar or vector of floating-point type.
The types of Operand 1 and Operand 2 both must be the same as Result Type.
The types of Operand 1 and Operand 2 both must be the same as Result
Type.
Results are computed per component.
Results are computed per component.
### Custom assembly form
``` {.ebnf}
float-scalar-vector-type ::= float-type |
`vector<` integer-literal `x` float-type `>`
execution-mode-op ::= `spv.FMul` ssa-use, ssa-use
`:` float-scalar-vector-type
```
For example:
```
spv.FMul %0, %1 : f32
spv.FMul %2, %3 : vector<4xf32>
```
}];
let arguments = (ins
@ -157,20 +235,22 @@ def SPV_FMulOp : SPV_Op<"FMul", [NoSideEffect, SameOperandsAndResultType]> {
let verifier = [{ return success(); }];
}
def SPV_LoadOp : SPV_Op<"Load"> {
let summary = "Load value through a pointer.";
// -----
def SPV_LoadOp : SPV_Op<"Load", []> {
let summary = "Load through a pointer.";
let description = [{
Result Type is the type of the loaded object. It must be a type
with fixed size; i.e., it cannot be, nor include, any
OpTypeRuntimeArray types.
Result Type is the type of the loaded object. It must be a type with
fixed size; i.e., it cannot be, nor include, any OpTypeRuntimeArray
types.
Pointer is the pointer to load through. Its type must be an
Pointer is the pointer to load through. Its type must be an
OpTypePointer whose Type operand is the same as Result Type.
If present, any Memory Operands must begin with a memory operand
literal. If not present, it is the same as specifying the memory
operand None.
literal. If not present, it is the same as specifying the memory operand
None.
### Custom assembly form
@ -203,11 +283,19 @@ def SPV_LoadOp : SPV_Op<"Load"> {
);
}
// -----
def SPV_ReturnOp : SPV_Op<"Return", [Terminator]> {
let summary = "Return with no value from a function with void return type";
let summary = "Return with no value from a function with void return type.";
let description = [{
This instruction must be the last instruction in a block.
### Custom assembly form
``` {.ebnf}
return-op ::= `spv.Return`
```
}];
let arguments = (ins);
@ -220,19 +308,20 @@ def SPV_ReturnOp : SPV_Op<"Return", [Terminator]> {
let verifier = [{ return verifyReturn(*this); }];
}
def SPV_StoreOp : SPV_Op<"Store"> {
// -----
def SPV_StoreOp : SPV_Op<"Store", []> {
let summary = "Store through a pointer.";
let description = [{
Pointer is the pointer to store through. Its type must be an
OpTypePointer whose Type operand is the same as the type of
Object.
Pointer is the pointer to store through. Its type must be an
OpTypePointer whose Type operand is the same as the type of Object.
Object is the object to store.
If present, any Memory Operands must begin with a memory operand
literal. If not present, it is the same as specifying the memory
operand None.
literal. If not present, it is the same as specifying the memory operand
None.
### Custom assembly form
@ -257,26 +346,31 @@ def SPV_StoreOp : SPV_Op<"Store"> {
OptionalAttr<SPV_MemoryAccessAttr>:$memory_access,
OptionalAttr<APIntAttr>:$alignment
);
let results = (outs);
}
def SPV_VariableOp : SPV_Op<"Variable"> {
// -----
def SPV_VariableOp : SPV_Op<"Variable", []> {
let summary = [{
Allocate an object in memory, resulting in a pointer to it, which can be
used with OpLoad and OpStore
used with OpLoad and OpStore.
}];
let description = [{
Result Type must be an OpTypePointer. Its Type operand is the type of object
in memory.
Result Type must be an OpTypePointer. Its Type operand is the type of
object in memory.
Storage Class is the Storage Class of the memory holding the object. It
cannot be Generic. It must be the same as the Storage Class operand of the
Result Type.
cannot be Generic. It must be the same as the Storage Class operand of
the Result Type.
Initializer is optional. If Initializer is present, it will be the initial
value of the variable’s memory content. Initializer must be an <id> from a
constant instruction or a global (module scope) OpVariable instruction.
Initializer must have the same type as the type pointed to by Result Type.
Initializer is optional. If Initializer is present, it will be the
initial value of the variable’s memory content. Initializer must be an
<id> from a constant instruction or a global (module scope) OpVariable
instruction. Initializer must have the same type as the type pointed to
by Result Type.
### Custom assembly form
@ -310,46 +404,6 @@ def SPV_VariableOp : SPV_Op<"Variable"> {
);
}
def SPV_CompositeExtractOp : SPV_Op<"CompositeExtract", [NoSideEffect]> {
let summary = "Extract a part of a composite object.";
let description = [{
Result Type must be the type of object selected by the last provided index.
The instruction result is the extracted object.
Composite is the composite to extract from.
Indexes walk the type hierarchy, potentially down to component granularity,
to select the part to extract. All indexes must be in bounds.
All composite constituents use zero-based numbering, as described by their
OpType… instruction.
### Custom assembly form
``` {.ebnf}
composite-extract-op ::= ssa-id `=` `spv.CompositeExtract` ssa-use
`[` integer-literal (',' integer-literal)* `]`
`:` composite-type
```
For example:
```
%0 = spv.Variable : !spv.ptr<!spv.array<4x!spv.array<4xf32>>, Function>
%1 = spv.Load "Function" %0 ["Volatile"] : !spv.array<4x!spv.array<4xf32>>
%2 = spv.CompositeExtract %1[1 : i32] : !spv.array<4x!spv.array<4xf32>>
```
}];
let arguments = (ins
SPV_Composite:$composite,
I32ArrayAttr:$indices
);
let results = (outs
SPV_Type:$component
);
}
// -----
#endif // SPIRV_OPS

36
mlir/utils/spirv/define_inst.sh Executable file
View File

@ -0,0 +1,36 @@
#!/bin/bash
# Copyright 2019 The MLIR Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Script for defining a new op using SPIR-V spec from the Internet.
#
# Run as:
# ./define_inst.sh <opname>
# For example:
# ./define_inst.sh OpIAdd
#
# If <opname> is missing, this script updates existing ones.
set -e
new_op=$1
current_file="$(readlink -f "$0")"
current_dir="$(dirname "$current_file")"
python3 ${current_dir}/gen_spirv_dialect.py \
--op-td-path ${current_dir}/../../include/mlir/SPIRV/SPIRVOps.td \
--new-inst "${new_op}"

401
mlir/utils/spirv/gen_spirv_dialect.py
View File

@ -20,29 +20,58 @@
#
# For example, to define the enum attribute for SPIR-V memory model:
#
# ./gen_spirv_dialect.py --bash_td_path /path/to/SPIRVBase.td \
# ./gen_spirv_dialect.py --base_td_path /path/to/SPIRVBase.td \
# --new-enum MemoryModel
#
# The 'operand_kinds' dict of spirv.core.grammar.json contains all supported
# SPIR-V enum classes.
import re
import requests
import textwrap
SPIRV_HTML_SPEC_URL = 'https://www.khronos.org/registry/spir-v/specs/unified1/SPIRV.html'
SPIRV_JSON_SPEC_URL = 'https://raw.githubusercontent.com/KhronosGroup/SPIRV-Headers/master/include/spirv/unified1/spirv.core.grammar.json'
AUTOGEN_OP_DEF_SEPARATOR = '\n// -----\n\n'
AUTOGEN_ENUM_SECTION_MARKER = 'enum section. Generated from SPIR-V spec; DO NOT MODIFY!'
AUTOGEN_INSTRUCTION_OPCODE_SECTION_MARKER = ('opcode section. Generated from '
'SPIR-V spec; DO NOT MODIFY!')
AUTOGEN_OPCODE_SECTION_MARKER = (
'opcode section. Generated from SPIR-V spec; DO NOT MODIFY!')
def get_spirv_doc_from_html_spec():
"""Extracts instruction documentation from SPIR-V HTML spec.
Returns:
- A dict mapping from instruction opcode to documentation.
"""
response = requests.get(SPIRV_HTML_SPEC_URL)
spec = response.content
from bs4 import BeautifulSoup
spirv = BeautifulSoup(spec, 'html.parser')
section_anchor = spirv.find('h3', {'id': '_a_id_instructions_a_instructions'})
doc = {}
for section in section_anchor.parent.find_all('div', {'class': 'sect3'}):
for table in section.find_all('table'):
inst_html = table.tbody.tr.td.p
opname = inst_html.a['id']
# Ignore the first line, which is just the opname.
doc[opname] = inst_html.text.split('\n', 1)[1].strip()
return doc
def get_spirv_grammar_from_json_spec():
"""Extracts operand kind and instruction grammar from SPIR-V JSON spec.
Returns:
- A list containing all operand kinds' grammar
- A list containing all instructions' grammar
"""
Returns:
- A list containing all operand kinds' grammar
- A list containing all instructions' grammar
"""
response = requests.get(SPIRV_JSON_SPEC_URL)
spec = response.content
@ -55,13 +84,13 @@ def get_spirv_grammar_from_json_spec():
def split_list_into_sublists(items, offset):
"""Split the list of items into multiple sublists.
This is to make sure the string composed from each sublist won't exceed
80 characters.
This is to make sure the string composed from each sublist won't exceed
80 characters.
Arguments:
- items: a list of strings
- offset: the offset in calculating each sublist's length
"""
Arguments:
- items: a list of strings
- offset: the offset in calculating each sublist's length
"""
chuncks = []
chunk = []
chunk_len = 0
@ -83,10 +112,10 @@ def split_list_into_sublists(items, offset):
def gen_operand_kind_enum_attr(operand_kind):
"""Generates the TableGen I32EnumAttr definition for the given operand kind.
Returns:
- The operand kind's name
- A string containing the TableGen I32EnumAttr definition
"""
Returns:
- The operand kind's name
- A string containing the TableGen I32EnumAttr definition
"""
if 'enumerants' not in operand_kind:
return '', ''
@ -134,9 +163,9 @@ def gen_operand_kind_enum_attr(operand_kind):
def gen_opcode(instructions):
""" Generates the TableGen definition to map opname to opcode
Returns:
- A string containing the TableGen SPV_OpCode definition
"""
Returns:
- A string containing the TableGen SPV_OpCode definition
"""
max_len = max([len(inst['opname']) for inst in instructions])
def_fmt_str = 'def SPV_OC_{name} {colon:>{offset}} '\
@ -172,15 +201,21 @@ def gen_opcode(instructions):
def update_td_opcodes(path, instructions, filter_list):
"""Updates SPIRBase.td with new generated opcode cases.
Arguments:
- path: the path to SPIRBase.td
- instructions: a list containing all SPIR-V instructions' grammar
- filter_list: a list containing new opnames to add
"""
with open(path, 'r') as f:
content = f.read()
content = content.split(AUTOGEN_INSTRUCTION_OPCODE_SECTION_MARKER)
content = content.split(AUTOGEN_OPCODE_SECTION_MARKER)
assert len(content) == 3
# Extend opcode list with existing list
import re
existing_opcodes = [k[11:] for k in re.findall('def SPV_OC_\w+', content[1])]
filter_list.extend(existing_opcodes)
filter_list = list(set(filter_list))
@ -193,8 +228,8 @@ def update_td_opcodes(path, instructions, filter_list):
opcode = gen_opcode(filter_instrs)
# Substitute the opcode
content = content[0] + AUTOGEN_INSTRUCTION_OPCODE_SECTION_MARKER + '\n\n' + \
opcode + '\n\n// End ' + AUTOGEN_INSTRUCTION_OPCODE_SECTION_MARKER \
content = content[0] + AUTOGEN_OPCODE_SECTION_MARKER + '\n\n' + \
opcode + '\n\n// End ' + AUTOGEN_OPCODE_SECTION_MARKER \
+ content[2]
with open(path, 'w') as f:
@ -204,10 +239,10 @@ def update_td_opcodes(path, instructions, filter_list):
def update_td_enum_attrs(path, operand_kinds, filter_list):
"""Updates SPIRBase.td with new generated enum definitions.
Arguments:
- path: the path to SPIRBase.td
- operand_kinds: a list containing all operand kinds' grammar
- filter_list: a list containing new enums to add
Arguments:
- path: the path to SPIRBase.td
- operand_kinds: a list containing all operand kinds' grammar
- filter_list: a list containing new enums to add
"""
with open(path, 'r') as f:
content = f.read()
@ -216,7 +251,6 @@ def update_td_enum_attrs(path, operand_kinds, filter_list):
assert len(content) == 3
# Extend filter list with existing enum definitions
import re
existing_kinds = [
k[8:-4] for k in re.findall('def SPV_\w+Attr', content[1])]
filter_list.extend(existing_kinds)
@ -238,25 +272,322 @@ def update_td_enum_attrs(path, operand_kinds, filter_list):
f.write(content)
def snake_casify(name):
"""Turns the given name to follow snake_case convension."""
name = re.sub('\W+', '', name).split()
name = [s.lower() for s in name]
return '_'.join(name)
def map_spec_operand_to_ods_argument(operand):
"""Maps a operand in SPIR-V JSON spec to an op argument in ODS.
Arguments:
- A dict containing the operand's kind, quantifier, and name
Returns:
- A string containing both the type and name for the argument
"""
kind = operand['kind']
quantifier = operand.get('quantifier', '')
# These instruction "operands" are for encoding the results; they should
# not be handled here.
assert kind != 'IdResultType', 'unexpected to handle "IdResultType" kind'
assert kind != 'IdResult', 'unexpected to handle "IdResult" kind'
if kind == 'IdRef':
if quantifier == '':
arg_type = 'SPV_Type'
elif quantifier == '?':
arg_type = 'SPV_Optional<SPV_Type>'
else:
arg_type = 'Variadic<SPV_Type>'
elif kind == 'IdMemorySemantics' or kind == 'IdScope':
# TODO(antiagainst): Need to further constrain 'IdMemorySemantics'
# and 'IdScope' given that they should be gernated from OpConstant.
assert quantifier == '', ('unexpected to have optional/variadic memory '
'semantics or scope <id>')
arg_type = 'I32'
elif kind == 'LiteralInteger':
if quantifier == '':
arg_type = 'I32Attr'
elif quantifier == '?':
arg_type = 'OptionalAttr<I32Attr>'
else:
arg_type = 'OptionalAttr<I32ArrayAttr>'
elif kind == 'LiteralString' or \
kind == 'LiteralContextDependentNumber' or \
kind == 'LiteralExtInstInteger' or \
kind == 'LiteralSpecConstantOpInteger' or \
kind == 'PairLiteralIntegerIdRef' or \
kind == 'PairIdRefLiteralInteger' or \
kind == 'PairIdRefIdRef':
assert False, '"{}" kind unimplemented'.format(kind)
else:
# The rest are all enum operands that we represent with op attributes.
assert quantifier != '*', 'unexpected to have variadic enum attribute'
arg_type = 'SPV_{}Attr'.format(kind)
if quantifier == '?':
arg_type = 'OptionalAttr<{}>'.format(arg_type)
name = operand.get('name', '')
name = snake_casify(name) if name else kind.lower()
return '{}:${}'.format(arg_type, name)
def get_op_definition(instruction, doc, existing_info):
"""Generates the TableGen op definition for the given SPIR-V instruction.
Arguments:
- instruction: the instruction's SPIR-V JSON grammar
- doc: the instruction's SPIR-V HTML doc
- existing_info: a dict containing potential manually specified sections for
this instruction
Returns:
- A string containing the TableGen op definition
"""
fmt_str = 'def SPV_{opname}Op : SPV_Op<"{opname}", [{traits}]> {{\n'\
' let summary = {summary};\n\n'\
' let description = [{{\n'\
'{description}\n\n'\
' ### Custom assembly form\n'\
'{assembly}'\
'}}];\n\n'\
' let arguments = (ins{args});\n\n'\
' let results = (outs{results});\n'\
'{extras}'\
'}}\n'
opname = instruction['opname'][2:]
summary, description = doc.split('\n', 1)
wrapper = textwrap.TextWrapper(
width=76, initial_indent=' ', subsequent_indent=' ')
# Format summary. If the summary can fit in the same line, we print it out
# as a "-quoted string; otherwise, wrap the lines using "[{...}]".
summary = summary.strip();
if len(summary) + len(' let summary = "";') <= 80:
summary = '"{}"'.format(summary)
else:
summary = '[{{\n{}\n }}]'.format(wrapper.fill(summary))
# Wrap description
description = description.split('\n')
description = [wrapper.fill(line) for line in description if line]
description = '\n\n'.join(description)
operands = instruction.get('operands', [])
# Set op's result
results = ''
if len(operands) > 0 and operands[0]['kind'] == 'IdResultType':
results = '\n SPV_Type:$result\n '
operands = operands[1:]
if 'results' in existing_info:
results = existing_info['results']
# Ignore the operand standing for the result <id>
if len(operands) > 0 and operands[0]['kind'] == 'IdResult':
operands = operands[1:]
# Set op' argument
arguments = existing_info.get('arguments', None)
if arguments is None:
arguments = [map_spec_operand_to_ods_argument(o) for o in operands]
arguments = '\n '.join(arguments)
if arguments:
# Prepend and append whitespace for formatting
arguments = '\n {}\n '.format(arguments)
assembly = existing_info.get('assembly', None)
if assembly is None:
assembly = ' ``` {.ebnf}\n'\
' [TODO]\n'\
' ```\n\n'\
' For example:\n\n'\
' ```\n'\
' [TODO]\n'\
' ```\n '
return fmt_str.format(
opname=opname,
traits=existing_info.get('traits', ''),
summary=summary,
description=description,
assembly=assembly,
args=arguments,
results=results,
extras=existing_info.get('extras', ''))
def extract_td_op_info(op_def):
"""Extracts potentially manually specified sections in op's definition.
Arguments: - A string containing the op's TableGen definition
- doc: the instruction's SPIR-V HTML doc
Returns:
- A dict containing potential manually specified sections
"""
# Get opname
opname = [o[8:-2] for o in re.findall('def SPV_\w+Op', op_def)]
assert len(opname) == 1, 'more than one ops in the same section!'
opname = opname[0]
# Get traits
op_tmpl_params = op_def.split('<', 1)[1].split('>', 1)[0].split(', ', 1)
if len(op_tmpl_params) == 1:
traits = ''
else:
traits = op_tmpl_params[1].strip('[]')
# Get custom assembly form
rest = op_def.split('### Custom assembly form\n')
assert len(rest) == 2, \
'{}: cannot find "### Custom assembly form"'.format(opname)
rest = rest[1].split(' let arguments = (ins')
assert len(rest) == 2, '{}: cannot find arguments'.format(opname)
assembly = rest[0].rstrip('}];\n')
# Get arguments
rest = rest[1].split(' let results = (outs')
assert len(rest) == 2, '{}: cannot find results'.format(opname)
args = rest[0].rstrip(');\n')
# Get results
rest = rest[1].split(');', 1)
assert len(rest) == 2, \
'{}: cannot find ");" ending results'.format(opname)
results = rest[0]
extras = rest[1].strip(' }\n')
if extras:
extras = '\n {}\n'.format(extras)
return {
# Prefix with 'Op' to make it consistent with SPIR-V spec
'opname': 'Op{}'.format(opname),
'traits': traits,
'assembly': assembly,
'arguments': args,
'results': results,
'extras': extras
}
def update_td_op_definitions(path, instructions, docs, filter_list):
"""Updates SPIRVOps.td with newly generated op definition.
Arguments:
- path: path to SPIRVOps.td
- instructions: SPIR-V JSON grammar for all instructions
- docs: SPIR-V HTML doc for all instructions
- filter_list: a list containing new opnames to include
Returns:
- A string containing all the TableGen op definitions
"""
with open(path, 'r') as f:
content = f.read()
# Split the file into chuncks, each containing one op.
ops = content.split(AUTOGEN_OP_DEF_SEPARATOR)
header = ops[0]
footer = ops[-1]
ops = ops[1:-1]
# For each existing op, extract the manually-written sections out to retain
# them when re-generating the ops. Also append the existing ops to filter
# list.
op_info_dict = {}
for op in ops:
info_dict = extract_td_op_info(op)
opname = info_dict['opname']
op_info_dict[opname] = info_dict
filter_list.append(opname)
filter_list = sorted(list(set(filter_list)))
op_defs = []
for opname in filter_list:
# Find the grammar spec for this op
instruction = next(
inst for inst in instructions if inst['opname'] == opname)
op_defs.append(
get_op_definition(instruction, docs[opname],
op_info_dict.get(opname, {})))
# Substitute the old op definitions
op_defs = [header] + op_defs + [footer]
content = AUTOGEN_OP_DEF_SEPARATOR.join(op_defs)
with open(path, 'w') as f:
f.write(content)
if __name__ == '__main__':
import argparse
cli_parser = argparse.ArgumentParser(
description='Update SPIR-V dialect definitions using SPIR-V spec')
cli_parser.add_argument('--base-td-path', dest='base_td_path', type=str,
help='Path to SPIRVBase.td')
cli_parser.add_argument('--new-enum', dest='new_enum', type=str,
help='SPIR-V enum to be added to SPIRVBase.td')
cli_parser.add_argument(
'--base-td-path',
dest='base_td_path',
type=str,
default=None,
help='Path to SPIRVBase.td')
cli_parser.add_argument(
'--op-td-path',
dest='op_td_path',
type=str,
default=None,
help='Path to SPIRVOps.td')
cli_parser.add_argument(
'--new-enum',
dest='new_enum',
type=str,
default=None,
help='SPIR-V enum to be added to SPIRVBase.td')
cli_parser.add_argument(
'--new-opcodes',
dest='new_opcodes',
type=str,
default=None,
nargs='*',
help='update SPIR-V opcodes in SPIRVBase.td')
cli_parser.add_argument(
'--new-inst',
dest='new_inst',
type=str,
default=None,
help='SPIR-V instruction to be added to SPIRVOps.td')
args = cli_parser.parse_args()
operand_kinds, instructions = get_spirv_grammar_from_json_spec()
update_td_enum_attrs(args.base_td_path, operand_kinds, [args.new_enum])
# Define new enum attr
if args.new_enum is not None:
assert args.base_td_path is not None
filter_list = [args.new_enum] if args.new_enum else []
update_td_enum_attrs(args.base_td_path, operand_kinds, filter_list)
update_td_opcodes(args.base_td_path, instructions, args.new_opcodes)
# Define new opcode
if args.new_opcodes is not None:
assert args.base_td_path is not None
update_td_opcodes(args.base_td_path, instructions, args.new_opcodes)
# Define new op
if args.new_inst is not None:
assert args.op_td_path is not None
filter_list = [args.new_inst] if args.new_inst else []
docs = get_spirv_doc_from_html_spec()
update_td_op_definitions(args.op_td_path, instructions, docs, filter_list)
print('Done. Note that this script just generates a template; ', end='')
print('please read the spec and update traits, arguments, and ', end='')
print('results accordingly.')