[ODS] Allow dialect to specify C++ namespaces

Previously we force the C++ namespaces to be `NS` if `SomeOp` is defined as
    `NS_SomeOp`. This is too rigid as it does not support nested namespaces
    well. This CL adds a "namespace" field into the Dialect class to allow
    flexible namespaces.

--

PiperOrigin-RevId: 249064981

mlir/g3doc/OpDefinitions.md

@@ -1,4 +1,4 @@
-# Table-driven Operation Definition Specification
+# Table-driven Operation Definition Specification (ODS)
 
 In addition to specializing the `mlir::Op` C++ template, MLIR also supports
 defining operations in a table-driven manner. This is achieved via
@@ -134,10 +134,14 @@ of the `Op` class for the complete list of fields supported.
 ### Operation name
 
 The operation name is a unique identifier of the operation within MLIR, e.g.,
-`Add` for addition operation. This is the equivalent of the mnemonic in assembly
-language. It is used for parsing and printing in the textual format. It is also
-used for pattern matching in graph rewrites. The operation name is provided as
-the first template parameter to the `Op` class.
+`tf.Add` for addition operation in the TensorFlow dialect. This is the
+equivalent of the mnemonic in assembly language. It is used for parsing and
+printing in the textual format. It is also used for pattern matching in graph
+rewrites.
+
+The full operation name is composed of the dialect name and the op name, with
+the former provided via the dialect and the latter provided as the second
+template parameter to the `Op` class.
 
 ### Operation documentation
 
@@ -388,6 +392,37 @@ Note that `extraClassDeclaration` is a mechanism intended for long-tail cases
 by power users; for not-yet-implemented widely-applicable cases, improving the
 infrastructure is preferable.
 
+### Generated C++ code
+
+[OpDefinitionsGen][OpDefinitionsGen] processes the op definition spec file and
+generates two files containing the corresponding C++ code: one for declarations,
+the other for definitions. The former is generated via the `-gen-op-decls`
+command-line option, while the latter is via the `-gen-op-defs` option.
+
+The definition file contains all the op method definitions, which can be
+included and enabled by defining `GET_OP_CLASSES`. Besides, it also
+contains a comma-separated list of all defined ops, which can be included
+and enabled by defining `GET_OP_LIST`.
+
+### Class name and namespaces
+
+For each operation, its generated C++ class name is the symbol `def`ed with
+TableGen with dialect prefix removed. The first `_` serves as the delimiter.
+For example, for `def TF_AddOp`, the C++ class name would be `AddOp`.
+We remove the `TF` prefix because it is for scoping ops; other dialects
+may as well define their own `AddOp`s.
+
+The namespaces of the generated C++ class will come from the dialect's
+`cppNamespace` field. For example, if a dialect's `cppNamespace` is `A::B`,
+then an op of that dialect will be placed in
+`namespace A { namespace B { ... } }`. If a dialect does not specify a
+`cppNamespace`, we then use the dialect's name as the namespace.
+
+This means the qualified name of the generated C++ class does not necessarily
+match exactly with the operation name as explained in
+[Operation name](#operation-name). This is to allow flexible naming to satisfy
+coding style requirements.
+
 ## Constraints
 
 Constraint is a core concept in table-driven operation definition: operation

mlir/include/mlir/IR/OpBase.td

@@ -214,6 +214,18 @@ class Dialect {
 
   // The description of the dialect.
   string description = ?;
+
+  // The C++ namespace that ops of this dialect should be placed into.
+  //
+  // By default, uses the name of the dialect as the only namespace. To avoid
+  // placing in any namespace, use "". To specify nested namespaces, use "::"
+  // as the delimiter, e.g., given "A::B", ops will be placed in
+  // `namespace A { namespace B { <ops> } }`.
+  //
+  // Note that this works in conjunction with dialect C++ code. Depending on how
+  // the generated files are included into the dialect, you may want to specify
+  // a full namespace path or a partial one.
+  string cppNamespace = name;
 }
 
 //===----------------------------------------------------------------------===//

mlir/include/mlir/LLVMIR/LLVMOpBase.td

@@ -31,6 +31,7 @@ include "mlir/IR/OpBase.td"
 
 def LLVM_Dialect : Dialect {
   let name = "llvm";
+  let cppNamespace = "LLVM";
 }
 
 // LLVM IR type wrapped in MLIR.

mlir/include/mlir/LLVMIR/NVVMOps.td

@@ -27,6 +27,7 @@ include "mlir/LLVMIR/LLVMOpBase.td"
 
 def NVVM_Dialect : Dialect {
   let name = "nvvm";
+  let cppNamespace = "NVVM";
 }
 
 class NVVM_Op<string mnemonic, list<OpTrait> traits = []> :

mlir/include/mlir/StandardOps/Ops.td

@@ -30,6 +30,7 @@ include "mlir/IR/OpBase.td"
 
 def Std_Dialect : Dialect {
   let name = "std";
+  let cppNamespace = "";
 }
 
 // Base class for Standard dialect ops.

mlir/include/mlir/TableGen/Dialect.h

@@ -0,0 +1,50 @@
+//
+// Copyright 2019 The MLIR Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// =============================================================================
+//
+// Dialect wrapper to simplify using TableGen Record defining a MLIR dialect.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef MLIR_TABLEGEN_DIALECT_H_
+#define MLIR_TABLEGEN_DIALECT_H_
+
+#include "mlir/Support/LLVM.h"
+
+namespace llvm {
+class Record;
+} // end namespace llvm
+
+namespace mlir {
+namespace tblgen {
+// Wrapper class that contains a MLIR dialect's information defined in TableGen
+// and provides helper methods for accessing them.
+class Dialect {
+public:
+  explicit Dialect(const llvm::Record *def) : def(*def) {}
+
+  // Returns the name of this dialect.
+  StringRef getName() const;
+
+  // Returns the C++ namespaces that ops of this dialect should be placed into.
+  StringRef getCppNamespace() const;
+
+private:
+  const llvm::Record &def;
+};
+} // end namespace tblgen
+} // end namespace mlir
+
+#endif // MLIR_TABLEGEN_DIALECT_H_

mlir/include/mlir/TableGen/Operator.h

@@ -25,6 +25,7 @@
 #include "mlir/Support/LLVM.h"
 #include "mlir/TableGen/Argument.h"
 #include "mlir/TableGen/Attribute.h"
+#include "mlir/TableGen/Dialect.h"
 #include "mlir/TableGen/OpTrait.h"
 #include "mlir/TableGen/Type.h"
 #include "llvm/ADT/PointerUnion.h"
@@ -50,25 +51,20 @@ public:
   explicit Operator(const llvm::Record &def);
   explicit Operator(const llvm::Record *def) : Operator(*def) {}
 
-  // Returns the operation name.
-  std::string getOperationName() const;
-
   // Returns this op's dialect name.
   StringRef getDialectName() const;
 
+  // Returns the operation name. The name will follow the "<dialect>.<op-name>"
+  // format if its dialect name is not empty.
+  std::string getOperationName() const;
+
+  // Returns this op's C++ namespaces.
+  StringRef getCppNamespaces() const;
+
   // Returns this op's C++ class name.
   StringRef getCppClassName() const;
 
-  // Returns this op's extra class declaration code.
-  StringRef getExtraClassDeclaration() const;
-
-  // Returns the qualified C++ class name for the given TableGen def `name`.
-  // The first `_` in `name` is treated as separating the dialect namespace
-  // and the op class name if the dialect namespace is not empty. Otherwise,
-  // if `name` starts with a `_`, the `_` is considered as part the class name.
-  static std::string getQualCppClassName(StringRef name);
-
-  // Returns this op's C++ class name prefixed with dialect namespace.
+  // Returns this op's C++ class name prefixed with namespaces.
   std::string getQualCppClassName() const;
 
   // Returns the number of results this op produces.
@@ -147,12 +143,15 @@ public:
   bool hasSummary() const;
   StringRef getSummary() const;
 
+  // Returns this op's extra class declaration code.
+  StringRef getExtraClassDeclaration() const;
+
 private:
   // Populates the vectors containing operands, attributes, results and traits.
   void populateOpStructure();
 
-  // The dialect name of the op.
-  StringRef dialectName;
+  // The dialect of this op.
+  Dialect dialect;
 
   // The unqualified C++ class name of the op.
   StringRef cppClassName;

mlir/lib/TableGen/CMakeLists.txt

@@ -2,6 +2,7 @@ add_llvm_library(LLVMMLIRTableGen
   Argument.cpp
   Attribute.cpp
   Constraint.cpp
+  Dialect.cpp
   Format.cpp
   Operator.cpp
   OpTrait.cpp

mlir/lib/TableGen/Dialect.cpp

@@ -0,0 +1,37 @@
+//===- Dialect.cpp - Dialect wrapper class --------------------------------===//
+//
+// Copyright 2019 The MLIR Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// =============================================================================
+//
+// Dialect wrapper to simplify using TableGen Record defining a MLIR dialect.
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/TableGen/Dialect.h"
+#include "llvm/TableGen/Record.h"
+
+namespace mlir {
+namespace tblgen {
+
+StringRef tblgen::Dialect::getName() const {
+  return def.getValueAsString("name");
+}
+
+StringRef tblgen::Dialect::getCppNamespace() const {
+  return def.getValueAsString("cppNamespace");
+}
+
+} // end namespace tblgen
+} // end namespace mlir

mlir/lib/TableGen/Operator.cpp

@@ -33,42 +33,46 @@ using llvm::DagInit;
 using llvm::DefInit;
 using llvm::Record;
 
-tblgen::Operator::Operator(const llvm::Record &def) : def(def) {
-  std::tie(dialectName, cppClassName) = def.getName().split('_');
-  if (dialectName.empty()) {
-    // Class name with a leading underscore and without dialect name
+tblgen::Operator::Operator(const llvm::Record &def)
+    : dialect(def.getValueAsDef("opDialect")), def(def) {
+  // The first `_` in the op's TableGen def name is treated as separating the
+  // dialect prefix and the op class name. The dialect prefix will be ignored if
+  // not empty. Otherwise, if def name starts with a `_`, the `_` is considered
+  // as part of the class name.
+  StringRef prefix;
+  std::tie(prefix, cppClassName) = def.getName().split('_');
+  if (prefix.empty()) {
+    // Class name with a leading underscore and without dialect prefix
     cppClassName = def.getName();
   } else if (cppClassName.empty()) {
-    // Class name without dialect name
-    std::swap(dialectName, cppClassName);
+    // Class name without dialect prefix
+    cppClassName = prefix;
   }
 
   populateOpStructure();
 }
 
 std::string tblgen::Operator::getOperationName() const {
-  auto *dialect = def.getValueAsDef("opDialect");
-  assert(dialect && "op defined without dialect");
-  auto prefix = dialect->getValueAsString("name");
+  auto prefix = dialect.getName();
+  auto opName = def.getValueAsString("opName");
   if (prefix.empty())
-    return def.getValueAsString("opName");
-  return llvm::formatv("{0}.{1}", prefix, def.getValueAsString("opName"));
+    return opName;
+  return llvm::formatv("{0}.{1}", prefix, opName);
 }
 
-StringRef tblgen::Operator::getDialectName() const { return dialectName; }
+StringRef tblgen::Operator::getDialectName() const { return dialect.getName(); }
+
+StringRef tblgen::Operator::getCppNamespaces() const {
+  return dialect.getCppNamespace();
+}
 
 StringRef tblgen::Operator::getCppClassName() const { return cppClassName; }
 
-std::string tblgen::Operator::getQualCppClassName(StringRef name) {
-  StringRef ns, cls;
-  std::tie(ns, cls) = name.split('_');
-  if (ns.empty() || cls.empty())
-    return name;
-  return (ns + "::" + cls).str();
-}
-
 std::string tblgen::Operator::getQualCppClassName() const {
-  return getQualCppClassName(def.getName());
+  auto prefix = dialect.getCppNamespace();
+  if (prefix.empty())
+    return cppClassName;
+  return llvm::formatv("{0}::{1}", prefix, cppClassName);
 }
 
 int tblgen::Operator::getNumResults() const {

mlir/test/mlir-tblgen/attr-enum.td

@@ -11,6 +11,7 @@ def NS_SomeEnum : EnumAttr<
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "NS";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;
@@ -29,7 +30,7 @@ def NS_OpA : NS_Op<"op_a_with_enum_attr", []> {
 
 // DEF-LABEL: OpA::verify()
 // DEF:         auto tblgen_attr = this->getAttr("attr");
-// DEF:         if (!(((tblgen_attr.isa<StringAttr>())) && (((tblgen_attr.cast<StringAttr>().getValue() == "A")) || ((tblgen_attr.cast<StringAttr>().getValue() == "B"))))) 
+// DEF:         if (!(((tblgen_attr.isa<StringAttr>())) && (((tblgen_attr.cast<StringAttr>().getValue() == "A")) || ((tblgen_attr.cast<StringAttr>().getValue() == "B")))))
 // DEF-SAME:      return emitOpError("attribute 'attr' failed to satisfy constraint: some enum");
 
 def NS_OpB : NS_Op<"op_b_with_enum_attr", []> {

mlir/test/mlir-tblgen/dialect.td

@@ -0,0 +1,53 @@
+// RUN: mlir-tblgen -gen-op-decls -I %S/../../include %s | FileCheck %s --check-prefix=DECL
+// RUN: mlir-tblgen -gen-op-defs -I %S/../../include %s | FileCheck %s --check-prefix=DEF
+
+include "mlir/IR/OpBase.td"
+
+// Check using the dialect name as the namespace
+def A_Dialect : Dialect {
+  let name = "a";
+}
+
+def A_SomeOp : Op<A_Dialect, "some_op", []>;
+
+// Check a single namespace
+def B_Dialect : Dialect {
+  let name = "b";
+  let cppNamespace = "BNS";
+}
+
+// Check nested namespaces
+def B_SomeOp : Op<B_Dialect, "some_op", []>;
+
+def C_Dialect : Dialect {
+  let name = "c";
+  let cppNamespace = "::C::CC";
+}
+
+def C_SomeOp : Op<C_Dialect, "some_op", []>;
+
+// Check no namespaces
+def D_Dialect : Dialect {
+  let name = "d";
+  let cppNamespace = "";
+}
+
+def D_DSomeOp : Op<D_Dialect, "some_op", []>;
+
+// DEF-LABEL: GET_OP_LIST
+// DEF:      a::SomeOp
+// DEF-NEXT: BNS::SomeOp
+// DEF-NEXT: ::C::CC::SomeOp
+// DEF-NEXT: DSomeOp
+
+// DEF-LABEL: GET_OP_CLASSES
+// DEF: a::SomeOp definitions
+// DEF: BNS::SomeOp definitions
+// DEF: ::C::CC::SomeOp definitions
+// DEF: DSomeOp definitions
+
+// DECL-LABEL: GET_OP_CLASSES
+// DECL: a::SomeOp declarations
+// DECL: BNS::SomeOp declarations
+// DECL: ::C::CC::SomeOp declarations
+// DECL: DSomeOp declarations

mlir/test/mlir-tblgen/directive-verifyUnusedValue.td

@@ -4,6 +4,7 @@ include "mlir/IR/OpBase.td"
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;

mlir/test/mlir-tblgen/op-decl.td

@@ -4,6 +4,7 @@ include "mlir/IR/OpBase.td"
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "NS";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;

mlir/test/mlir-tblgen/pattern-NativeCodeCall.td

@@ -4,6 +4,7 @@ include "mlir/IR/OpBase.td"
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "NS";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;

mlir/test/mlir-tblgen/pattern-bound-symbol.td

@@ -4,6 +4,7 @@ include "mlir/IR/OpBase.td"
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;

mlir/test/mlir-tblgen/pattern-multi-result-op.td

@@ -4,6 +4,7 @@ include "mlir/IR/OpBase.td"
 
 def Test_Dialect : Dialect {
   let name = "test";
+  let cppNamespace = "";
 }
 class NS_Op<string mnemonic, list<OpTrait> traits> :
     Op<Test_Dialect, mnemonic, traits>;

mlir/tools/mlir-tblgen/OpDefinitionsGen.cpp

@@ -417,9 +417,16 @@ void OpEmitter::emitDef(const Record &def, raw_ostream &os) {
   OpEmitter(def).emitDef(os);
 }
 
-void OpEmitter::emitDecl(raw_ostream &os) { opClass.writeDeclTo(os); }
+void OpEmitter::emitDecl(raw_ostream &os) {
+  os << formatv(opCommentHeader, op.getQualCppClassName(), "declarations");
+  opClass.writeDeclTo(os);
+}
 
-void OpEmitter::emitDef(raw_ostream &os) { opClass.writeDefTo(os); }
+void OpEmitter::emitDef(raw_ostream &os) {
+  os << formatv(opCommentHeader, op.getQualCppClassName(), "definitions");
+
+  opClass.writeDefTo(os);
+}
 
 void OpEmitter::genAttrGetters() {
   FmtContext fctx;
@@ -1028,14 +1035,8 @@ static void emitOpClasses(const std::vector<Record *> &defs, raw_ostream &os,
   IfDefScope scope("GET_OP_CLASSES", os);
   for (auto *def : defs) {
     if (emitDecl) {
-      os << formatv(opCommentHeader,
-                    Operator::getQualCppClassName(def->getName()),
-                    "declarations");
       OpEmitter::emitDecl(*def, os);
     } else {
-      os << formatv(opCommentHeader,
-                    Operator::getQualCppClassName(def->getName()),
-                    "definitions");
       OpEmitter::emitDef(*def, os);
     }
   }
@@ -1046,10 +1047,10 @@ static void emitOpList(const std::vector<Record *> &defs, raw_ostream &os) {
   IfDefScope scope("GET_OP_LIST", os);
 
   interleave(
-      defs,
-      [&os](Record *def) {
-        os << Operator::getQualCppClassName(def->getName());
-      },
+      // TODO: We are constructing the Operator wrapper instance just for
+      // getting it's qualified class name here. Reduce the overhead by having a
+      // lightweight version of Operator class just for that purpose.
+      defs, [&os](Record *def) { os << Operator(def).getQualCppClassName(); },
       [&os]() { os << ",\n"; });
 }