[mlir] Allow C-style escapes in Lexer

This patch passes the raw, unescaped value through to the rest of the stack. Partial escaping is a total pain to deal with, so we either need to implement escaping properly (ideally using a third party library like absl, I don't think LLVM has one that can handle the proper gamut of escape codes) or don't escape. I chose the latter for this patch.

PiperOrigin-RevId: 208608945
This commit is contained in:
James Molloy 2018-08-14 01:16:45 -07:00 committed by jpienaar
parent 3e92be9c71
commit ab60afb234
3 changed files with 16 additions and 3 deletions

View File

@ -323,6 +323,12 @@ Token Lexer::lexString(const char *tokStart) {
case '\v':
case '\f':
return emitError(curPtr-1, "expected '\"' in string literal");
case '\\':
// Handle explicitly \" -> ".
// TODO(someone): define more escaping rules.
if (*curPtr == '"')
++curPtr;
continue;
default:
continue;

View File

@ -81,9 +81,8 @@ Optional<unsigned> Token::getIntTypeBitwidth() const {
/// Given a 'string' token, return its value, including removing the quote
/// characters and unescaping the contents of the string.
std::string Token::getStringValue() const {
// TODO: Handle escaping.
// Just drop the quotes off for now.
// Start by dropping the quotes.
// TODO: Un-escape the string here instead of passing through the raw content.
return getSpelling().drop_front().drop_back().str();
}

View File

@ -315,3 +315,11 @@ bb0:
"foo"(){bar: tensor<??f32>} : () -> ()
return
}
// CHECK-LABEL: cfgfunc @stringquote
cfgfunc @stringquote() -> () {
bb0:
// CHECK: "foo"() {bar: "a\"quoted\"string"} : () -> ()
"foo"(){bar: "a\"quoted\"string"} : () -> ()
return
}