Commit Graph

73 Commits

Author SHA1 Message Date
Graydon Hoare 926cd9b837 [YAML] Escape non-printable multibyte UTF8 in Output::scalarString.
The existing YAML Output::scalarString code path includes a partial and
incorrect implementation of YAML escaping logic. In particular, the logic put
in place in rL321283 escapes non-printable bytes only if they are not part of a
multibyte UTF8 sequence; implicitly this means that all multibyte UTF8
sequences -- printable and non -- are passed through verbatim.

The simplest solution to this is to direct the Output::scalarString method to
use the standalone yaml::escape function, and this _almost_ works, except that
the existing code in that function _over_ escapes: any multibyte UTF8 sequence
is escaped, even printable ones. While this is permitted for YAML, it is also
more aggressive (and hard to read for non-English locales) than necessary,
and the entire point of rL321283 was to back off such aggressive over-escaping.

So in this change, I have both redirected Output::scalarString to use
yaml::escape _and_ modified yaml::escape to optionally restrict its escaping to
non-printables. This preserves behaviour of any existing clients while giving
them a path to more moderate escaping should they desire.

Reviewers: JDevlieghere, thegameg, MatzeB, vladimir.plyashkun

Reviewed By: thegameg

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44863

llvm-svn: 328661
2018-03-27 19:52:45 +00:00
Francis Visoiu Mistrih b2b961a3db [YAML] Fix UTF-8 handling
Previous YAML quoting patches broke UTF-8 printing in YAML: see https://reviews.llvm.org/D41290#961801.

Differential Revision: https://reviews.llvm.org/D41490

llvm-svn: 321283
2017-12-21 17:14:09 +00:00
Francis Visoiu Mistrih b213b27ee3 [YAML] Add support for non-printable characters
LLVM IR function names which disable mangling start with '\01'
(https://www.llvm.org/docs/LangRef.html#identifiers).

When an identifier like "\01@abc@" gets dumped to MIR, it is quoted, but
only with single quotes.

http://www.yaml.org/spec/1.2/spec.html#id2770814:

"The allowed character range explicitly excludes the C0 control block
allowed), the surrogate block #xD800-#xDFFF, #xFFFE, and #xFFFF."

http://www.yaml.org/spec/1.2/spec.html#id2776092:

"All non-printable characters must be escaped.
[...]
Note that escape sequences are only interpreted in double-quoted scalars."

This patch adds support for printing escaped non-printable characters
between double quotes if needed.

Should also fix PR31743.

Differential Revision: https://reviews.llvm.org/D41290

llvm-svn: 320996
2017-12-18 17:38:03 +00:00
Dave Lee c6f2e69695 Allow empty mappings for optional YAML input
Summary:
This change fixes a bug where `obj2yaml` can in some cases produce YAML that
causes `yaml2obj` to error.

The ELF YAML document structure has a `Sections` mapping, which contains three
mappings, all of which are optional: `Local`, `Global`, and `Weak.` Any one of
these can be missing, but if all three are missing, then `yaml2obj` errors. This
change allows YAML input for cases like this one.

I have tested this with check-llvm and check-lld, and all tests passed.

This change is the result of test failures while working on D39582, which
introduces a `DynamicSymbols` mapping, which will be empty at times.

Reviewers: compnerd, jakehehrlich, silvas, kledzik, mehdi_amini, pcc

Reviewed By: compnerd

Subscribers: silvas, llvm-commits

Differential Revision: https://reviews.llvm.org/D39908

llvm-svn: 318428
2017-11-16 17:46:11 +00:00
George Rimar 3674fb6f2c [yaml2obj] - Don't crash on one more invalid document.
This fixes one more crash I faced.
Testcase contains minimal reduced case.

Differential revision: https://reviews.llvm.org/D38082

llvm-svn: 313868
2017-09-21 08:25:59 +00:00
Alex Bradbury 1684317583 [YAMLTraits] Add filename support to yaml::Input
Summary:
The current yaml::Input constructor takes a StringRef of data as its
first parameter, discarding any filename information that may have been
present when a YAML file was opened. Add an alterate yaml::Input
constructor that takes a MemoryBufferRef, which can have a filename
associated with it. This leads to clearer diagnostic messages.

Sponsored By: DARPA, AFRL

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D35398

Patch by: Jonathan Anderson (trombonehero)

llvm-svn: 308172
2017-07-17 11:41:30 +00:00
Pavel Labath ec000f42fa [ADT] Add llvm::to_float
Summary:
The function matches the interface of llvm::to_integer, but as we are
calling out to a C library function, I let it take a Twine argument, so
we can avoid a string copy at least in some cases.

I add a test and replace a couple of existing uses of strtod with this
function.

Reviewers: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34518

llvm-svn: 306096
2017-06-23 12:55:02 +00:00
Eugene Zelenko 72208a8226 [ProfileData, Support] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 305969
2017-06-21 23:19:47 +00:00
Zachary Turner 84efd4deb6 [YAML] When outputting, provide the ability to write default values.
Previously, if you attempted to write a key/value pair and the
value was equal to the key's default value, we would not output
the value.  Sometimes it is useful to be able to see this value
in the output anyway.

llvm-svn: 297864
2017-03-15 17:47:39 +00:00
Peter Collingbourne efdff71b05 YAML: Remove Input::MapHNode::isValidKey(), use llvm::is_contained() instead. NFC.
llvm-svn: 290999
2017-01-04 20:10:43 +00:00
Peter Collingbourne 87dd2ab000 Support: Add YAML I/O support for custom mappings.
This will be used to YAMLify parts of the module summary.

Differential Revision: https://reviews.llvm.org/D28014

llvm-svn: 290935
2017-01-04 03:51:36 +00:00
Mehdi Amini 3ab3fef2f1 Improve error handling in YAML parsing
Some scanner errors were not checked and reported by the parser.

Fix PR30934. Recommit r288014 after fixing unittest.

Patch by: Serge Guelton <serge.guelton@telecom-bretagne.eu>

Differential Revision: https://reviews.llvm.org/D26419

llvm-svn: 288071
2016-11-28 21:38:52 +00:00
Mehdi Amini 43c2428203 Revert "Improve error handling in YAML parsing"
This reverts commit r288014, the unittest isn't passing

llvm-svn: 288015
2016-11-28 04:57:04 +00:00
Mehdi Amini c54281be4f Improve error handling in YAML parsing
Some scanner errors were not checked and reported by the parser.

Fix PR30934

Patch by: Serge Guelton <serge.guelton@telecom-bretagne.eu>

Differential Revision: https://reviews.llvm.org/D26419

llvm-svn: 288014
2016-11-28 04:44:13 +00:00
Chris Bieneman 92b2e8a295 [YAML] Fix YAML tags appearing before the start of sequence elements
Our existing yaml::Output code writes tags immediately when mapTag is called, without any state handling. This results in tags on sequence elements being written before the element itself. For example, we see this:

SomeArray:     !elem_type
  - key1:         1
    key2:         2 !elem_type2
  - key3:         3
    key4:         4

We should instead see:

SomeArray:
  - !elem_type
    key1:         1
    key2:         2
  - !elem_type2
    key3:         3
    key4:         4

Our reader handles reading properly, so this bug only impacts writing yaml sequences with tagged elements.

As a test for this I've modified the Mach-O yaml encoding to allways apply the !mach-o tag when encoding MachOYAML::Object entries. This results in the !mach-o tag appearing as expected in dumped fat files.

llvm-svn: 274067
2016-06-28 21:10:26 +00:00
Benjamin Kramer 7a92377a2b [YAMLTraits] Use StringRef::copy. No functionality change.
llvm-svn: 244044
2015-08-05 14:16:38 +00:00
Frederic Riss 4939e6a1b8 [YAMLIO] Make line-wrapping configurable and test it.
Summary:
We would wrap flow mappings and sequences when they go over a hardcoded 70
characters limit. Make the wrapping column configurable (and default to 70
co the change should be NFC for current users). Passing 0 allows to completely
suppress the wrapping which makes it easier to handle in tools like FileCheck.

Reviewers: bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10109

llvm-svn: 238584
2015-05-29 17:56:28 +00:00
Alex Lorenz 2bdb4e1063 Resubmit r237954 (MIR Serialization: print and parse LLVM IR using MIR format).
This commit a 3rd attempt at comitting the initial MIR serialization patch.
The first commit (r237708) was reverted in 237730. Then the second commit
(r237954) was reverted in r238007, as the MIR library under CodeGen caused
a circular dependency where the CodeGen library depended on MIR and MIR
library depended on CodeGen.

This commit has fixed the dependencies between CodeGen and MIR by
reorganizing the MIR serialization code - the code that prints out
MIR has been moved to CodeGen, and the MIR library has been renamed
to MIRParser. Now the CodeGen library doesn't depend on the
MIRParser library, thus the circular dependency no longer exists.

--Original Commit Message--

MIR Serialization: print and parse LLVM IR using MIR format.

This commit is the initial commit for the MIR serialization project.
It creates a new library under CodeGen called 'MIR'. This new
library adds a new machine function pass that prints out the LLVM IR
using the MIR format. This pass is then added as a last pass when a
'stop-after' option is used in llc. The new library adds the initial
functionality for parsing of MIR files as well. This commit also
extends the llc tool so that it can recognize and parse MIR input files.

Reviewers: Duncan P. N. Exon Smith, Matthias Braun, Philip Reames

Differential Revision: http://reviews.llvm.org/D9616 

llvm-svn: 238341
2015-05-27 18:02:19 +00:00
NAKAMURA Takumi 263b27997d Revert r237954, "Resubmit r237708 (MIR Serialization: print and parse LLVM IR using MIR format)."
It brought cyclic dependencies between LLVMCodeGen and LLVMMIR.

llvm-svn: 238007
2015-05-22 07:17:07 +00:00
Alex Lorenz c37baf82a9 Resubmit r237708 (MIR Serialization: print and parse LLVM IR using MIR format).
This commit is a 2nd attempt at committing the initial MIR serialization patch.
The first commit (r237708) made the incremental buildbots unstable and was 
reverted in r237730. The original commit didn't add a terminating null 
character to the LLVM IR source which was passed to LLParser, and this 
sometimes caused the test 'llvmIR.mir' to fail with a parsing error because 
the LLVM IR source didn't have a null character immediately after the end 
and thus LLLexer encountered some garbage characters that ultimately caused 
the error.

This commit also includes the other test fixes I committed in
r237712 (llc path fix) and r237723 (remove target triple) which
also got reverted in r237730.

--Original Commit Message--

MIR Serialization: print and parse LLVM IR using MIR format.

This commit is the initial commit for the MIR serialization project.
It creates a new library under CodeGen called 'MIR'. This new
library adds a new machine function pass that prints out the LLVM IR 
using the MIR format. This pass is then added as a last pass when a 
'stop-after' option is used in llc. The new library adds the initial 
functionality for parsing of MIR files as well. This commit also 
extends the llc tool so that it can recognize and parse MIR input files.

Reviewers: Duncan P. N. Exon Smith, Matthias Braun, Philip Reames

Differential Revision: http://reviews.llvm.org/D9616

llvm-svn: 237954
2015-05-21 20:54:45 +00:00
Alex Lorenz de1970fe66 Revert r237708 (MIR serialization) - incremental buildbots became unstable.
The incremental buildbots entered a pass-fail cycle where during the fail
cycle one of the tests from this commit fails for an unknown reason. I
have reverted this commit and will investigate the cause of this problem.

llvm-svn: 237730
2015-05-19 21:41:28 +00:00
Alex Lorenz c5e0d4d146 MIR Serialization: print and parse LLVM IR using MIR format.
This commit is the initial commit for the MIR serialization project.
It creates a new library under CodeGen called 'MIR'. This new
library adds a new machine function pass that prints out the LLVM IR 
using the MIR format. This pass is then added as a last pass when a 
'stop-after' option is used in llc. The new library adds the initial 
functionality for parsing of MIR files as well. This commit also 
extends the llc tool so that it can recognize and parse MIR input files.

Reviewers: Duncan P. N. Exon Smith, Matthias Braun, Philip Reames

Differential Revision: http://reviews.llvm.org/D9616

llvm-svn: 237708
2015-05-19 18:17:39 +00:00
Alex Lorenz 68e787bdb0 YAML: Add support for literal block scalar I/O.
This commit gives the users of the YAML Traits I/O library 
the ability to serialize scalars using the YAML literal block 
scalar notation by allowing them to implement a specialization 
of the `BlockScalarTraits` struct for their custom types.

Reviewers: Duncan P. N. Exon Smith

Differential Revision: http://reviews.llvm.org/D9613

llvm-svn: 237404
2015-05-14 23:08:22 +00:00
Alex Lorenz 7a38d75bcd Revert r237157, "YAML: Fix typos. NFC".
'Iff' isn't a typo, it's a shorthand for 'if and only if'.

llvm-svn: 237160
2015-05-12 17:44:32 +00:00
Alex Lorenz f63ddf1d3a YAML: Fix typos. NFC.
llvm-svn: 237157
2015-05-12 17:31:17 +00:00
Alex Lorenz b122508501 YAML: Add an optional 'flow' field to the mapping trait to allow flow mapping output.
This patch adds an optional 'flow' field to the MappingTrait
class so that yaml IO will be able to output flow mappings.

Reviewers: Justin Bogner

Differential Revision: http://reviews.llvm.org/D9450

llvm-svn: 236456
2015-05-04 20:11:40 +00:00
Alex Lorenz 42e91fa312 YAML: Fix the output of sequences that contain flow sequences.
This patch fixes a bug where the YAML Output class emitted
a sequence of flow sequences without the '-' characters.
Before:
  
  seq:
    [ a, b ]
    [ c, d ]

After:

  seq:
    - [ a, b ]
    - [ c, d ]


Reviewers: Justin Bogner

Differential Revision: http://reviews.llvm.org/D9206

llvm-svn: 236329
2015-05-01 18:34:25 +00:00
Benjamin Kramer 16132e6faa Purge unused includes throughout libSupport.
NFC.

llvm-svn: 232976
2015-03-23 18:07:13 +00:00
Justin Bogner 64d2cdf4ec Detect malformed YAML sequence in yaml::Input::beginSequence()
When reading a yaml::SequenceTraits object, YAMLIO does not report an
error if the yaml item is not a sequence. Instead, YAMLIO reads an
empty sequence. For example:

---
seq:
    foo: 1
    bar: 2
...

If `seq` is a SequenceTraits object, then reading the above yaml will
yield `seq` as an empty sequence.

Fix this to report an error for the above mapping ("not a sequence")

Patch by William Fisher. Thanks!

llvm-svn: 230976
2015-03-02 17:26:43 +00:00
Alex Rosenberg f298f16ccf Remove trailing whitespace. NFC ®
llvm-svn: 227105
2015-01-26 18:02:18 +00:00
Michael J. Spencer 731cae3839 [YAMLIO] Add support for numeric values in enums.
llvm-svn: 226942
2015-01-23 21:57:50 +00:00
Chandler Carruth d9903888d9 [cleanup] Re-sort all the #include lines in LLVM using
utils/sort_includes.py.

I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.

llvm-svn: 225974
2015-01-14 11:23:27 +00:00
David Blaikie d759fe5764 Fix memory leak in error paths in YAMLTraits by using unique_ptr
There's some other cleanup that could happen here, but this is at least
the mechanical transformation to unique_ptr.

Derived from a patch by Anton Yartsev.

llvm-svn: 217803
2014-09-15 18:39:24 +00:00
Rafael Espindola a97373f235 Fix bug 20125 - clang-format segfaults on bad config.
The problem was in unchecked dyn_cast inside of Input::createHNodes.
Patch by Roman Kashitsyn!

llvm-svn: 215205
2014-08-08 13:58:00 +00:00
Rafael Espindola 2a826e40fa Finishing touch for the std::error_code transition.
While std::error_code itself seems to work OK in all platforms, there
are few annoying differences with regards to the std::errc enumeration.

This patch adds a simple llvm enumeration, which will hopefully avoid build
breakages in other platforms and surprises as we get more uses of
std::error_code.

llvm-svn: 210920
2014-06-13 17:20:48 +00:00
Rafael Espindola db4ed0bdab Remove 'using std::errro_code' from lib.
llvm-svn: 210871
2014-06-13 02:24:39 +00:00
Rafael Espindola 3acea39853 Don't use 'using std::error_code' in include/llvm.
This should make sure that most new uses use the std prefix.

llvm-svn: 210835
2014-06-12 21:46:39 +00:00
Rafael Espindola ed6882b835 Don't import make_error_code into the llvm namespace.
llvm-svn: 210772
2014-06-12 11:58:49 +00:00
Rafael Espindola 5c4f829424 Use std::error_code instead of llvm::error_code.
The idea of this patch is to turn llvm/Support/system_error.h into a
transitional header that just brings in the erorr_code api to the llvm
namespace. I will remove it shortly afterwards.

The cases where the general idea needed some tweaking:

* std::errc is a namespace in msvc, so we cannot use "using std::errc". I could
add an #ifdef, but there were not that many uses, so I just added std:: to
them in this patch.

* Template specialization had to be moved to the std namespace in this
patch set already.

* The msvc implementation of default_error_condition doesn't seem to
provide the same transformations as we need. Not too surprising since
the standard doesn't actually say what "equivalent" means. I fixed the
problem by keeping our old mapping and using it at error_code
construction time.

Despite these shortcomings I think this is still a good thing. Some reasons:

* The different implementations of system_error might improve over time.
* It removes 925 lines of code from llvm already.
* It removes 6313 bytes from the text segment of the clang binary when
it is built with gcc and 2816 bytes when building with clang and
libstdc++.

llvm-svn: 210687
2014-06-11 19:05:50 +00:00
Simon Atanasyan f97af8a084 [yaml2obj] Add new command line option `-docnum`.
Input YAML file might contain multiple object file definitions.
New option `-docnum` allows to specify an ordinal number (starting from 1)
of definition used for an object file generation.

Patch reviewed by Sean Silva.

llvm-svn: 209967
2014-05-31 04:51:07 +00:00
David Majnemer 7788033be6 YAMLIO: Allow scalars to dictate quotation rules
Introduce ScalarTraits::mustQuote which determines whether or not a
StringRef needs quoting before it is acceptable to output.

llvm-svn: 205955
2014-04-10 07:37:33 +00:00
Simon Atanasyan 878bd8ad9c Use range-based for loops. No functionality change.
llvm-svn: 205953
2014-04-10 06:02:49 +00:00
David Majnemer 97d8ee3824 Revert "Revert "YAMLIO: Encode ambiguous hex strings explicitly""
Don't quote octal compatible strings if they are only two wide, they
aren't ambiguous.

This reverts commit r205857 which reverted r205857.

llvm-svn: 205914
2014-04-09 17:04:27 +00:00
Filipe Cabecinhas 2c4e8ae0fd Revert "YAMLIO: Encode ambiguous hex strings explicitly"
This reverts commit r205839.

It broke several tests in lld.

llvm-svn: 205857
2014-04-09 14:35:17 +00:00
David Majnemer 815433587c YAMLIO: Encode ambiguous hex strings explicitly
YAMLIO would turn a BinaryRef into the string 0000000004000000.
However, the leading zero causes parsers to interpret it as being an
octal number instead of a hexadecimal one.

Instead, escape such strings as needed.

llvm-svn: 205839
2014-04-09 07:56:27 +00:00
Craig Topper c10719f55d [C++11] Make use of 'nullptr' in the Support library.
llvm-svn: 205697
2014-04-07 04:17:22 +00:00
Chandler Carruth 8a8cd2bab9 Re-sort all of the includes with ./utils/sort_includes.py so that
subsequent changes are easier to review. About to fix some layering
issues, and wanted to separate out the necessary churn.

Also comment and sink the include of "Windows.h" in three .inc files to
match the usage in Memory.inc.

llvm-svn: 198685
2014-01-07 11:48:04 +00:00
Nick Kledzik 4761c60eef revert r194655
llvm-svn: 195285
2013-11-21 00:20:10 +00:00
John Thompson 48e018a314 YAML I/O - Added default trait support for std:string. Making another attempt at this, this time doing a clean build on Linux, and running the LLVM, clang, and extra tests, to try to make sure there's no problems.
llvm-svn: 195134
2013-11-19 17:28:21 +00:00
Juergen Ributzka d12ccbd343 [weak vtables] Remove a bunch of weak vtables
This patch removes most of the trivial cases of weak vtables by pinning them to
a single object file. The memory leaks in this version have been fixed. Thanks
Alexey for pointing them out.

Differential Revision: http://llvm-reviews.chandlerc.com/D2068

Reviewed by Andy

llvm-svn: 195064
2013-11-19 00:57:56 +00:00