a NaN-test prior to the call to the library function.
This should automatically make fastmath (including just non-NaNs) able to avoid
the expensive libcalls and also open the door to more advanced folding in LLVM
based on the rules for complex math.
Two important notes to remember: first is that this isn't yet a proper
limited range mode, it's still just improving the unlimited range mode.
Also, it isn't really perfecet w.r.t. what an unlimited range mode
should be doing because it isn't quite handling the flags produced by
all the operations in the way desirable for that mode, but then neither
is compiler-rt's libcall. When the compiler-rt libcall is improved to
carefully manage flags, the code emitted here should be improved
correspondingly. And it is still a long-term desirable thing to add
a limited range mode to Clang that would be able to use direct math
without library calls here.
Special thanks to Steve Canon for the careful review on this patch and
teaching me about these issues. =D
Differential Revision: http://reviews.llvm.org/D5756
llvm-svn: 220167
for complex math.
This should fix the windows build bots that started having trouble here
and generally fix complex libcall emission on targets which use sret for
complex data types. It also makes the code a bit simpler (despite
calling into a much more complex bucket of code).
llvm-svn: 219565
operators where one type is a C complex type, and to emit both the
efficient and correct implementation for complex arithmetic according to
C11 Annex G using this extra information.
For both multiply and divide the old code was writing a long-hand
reduced version of the math without any of the special handling of inf
and NaN recommended by the standard here. Instead of putting more
complexity here, this change does what GCC does which is to emit
a libcall for the fully general case.
However, the old code also failed to do the proper minimization of the
set of operations when there was a mixed complex and real operation. In
those cases, C provides a spec for much more minimal operations that are
valid. Clang now emits the exact suggested operations. This change isn't
*just* about performance though, without minimizing these operations, we
again lose the correct handling of infinities and NaNs. It is critical
that this happen in the frontend based on assymetric type operands to
complex math operations.
The performance implications of this change aren't trivial either. I've
run a set of benchmarks in Eigen, an open source mathematics library
that makes heavy use of complex. While a few have slowed down due to the
libcall being introduce, most sped up and some by a huge amount: up to
100% and 140%.
In order to make all of this work, also match the algorithm in the
constant evaluator to the one in the runtime library. Currently it is
a broken port of the simplifications from C's Annex G to the long-hand
formulation of the algorithm.
Splitting this patch up is very hard because none of this works without
the AST change to preserve non-complex operands. Sorry for the enormous
change.
Follow-up changes will include support for sinking the libcalls onto
cold paths in common cases and fastmath improvements to allow more
aggressive backend folding.
Differential Revision: http://reviews.llvm.org/D5698
llvm-svn: 219557
Previously, we made one traversal of the AST prior to codegen to assign
counters to the ASTs and then propagated the count values during codegen. This
patch now adds a separate AST traversal prior to codegen for the
-fprofile-instr-use option to propagate the count values. The counts are then
saved in a map from which they can be retrieved during codegen.
This new approach has several advantages:
1. It gets rid of a lot of extra PGO-related code that had previously been
added to codegen.
2. It fixes a serious bug. My original implementation (which was mailed to the
list but never committed) used 3 counters for every loop. Justin improved it to
move 2 of those counters into the less-frequently executed breaks and continues,
but that turned out to produce wrong count values in some cases. The solution
requires visiting a loop body before the condition so that the count for the
condition properly includes the break and continue counts. Changing codegen to
visit a loop body first would be a fairly invasive change, but with a separate
AST traversal, it is easy to control the order of traversal. I've added a
testcase (provided by Justin) to make sure this works correctly.
3. It improves the instrumentation overhead, reducing the number of counters for
a loop from 3 to 1. We no longer need dedicated counters for breaks and
continues, since we can just use the propagated count values when visiting
breaks and continues.
To make this work, I needed to make a change to the way we count case
statements, going back to my original approach of not including the fall-through
in the counter values. This was necessary because there isn't always an AST node
that can be used to record the fall-through count. Now case statements are
handled the same as default statements, with the fall-through paths branching
over the counter increments. While I was at it, I also went back to using this
approach for do-loops -- omitting the fall-through count into the loop body
simplifies some of the calculations and make them behave the same as other
loops. Whenever we start using this instrumentation for coverage, we'll need
to add the fall-through counts into the counter values.
llvm-svn: 201528
adjustFallThroughCount isn't a good name, and the documentation was
even worse. This commit attempts to clarify what it's for and when to
use it.
llvm-svn: 199139
With the introduction of explicit address space casts into LLVM, there's
a need to provide a new cast kind the front-end can create for C/OpenCL/CUDA
and code to produce address space casts from those kinds when appropriate.
Patch by Michele Scandale!
llvm-svn: 197036
This is the same way GenericSelectionExpr works, and it's generally a
more consistent approach.
A large part of this patch is devoted to caching the value of the condition
of a ChooseExpr; it's needed to avoid threading an ASTContext into
IgnoreParens().
Fixes <rdar://problem/14438917>.
llvm-svn: 186738
_Complex load/store didn't have their alignment set properly, which was visible when GCC's torture tests use volatile _Complex.
Update some existing tests to check for alignment, and add a new test which also has over-aligned volatile _Complex (since the imaginary part shouldn't be overaligned, only the real part).
llvm-svn: 186490
Add a CXXDefaultInitExpr, analogous to CXXDefaultArgExpr, and use it both in
CXXCtorInitializers and in InitListExprs to represent a default initializer.
There's an additional complication here: because the default initializer can
refer to the initialized object via its 'this' pointer, we need to make sure
that 'this' points to the right thing within the evaluation.
llvm-svn: 179958
aggregate types in a profoundly wrong way that has to be
worked around in every call site, to getEvaluationKind,
which classifies and distinguishes between all of these
cases.
Also, normalize the API for loading and storing complexes.
I'm working on a larger patch and wanted to pull these
changes out, but it would have be annoying to detangle
them from each other.
llvm-svn: 176656
uncovered.
This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.
I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.
llvm-svn: 169237
(__builtin_* etc.) so that it isn't possible to take their address.
Specifically, introduce a new type to represent a reference to a builtin
function, and a new cast kind to convert it to a function pointer in the
operand of a call. Fixes PR13195.
llvm-svn: 162962
track whether the referenced declaration comes from an enclosing
local context. I'm amenable to suggestions about the exact meaning
of this bit.
llvm-svn: 152491
we correctly emit loads of BlockDeclRefExprs even when they
don't qualify as ODR-uses. I think I'm adequately convinced
that BlockDeclRefExpr can die.
llvm-svn: 152479
block pointer that returns a block literal which captures (by copy)
the lambda closure itself. Some aspects of the block literal are left
unspecified, namely the capture variable (which doesn't actually
exist) and the body (which will be filled in by IRgen because it can't
be written as an AST).
Because we're switching to this model, this patch also eliminates
tracking the copy-initialization expression for the block capture of
the conversion function, since that information is now embedded in the
synthesized block literal. -1 side tables FTW.
llvm-svn: 151131
is general goodness because representations of member pointers are
not always equivalent across member pointer types on all ABIs
(even though this isn't really standard-endorsed).
Take advantage of the new information to teach IR-generation how
to do these reinterprets in constant initializers. Make sure this
works when intermingled with hierarchy conversions (although
this is not part of our motivating use case). Doing this in the
constant-evaluator would probably have been better, but that would
require a *lot* of extra structure in the representation of
constant member pointers: you'd really have to track an arbitrary
chain of hierarchy conversions and reinterpretations in order to
get this right. Ultimately, this seems less complex. I also
wasn't quite sure how to extend the constant evaluator to handle
foldings that we don't actually want to treat as extended
constant expressions.
llvm-svn: 150551
- Add atomic-to/from-nonatomic cast types
- Emit atomic operations for arithmetic on atomic types
- Emit non-atomic stores for initialisation of atomic types, but atomic stores and loads for every other store / load
- Add a __atomic_init() intrinsic which does a non-atomic store to an _Atomic() type. This is needed for the corresponding C11 stdatomic.h function.
- Enables the relevant __has_feature() checks. The feature isn't 100% complete yet, but it's done enough that we want people testing it.
Still to do:
- Make the arithmetic operations on atomic types (e.g. Atomic(int) foo = 1; foo++;) use the correct LLVM intrinsic if one exists, not a loop with a cmpxchg.
- Add a signal fence builtin
- Properly set the fenv state in atomic operations on floating point values
- Correctly handle things like _Atomic(_Complex double) which are too large for an atomic cmpxchg on some platforms (this requires working out what 'correctly' means in this context)
- Fix the many remaining corner cases
llvm-svn: 148242
full-expression. Naturally they're inactive before we enter
the block literal expression. This restores the intended
behavior that blocks belong to their enclosing scope.
There's a useful -O0 / compile-time optimization that we're
missing here with activating cleanups following straight-line
code from their inactive beginnings.
llvm-svn: 144268
property references to use a new PseudoObjectExpr
expression which pairs a syntactic form of the expression
with a set of semantic expressions implementing it.
This should significantly reduce the complexity required
elsewhere in the compiler to deal with these kinds of
expressions (e.g. IR generation's special l-value kind,
the static analyzer's Message abstraction), at the lower
cost of specifically dealing with the odd AST structure
of these expressions. It should also greatly simplify
efforts to implement similar language features in the
future, most notably Managed C++'s properties and indexed
properties.
Most of the effort here is in dealing with the various
clients of the AST. I've gone ahead and simplified the
ObjC rewriter's use of properties; other clients, like
IR-gen and the static analyzer, have all the old
complexity *and* all the new complexity, at least
temporarily. Many thanks to Ted for writing and advising
on the necessary changes to the static analyzer.
I've xfailed a small diagnostics regression in the static
analyzer at Ted's request.
llvm-svn: 143867
expressions: expressions which refer to a logical rather
than a physical l-value, where the logical object is
actually accessed via custom getter/setter code.
A subsequent patch will generalize the AST for these
so that arbitrary "implementing" sub-expressions can
be provided.
Right now the only client is ObjC properties, but
this should be generalizable to similar language
features, e.g. Managed C++'s __property methods.
llvm-svn: 142914
the lifetime of the block by copying it to the heap, or else we'll get
a dangling reference because the code working with the non-block-typed
object will not know it needs to copy.
There is some danger here, e.g. with assigning a block literal to an
unsafe variable, but, well, it's an unsafe variable.
llvm-svn: 139451
than conversions of C pointers to ObjC pointers. In order to ensure that
we've caught every case, add asserts to CastExpr that strictly determine
which cast kind is used for which kind of bit cast.
llvm-svn: 139352
to represent a fully-substituted non-type template parameter.
This should improve source fidelity, as well as being generically
useful for diagnostics and such.
llvm-svn: 135243
where we have an immediate need of a retained value.
As an exception, don't do this when the call is made as the immediate
operand of a __bridge retain. This is more in the way of a workaround
than an actual guarantee, so it's acceptable to be brittle here.
rdar://problem/9504800
llvm-svn: 134605
class and to bind the shared value using OpaqueValueExpr. This fixes an
unnoticed problem with deserialization of these expressions where the
deserialized form would lose the vital pointer-equality trait; or rather,
it fixes it because this patch also does the right thing for deserializing
OVEs.
Change OVEs to not be a "temporary object" in the sense that copy elision is
permitted.
This new representation is not totally unawkward to work with, but I think
that's really part and parcel with the semantics we're modelling here. In
particular, it's much easier to fix things like the copy elision bug and to
make the CFG look right.
I've tried to update the analyzer to deal with this in at least some
obvious cases, and I think we get a much better CFG out, but the printing
of OpaqueValueExprs probably needs some work.
llvm-svn: 125744
I'm separately committing this because it incidentally changes some
block orderings and minor IR issues, like using a phi instead of
an unnecessary alloca.
llvm-svn: 124277
the LHS, or else the pointer might be invalid. This is kindof dumb, but
go ahead and make sure we're doing that for l-value scalar assignment,
which fixes a miscompile of obj-c++.dg/block-seq.mm.
Leave a FIXME for how to solve this problem for agg __blocks.
llvm-svn: 120992
Fix a bug in the emission of complex compound assignment l-values.
Introduce a method to emit an expression whose value isn't relevant.
Make that method evaluate its operand as an l-value if it is one.
Fixes our volatile compliance in C++.
llvm-svn: 120931
not actually frequently used, because ImpCastExprToType only creates a node
if the types differ. So explicitly create an ICE in the lvalue-to-rvalue
conversion code in DefaultFunctionArrayLvalueConversion() as well as several
other new places, and consistently deal with the consequences throughout the
compiler.
In addition, introduce a new cast kind for loading an ObjCProperty l-value,
and make sure we emit those nodes whenever an ObjCProperty l-value appears
that's not on the LHS of an assignment operator.
This breaks a couple of rewriter tests, which I've x-failed until future
development occurs on the rewriter.
Ted Kremenek kindly contributed the analyzer workarounds in this patch.
llvm-svn: 120890
assignment to volatiles in C. This in effect reverts some of mjs's
work in and around r72572. Basically, the C++ standard is quite
clear, except that it lies about volatile behavior approximating
C's, whereas the C standard is almost actively misleading.
llvm-svn: 119344
Return the result of a complex assignment with the original values,
not by performing a load from the l-value; this is the correct
semantics in C, although not in C++.
llvm-svn: 119037
implicit conversions; the last batch was specific to promotions.
I think this is the full set we need. I do think dividing the cast
kinds into floating and integral is probably a good idea.
Annotate a *lot* more C casts with useful cast kinds.
llvm-svn: 119036
(e.g., a call, cast, etc.), immediately adjust the expression's type
to strip cv-qualifiers off of all non-class types (in C++) or all
types (in C). This effectively extends my previous fix for PR7463,
which was restricted to calls, to other kinds of expressions within
similar characteristics. I've audited every use of
getNonReferenceType() in the code base, switching to the newly-renamed
getNonLValueExprType() where necessary.
Big thanks to Eli for pointing out just how incomplete my original fix
for PR7463 actually was. We've been handling cv-qualifiers on rvalues
wrong for a very, very long time. Fixes PR7463.
llvm-svn: 108253
complex values either. Previously we did this properly for regular assignment,
but not for compound assignment.
- Also, tidy up assignment code a bit to look more like the scalar path.
llvm-svn: 107217
Several of the existing methods were identical to their respective
specializations, and so have been removed entirely. Several more 'leaf'
optimizations were introduced.
The getAsFoo() methods which imposed extra conditions, like
getAsObjCInterfacePointerType(), have been left in place.
llvm-svn: 82501
Craig Topper