during deserialization from a precompiled header, and update all of
its callers to note when this problem occurs and recover (more)
gracefully. Fixes <rdar://problem/9119249>.
llvm-svn: 129839
should report the original file name for contents of files that were overriden by other files,
otherwise it should report the name of the new file. Default is true.
Also add similar field in PreprocessorOptions and pass similar parameter in ASTUnit::LoadFromCommandLine.
llvm-svn: 127289
Allow remapping a file by specifying another filename whose contents should be loaded if the original
file gets loaded. This allows to override files without having to create & load buffers in advance.
llvm-svn: 127052
whose inode has changed since the file was first created and that is
being seen through a different path name (e.g., due to symlinks or
relative path elements), such that its FileEntry pointer doesn't match
a known FileEntry pointer. Since this requires a system call (to
stat()), we only perform this deeper checking if we can't find the
file by comparing FileEntry pointers.
Also, add a micro-optimization where we don't bother to compute line
numbers when given the location (1, 1). This improves the
efficiency of clang_getLocationForOffset().
llvm-svn: 124800
FileSystemOpts through a ton of apis, simplifying a lot of code.
This also fixes a latent bug in ASTUnit where it would invoke
methods on FileManager without creating one in some code paths
in cindextext.
llvm-svn: 120010
This patch completely defeated the "passing in a prestat'd size
to MemoryBuffer" optimization, leading to an extra fstat call for
every buffer opened, in order to find out if the datestamp and size
of the file on disk matches what is in the stat cache.
I fully admit that I don't completely understand what is going on here:
why punish code when a stat cache isn't in use? what is the point of a
stat cache if you have to turn around and stat stuff to validate it?
To resolve both these issues, just drop the modtime check and check the
file size, which is the important thing anyway. This should also resolve
PR6812, because presumably windows is stable when it comes to file sizes.
If the modtime is actually important, we should get it and keep it on the
first stat.
This eliminates 833 fstat syscalls when processing Cocoa.h, speeding up
system time on -Eonly Cocoa.h from 0.041 to 0.038s.
llvm-svn: 120001
-Move the stuff of Diagnostic related to creating/querying diagnostic IDs into a new DiagnosticIDs class.
-DiagnosticIDs can be shared among multiple Diagnostics for multiple translation units.
-The rest of the state in Diagnostic object is considered related and tied to one translation unit.
-Have Diagnostic point to the SourceManager that is related with. Diagnostic can now accept just a
SourceLocation instead of a FullSourceLoc.
-Reflect the changes to various interfaces.
llvm-svn: 119730
When -working-directory is passed in command line, file paths are resolved relative to the specified directory.
This helps both when using libclang (where we can't require the user to actually change the working directory)
and to help reproduce test cases when the reproduction work comes along.
--FileSystemOptions is introduced which controls how file system operations are performed (currently it just contains
the working directory value if set).
--FileSystemOptions are passed around to various interfaces that perform file operations.
--Opening & reading the content of files should be done only through FileManager. This is useful in general since
file operations will be abstracted in the future for the reproduction mechanism.
FileSystemOptions is independent of FileManager so that we can have multiple translation units sharing the same
FileManager but with different FileSystemOptions.
Addresses rdar://8583824.
llvm-svn: 118203
reparsing an ASTUnit. When saving a preamble, create a buffer larger
than the actual file we're working with but fill everything from the
end of the preamble to the end of the file with spaces (so the lexer
will quickly skip them). When we load the file, create a buffer of the
same size, filling it with the file and then spaces. Then, instruct
the lexer to start lexing after the preamble, therefore continuing the
parse from the spot where the preamble left off.
It's now possible to perform a simple preamble build + parse (+
reparse) with ASTUnit. However, one has to disable a bunch of checking
in the PCH reader to do so. That part isn't committed; it will likely
be handled with some other kind of flag (e.g., -fno-validate-pch).
As part of this, fix some issues with null termination of the memory
buffers created for the preamble; we were trying to explicitly
NULL-terminate them, even though they were also getting implicitly
NULL terminated, leading to excess warnings about NULL characters in
source files.
llvm-svn: 109445
to be algorithmically faster and avoid an std::map. This routine
basically boils down to finding the nearest common ancestor in a
tree, and we (implicitly) have information about nesting depth,
use it!
This wraps up rdar://7948633 - SourceManager::isBeforeInTranslationUnit has poor performance
llvm-svn: 103239
method to be correct. Right now it correctly computes the cache, then
goes ahead and computes the result the hard way, then asserts that they
match. Next I'll actually turn it on.
llvm-svn: 103231
method will sometimes return different results for the same input SourceLocations. I haven't
unraveled this method completely yet, so this truly is a workaround until a better fix comes
along.
llvm-svn: 103143
about it instead of producing tons of garbage from the lexer.
It would be even better for sourcemgr to dynamically transcode (e.g.
from UTF16 -> UTF8).
llvm-svn: 101924
precompiled headers and/or when reading the contents of the file into
memory. These checks seem to be causing spurious regression-test
failures on Windows.
llvm-svn: 100866
Diagnostic subsystem, which is used in the rare case where we find a
serious problem (i.e., an inconsistency in the file system) while
we're busy formatting another diagnostic. In this case, the delayed
diagnostic will be emitted after we're done with the other
diagnostic. This is only to be used for fatal conditions detected at
very inconvenient times, where we can neither stop the current
diagnostic in flight nor can we suppress the second error.
llvm-svn: 99175
deserialization of precompiled headers, where the deserialization of
the source location entry for a buffer (e.g., macro instantiation
scratch space) would overwrite a one-element FileID cache in the
source manager. When tickled at the wrong time, we would return the
wrong decomposed source location and eventually cause c-index-test to
crash.
Found by dumb luck. It's amazing this hasn't shown up before.
llvm-svn: 98940
buffer was invalid when it was created, and use that bit to always set
the "Invalid" flag according to whether the buffer is invalid. This
ensures that all accesses to an invalid buffer are marked invalid,
improving recovery.
llvm-svn: 98690
SourceManager's getBuffer() and, therefore, could fail, along with
Preprocessor::getSpelling(). Use the Invalid parameters in the literal
parsers (string, floating point, integral, character) to make them
robust against errors that stem from, e.g., PCH files that are not
consistent with the underlying file system.
I still need to audit every use caller to all of these routines, to
determine which ones need specific handling of error conditions.
llvm-svn: 98608
SourceManager's getBuffer() (and similar) operations. This abstract
can be used to force callers to cope with errors in getBuffer(), such
as missing files and changed files. Fix a bunch of callers to use the
new interface.
Add some very basic checks for file consistency (file size,
modification time) into ContentCache::getBuffer(), although these
checks don't help much until we've updated the main callers (e.g.,
SourceManager::getSpelling()).
llvm-svn: 98585
end-of-line source location when given a column number beyond the
length of the line, or an end-of-file source location when given a
line number beyond the length of the file. Previously, we would return
an invalid location.
llvm-svn: 97299
we'd add an offset from the spelling location space to the
instantiation location, which doesn't make sense and would
lead up to the text diagnostics crashing when presented with
non-sensical locations.
This fixes rdar://7597492, a crash on 255.vortex.
llvm-svn: 96004
files.
- The issue is that PCH uses a stat cache, which may reference files which have
been deleted or moved. In such cases ContentCache::getBuffer was returning 0
but most clients are incapable of dealing with this (i.e., they don't).
For the time being, resolve this issue by just making up some invalid file
contents and. Eventually we should detect that we are in an inconsistent
situation and error out with a nice message that the PCH is out of date.
llvm-svn: 90699
files with the contents of an arbitrary memory buffer. Use this new
functionality to drastically clean up the way in which we handle file
truncation for code-completion: all of the truncation/completion logic
is now encapsulated in the preprocessor where it belongs
(<rdar://problem/7434737>).
llvm-svn: 90300
The later assumption is patently false, but this was already broken -- this situation is conceptually impossible, my feeling is we should fix SourceManager and friends to make it impossible in practice as well. However, we need to fix PR5662 and perhaps some other things involving memory buffers first. In the short term I'm pretty sure this is reliable.
Chris, Argiris, is this going to break anything that wasn't already broken?
llvm-svn: 90280
in diagnostics when we fail to open a file. This allows us to
report things like:
$ clang test.c -I.
test.c:2:10: fatal error: error opening file './foo.h': Permission denied
#include "foo.h"
^
llvm-svn: 90276
-code-completion-at=filename:line:column
which performs code completion at the specified location by truncating
the file at that position and enabling code completion. This approach
makes it possible to run multiple tests from a single test file, and
gives a more natural command-line interface.
llvm-svn: 82571
file. In particular, only eagerly load source location entries for
files and for the predefines buffer. Other buffers and
macro-instantiation source location entries are loaded lazily.
With the Cocoa-prefixed "Hello, World", we only load 815/26555 source
location entities. This halves the amount of user time we spend in
this "Hello, World" program with -fsyntax-only (down to .007s).
This optimization is part 1 of 2 for the source manager. This
eliminates most of the user time in loading a PCH file. We still spend
too much time initialize File structures (especially in the calls to
stat), so we need to either make the loading of source location
entries for files lazy or import the stat cache from the PTH
implementation.
llvm-svn: 70196
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.
This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.
llvm-svn: 64601
line markers, including maintenance of the virtual include stack.
For something like this:
# 42 "bar.c" 1
# 142 "bar2.c" 1
#warning zappa
# 92 "bar.c" 2
#warning gonzo
# 102 "foo.c" 2
#warning bonkta
we now produce these three warnings:
#1:
In file included from foo.c:3:
In file included from bar.c:42:
bar2.c:143:2: warning: #warning zappa
#warning zappa
^
#2:
In file included from foo.c:3:
bar.c:92:2: warning: #warning gonzo
#warning gonzo
^
#3:
foo.c:102:2: warning: #warning bonkta
#warning bonkta
^
llvm-svn: 63722
location below it report as coming from the #line location. For example,
with:
#line 92 "foo.h"
#warning blarg!
#warning blarg!
we now emit:
foo.h:92:2: warning: #warning blarg!
#warning blarg!
^
foo.h:92:2: warning: #warning blarg!
#warning blarg!
^
llvm-svn: 63709
makes it clear to clients that they have to pick an instantiation
or spelling location before calling it and allows optimization based
on that.
llvm-svn: 63698
ContentCache objects to using a densemap and list, and allocating
the ContentCache objects from a bump pointer. This does not speed
up or slow down things substantially, but gives us control over
their alignment.
llvm-svn: 63628
as reported to the user and as manipulated by #line. This is what __FILE__,
__INCLUDE_LEVEL__, diagnostics and other things should follow (but not
dependency generation!).
This patch also includes several cleanups along the way:
- SourceLocation now has a dump method, and several other places
that did similar things now use it.
- I cleaned up some code in AnalysisConsumer, but it should probably be
simplified further now that NamedDecl is better.
- TextDiagnosticPrinter is now simplified and cleaned up a bit.
This patch is a prerequisite for #line, but does not actually provide
any #line functionality.
llvm-svn: 63098
address space we used up. Some interesting data:
For c99-intconst-1.c:
6912762 SLocEntry's allocated, 25592386B of Sloc address space used.
For cocoa.h:
26469 SLocEntry's allocated, 10278752B of Sloc address space used.
For carbon.h:
27364 SLocEntry's allocated, 12398141B of Sloc address space used.
Clearly 2G of sloc address space should be enough for anyone?!
llvm-svn: 63093
ground work for implementing #line, and fixes the "out of macro ID's"
problem.
There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.
llvm-svn: 62978
the chunk ID not the file ID. This exposes problems in
TextDiagnosticPrinter where it should have been using the canonical
file ID but wasn't. Fix these along the way.
llvm-svn: 62427
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.
This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.
FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.
llvm-svn: 62407
the "physical" location of tokens, refer to the "spelling" location.
This is more concrete and useful, tokens aren't really physical objects!
llvm-svn: 62309
- Big Idea:
Source files are now mmaped when ContentCache::getBuffer() is first called.
While this doesn't change the functionality when lexing regular source files,
it can result in source files not being paged in when using PTH.
- Performance change:
- No observable difference (-fsyntax-only/-Eonly) on Cocoa.h when doing
regular source lexing.
- No observable time difference (-fsyntax-only/-Eonly) on Cocoa.h when using
PTH. We do observe, however, a reduction of 279K in memory mapped source
code (3% reduction). The majority of pages from Cocoa.h (and friends) are
still being pulled in, however, because any literal will cause
Preprocessor::getSpelling() to be called (causing the source for the file to
get pulled in). The next possible optimization is to cache literal strings
in the PTH file to avoid the need for the original header sources entirely.
- Right now there is a preprocessor directive to toggle between "lazy" and
"eager" creation of MemBuffers. This is not permanent, and is there in the
short term to just test additional optimizations.
llvm-svn: 61827
- 'Buffer' is now private and must be accessed via 'getBuffer()'.
This paves the way for lazily mapping in source files on demand.
- Added 'getSize()' (which gets the size of the content without
necessarily accessing the MemBuffer) and 'getSizeBytesMapped()'.
- Modifed SourceManager to use these new methods. This reduces the
number of places that actually access the MemBuffer object for a file
to those that actually look at the character data.
These changes result in no performance change for -fsyntax-only on Cocoa.h.
llvm-svn: 61782
to whether the fileid is a 'extern c system header' in addition to whether it
is a system header, most of this is spreading plumbing around. Once we have that,
PPLexerChange bases its "file enter/exit" notifications to PPCallbacks to
base the system header state on FileIDInfo instead of HeaderSearch. Finally,
in Preprocessor::HandleIncludeDirective, mirror logic in GCC: the system headerness
of a file being entered can be set due to the #includer or the #includee.
llvm-svn: 56688
* Move FormatError() from TextDiagnostic up to DiagClient, remove now
empty class TextDiagnostic
* Make DiagClient optional for Diagnostic
This fixes the following problems:
* -html-diags (and probably others) does now output the same set of
warnings as console clang does
* nothing crashes if one forgets to call setHeaderSearch() on
TextDiagnostic
* some code duplication is removed
llvm-svn: 54620
This is a temporary solution to avoid running out of file descriptors (which defaults to 256).
Need to benchmark to understand the speed benefit. If the benefit is small, the simple solution is to avoid memory mapping files. If the benefit is significant, more thought is necessary.
llvm-svn: 48991
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.
llvm-svn: 48402
Argyrios Kyrtzidis