The sketch test case writes avg+stddev for all its metrics:
<key>fetch-frames</key>
<dict>
<key>description</key>
<string>time to dump backtrace for every frame in every thread</string>
<key>stddev</key>
<real>0.006270938361432314</real>
<key>value</key>
<real>0.011568079851851851</real>
</dict>
llvm-svn: 179550
Moving over to source-regex might be a good idea, but it’s not what we really want to test. Ideally, we would set a regex breakpoint to find the right line, then delete it and set one by file and line with the right line info from before.
llvm-svn: 179246
LLDB now can use a single dash for all long options for all commands form the command line and from the command interpreter. This involved just switching all calls from getopt_long() to getopt_long_only().
llvm-svn: 178789
To hook it up to individual test cases:
- define GetLongOptions() in your test case class to return something other than NULL (hopefully an array of options :-)
- implement ParseOption() to check for the short option char and do the right thing - return true at the end if you want more options to come your way or false if you don’t
- make sure that your Setup() call takes int& and char**& so that optind post-processing can happen - and call TestCase::Setup from your setup
llvm-svn: 178482
- Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings:
Results::Result::Dictionary
Results::Result::Array
Results::Result::Unsigned
Results::Result::Double
Results::Result::String
- Gauge<T> can now write themselves out via a templatized write to results function:
template <class T>
Results::ResultSP GetResult (const char *description, T value);
- There are four specializations of this so far:
template <>
Results::ResultSP GetResult (const char *description, double value);
template <>
Results::ResultSP GetResult (const char *description, uint64_t value);
template <>
Results::ResultSP GetResult (const char *description, std::string value);
template <>
Results::ResultSP GetResult (const char *description, MemoryStats value);
- Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over)
- Fixed other test cases to build correctly and use the new classes
llvm-svn: 177696
This is especially useful to take measurements that span multiple test steps, or where you need to have different operations fall under the same measurement
An example of use is in the formatters perf test case
llvm-svn: 177597
- TestCase.m_thread is now filled in with the first thread that has a valid
stop reason. This eliminates the need for the SelectMyThread() functions.
- The first thread that stops for a reason is also set as the selected thread
in the process in case any command line commands are run.
- Changed launch over to take a SBLaunchInfo parameter so that the launch
function doesn't keep getting new arguments as they are needed.
- TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to
indicate success of setup and launch.
- ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread)
- ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread)
llvm-svn: 177376
- don't use preprocessor macros
- use switch statements
- don't put anything in the lldb namespace, use "lldb_perf" namespace.
- Pass the action struct into each TestStep() for each step fill in
- Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros
llvm-svn: 177332
This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance
This is separate from the LLDB testsuite in test/ in that:
a) this uses C++ instead of Python to avoid measures being affected by SWIG
b) this is in very early development and needs lots of tweaking before it can be considered functionally complete
c) this is not meant to test correctness but to help catch performance regressions
There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior.
The resulting output is a PLIST much like the following:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
<dict>
<key>fetch-frames</key>
<real>0.13161715522222225</real>
</dict>
<dict>
<key>file-line-bkpt</key>
<real>0.029111678750000002</real>
</dict>
<dict>
<key>fetch-modules</key>
<real>0.00026376766666666668</real>
</dict>
<dict>
<key>fetch-vars</key>
<real>0.17820429311111111</real>
</dict>
<dict>
<key>run-expr</key>
<real>0.029676525769230768</real>
</dict>
</array>
</plist>
Areas for improvement:
- code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!)
- more metrics and test cases
- better error checking
This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool.
llvm-svn: 176715
Enrico Granata