maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/lldb/tools/lldb-perf/lib/TestCase.cpp

322 lines
10 KiB
C++
Raw Normal View History

Add correct file headers to all source files. llvm-svn: 177625
2013-03-21 11:39:51 +08:00
//===-- TestCase.cpp --------------------------------------------*- C++ -*-===//
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
//
Add correct file headers to all source files. llvm-svn: 177625
2013-03-21 11:39:51 +08:00
// The LLVM Compiler Infrastructure
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
//
Add correct file headers to all source files. llvm-svn: 177625
2013-03-21 11:39:51 +08:00
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
//
Add correct file headers to all source files. llvm-svn: 177625
2013-03-21 11:39:51 +08:00
//===----------------------------------------------------------------------===//
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
#include "TestCase.h"
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
#include "Results.h"
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
#include "Xcode.h"
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
using namespace lldb_perf;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
TestCase::TestCase () :
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
m_debugger(),
m_target(),
m_process(),
m_thread(),
m_listener(),
Making the test step count a member variable so that it can be accessed easily llvm-svn: 177594
2013-03-21 06:42:34 +08:00
m_verbose(false),
m_step(0)
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
{
<rdar://problem/13228487> A test case for the performance of some LLDB formatters Changes and improvements to the testing infrastructure itself llvm-svn: 177100
2013-03-15 03:00:42 +08:00
SBDebugger::Initialize();
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
SBHostOS::ThreadCreated ("<lldb-tester.app.main>");
m_debugger = SBDebugger::Create(false);
m_listener = m_debugger.GetListener();
}
Integrating option parsing in TestCase for added convenience To hook it up to individual test cases: - define GetLongOptions() in your test case class to return something other than NULL (hopefully an array of options :-) - implement ParseOption() to check for the short option char and do the right thing - return true at the end if you want more options to come your way or false if you don’t - make sure that your Setup() call takes int& and char**& so that optind post-processing can happen - and call TestCase::Setup from your setup llvm-svn: 178482
2013-04-02 02:02:25 +08:00
static std::string
GetShortOptionString (struct option *long_options)
{
std::string option_string;
for (int i = 0; long_options[i].name != NULL; ++i)
{
if (long_options[i].flag == NULL)
{
option_string.push_back ((char) long_options[i].val);
switch (long_options[i].has_arg)
{
default:
case no_argument:
break;
case required_argument:
option_string.push_back (':');
break;
case optional_argument:
option_string.append (2, ':');
break;
}
}
}
return option_string;
}
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
bool
Integrating option parsing in TestCase for added convenience To hook it up to individual test cases: - define GetLongOptions() in your test case class to return something other than NULL (hopefully an array of options :-) - implement ParseOption() to check for the short option char and do the right thing - return true at the end if you want more options to come your way or false if you don’t - make sure that your Setup() call takes int& and char**& so that optind post-processing can happen - and call TestCase::Setup from your setup llvm-svn: 178482
2013-04-02 02:02:25 +08:00
TestCase::Setup (int& argc, const char**& argv)
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
{
Integrating option parsing in TestCase for added convenience To hook it up to individual test cases: - define GetLongOptions() in your test case class to return something other than NULL (hopefully an array of options :-) - implement ParseOption() to check for the short option char and do the right thing - return true at the end if you want more options to come your way or false if you don’t - make sure that your Setup() call takes int& and char**& so that optind post-processing can happen - and call TestCase::Setup from your setup llvm-svn: 178482
2013-04-02 02:02:25 +08:00
bool done = false;
struct option* long_options = GetLongOptions();
if (long_options)
{
std::string short_option_string (GetShortOptionString(long_options));
#if __GLIBC__
optind = 0;
#else
optreset = 1;
optind = 1;
#endif
while (!done)
{
int long_options_index = -1;
const int short_option = ::getopt_long_only (argc,
const_cast<char **>(argv),
short_option_string.c_str(),
long_options,
&long_options_index);
switch (short_option)
{
case 0:
// Already handled
break;
case -1:
done = true;
break;
default:
done = !ParseOption(short_option, optarg);
break;
}
}
argc -= optind;
argv += optind;
}
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
return false;
}
<rdar://problem/13228487> A test case for the performance of some LLDB formatters Changes and improvements to the testing infrastructure itself llvm-svn: 177100
2013-03-15 03:00:42 +08:00
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
bool
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
TestCase::Launch (lldb::SBLaunchInfo &launch_info)
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
{
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
lldb::SBError error;
m_process = m_target.Launch (launch_info, error);
if (!error.Success())
fprintf (stderr, "error: %s\n", error.GetCString());
if (m_process.IsValid())
{
m_process.GetBroadcaster().AddListener(m_listener, SBProcess::eBroadcastBitStateChanged | SBProcess::eBroadcastBitInterrupt);
return true;
}
return false;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Misc fixes: - make an overload of Launch() that takes an init list of const char* if all you need to tweak in the launch info are the command-line arguments - make Run() return an int that you can use as an exit-code - make dynamic values work properly when recursing in FetchVariables() - make the po output more obvious in verbose mode llvm-svn: 178578
2013-04-03 05:31:18 +08:00
bool
TestCase::Launch (std::initializer_list<const char*> args)
{
std::vector<const char*> args_vect(args);
args_vect.push_back(NULL);
lldb::SBLaunchInfo launch_info((const char**)&args_vect[0]);
return Launch(launch_info);
}
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
void
TestCase::SetVerbose (bool b)
{
m_verbose = b;
}
bool
TestCase::GetVerbose ()
{
return m_verbose;
}
void
TestCase::Loop ()
{
while (true)
{
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
bool call_test_step = false;
if (m_process.IsValid())
{
SBEvent evt;
m_listener.WaitForEvent (UINT32_MAX, evt);
StateType state = SBProcess::GetStateFromEvent (evt);
if (m_verbose)
printf("event = %s\n",SBDebugger::StateAsCString(state));
if (SBProcess::GetRestartedFromEvent(evt))
continue;
switch (state)
{
case eStateInvalid:
case eStateDetached:
case eStateCrashed:
case eStateUnloaded:
break;
case eStateExited:
return;
case eStateConnected:
case eStateAttaching:
case eStateLaunching:
case eStateRunning:
case eStateStepping:
continue;
case eStateStopped:
case eStateSuspended:
{
call_test_step = true;
bool fatal = false;
bool selected_thread = false;
for (auto thread_index = 0; thread_index < m_process.GetNumThreads(); thread_index++)
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
{
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
SBThread thread(m_process.GetThreadAtIndex(thread_index));
SBFrame frame(thread.GetFrameAtIndex(0));
bool select_thread = false;
StopReason stop_reason = thread.GetStopReason();
if (m_verbose) printf("tid = 0x%llx pc = 0x%llx ",thread.GetThreadID(),frame.GetPC());
switch (stop_reason)
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
{
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
case eStopReasonNone:
if (m_verbose)
printf("none\n");
break;
case eStopReasonTrace:
select_thread = true;
if (m_verbose)
printf("trace\n");
break;
case eStopReasonPlanComplete:
select_thread = true;
if (m_verbose)
printf("plan complete\n");
break;
case eStopReasonThreadExiting:
if (m_verbose)
printf("thread exiting\n");
break;
case eStopReasonExec:
if (m_verbose)
printf("exec\n");
break;
case eStopReasonInvalid:
if (m_verbose)
printf("invalid\n");
break;
case eStopReasonException:
select_thread = true;
if (m_verbose)
printf("exception\n");
fatal = true;
break;
case eStopReasonBreakpoint:
select_thread = true;
if (m_verbose)
printf("breakpoint id = %lld.%lld\n",thread.GetStopReasonDataAtIndex(0),thread.GetStopReasonDataAtIndex(1));
break;
case eStopReasonWatchpoint:
select_thread = true;
if (m_verbose)
printf("watchpoint id = %lld\n",thread.GetStopReasonDataAtIndex(0));
break;
case eStopReasonSignal:
select_thread = true;
if (m_verbose)
printf("signal %d\n",(int)thread.GetStopReasonDataAtIndex(0));
break;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
if (select_thread && !selected_thread)
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
{
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
m_thread = thread;
selected_thread = m_process.SetSelectedThread(thread);
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
}
if (fatal)
{
if (m_verbose) Xcode::RunCommand(m_debugger,"bt all",true);
exit(1);
}
}
break;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
}
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
else
{
call_test_step = true;
}
if (call_test_step)
{
if (m_verbose)
printf("RUNNING STEP %d\n",m_step);
ActionWanted action;
TestStep(m_step, action);
m_step++;
SBError err;
switch (action.type)
{
case ActionWanted::Type::eContinue:
err = m_process.Continue();
break;
case ActionWanted::Type::eStepOut:
if (action.thread.IsValid() == false)
{
if (m_verbose)
{
Xcode::RunCommand(m_debugger,"bt all",true);
printf("error: invalid thread for step out on step %d\n", m_step);
}
exit(501);
}
m_process.SetSelectedThread(action.thread);
action.thread.StepOut();
break;
case ActionWanted::Type::eStepOver:
if (action.thread.IsValid() == false)
{
if (m_verbose)
{
Xcode::RunCommand(m_debugger,"bt all",true);
printf("error: invalid thread for step over %d\n",m_step);
}
exit(500);
}
m_process.SetSelectedThread(action.thread);
action.thread.StepOver();
break;
case ActionWanted::Type::eKill:
if (m_verbose)
printf("kill\n");
m_process.Kill();
return;
}
}
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
Making the test step count a member variable so that it can be accessed easily llvm-svn: 177594
2013-03-21 06:42:34 +08:00
if (GetVerbose()) printf("I am gonna die at step %d\n",m_step);
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Misc fixes: - make an overload of Launch() that takes an init list of const char* if all you need to tweak in the launch info are the command-line arguments - make Run() return an int that you can use as an exit-code - make dynamic values work properly when recursing in FetchVariables() - make the po output more obvious in verbose mode llvm-svn: 178578
2013-04-03 05:31:18 +08:00
int
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
TestCase::Run (TestCase& test, int argc, const char** argv)
{
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
if (test.Setup(argc, argv))
{
test.Loop();
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00
Results results;
test.WriteResults(results);
Misc fixes: - make an overload of Launch() that takes an init list of const char* if all you need to tweak in the launch info are the command-line arguments - make Run() return an int that you can use as an exit-code - make dynamic values work properly when recursing in FetchVariables() - make the po output more obvious in verbose mode llvm-svn: 178578
2013-04-03 05:31:18 +08:00
return RUN_SUCCESS;
More cleanup on the lldb-perf code: - TestCase.m_thread is now filled in with the first thread that has a valid stop reason. This eliminates the need for the SelectMyThread() functions. - The first thread that stops for a reason is also set as the selected thread in the process in case any command line commands are run. - Changed launch over to take a SBLaunchInfo parameter so that the launch function doesn't keep getting new arguments as they are needed. - TestCase::Setup() and TestCase::Launch(SBLaunchInfo) now return bool to indicate success of setup and launch. - ActionWanted::Next(SBThread) was renamed to ActionWanted::StepOver(SBThread) - ActionWanted::Finish(SBThread) was renamed to ActionWanted::StepOut(SBThread) llvm-svn: 177376
2013-03-19 12:41:22 +08:00
}
Misc fixes: - make an overload of Launch() that takes an init list of const char* if all you need to tweak in the launch info are the command-line arguments - make Run() return an int that you can use as an exit-code - make dynamic values work properly when recursing in FetchVariables() - make the po output more obvious in verbose mode llvm-svn: 178578
2013-04-03 05:31:18 +08:00
else
return RUN_SETUP_ERROR;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
}
Much more cleanup on the performance testing infrastructure: - Added new abtract Results class to keep CoreFoundation out of the tests. There are many subclasses for different settings: Results::Result::Dictionary Results::Result::Array Results::Result::Unsigned Results::Result::Double Results::Result::String - Gauge<T> can now write themselves out via a templatized write to results function: template <class T> Results::ResultSP GetResult (const char *description, T value); - There are four specializations of this so far: template <> Results::ResultSP GetResult (const char *description, double value); template <> Results::ResultSP GetResult (const char *description, uint64_t value); template <> Results::ResultSP GetResult (const char *description, std::string value); template <> Results::ResultSP GetResult (const char *description, MemoryStats value); - Don't emit the virtual memory reading from the task info call as it really doesn't mean much as it includes way too much (shared cache + other stuff we don't have control over) - Fixed other test cases to build correctly and use the new classes llvm-svn: 177696
2013-03-22 10:31:35 +08:00