maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/lldb/tools/lldb-perf/lib/TestCase.cpp

180 lines
5.3 KiB
C++
Raw Normal View History

Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
//
// TestCase.cpp
// PerfTestDriver
//
// Created by Enrico Granata on 3/7/13.
// Copyright (c) 2013 Apple Inc. All rights reserved.
//
#include "TestCase.h"
#include "Xcode.h"
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
using namespace lldb_perf;
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
TestCase::TestCase () :
m_debugger(),
m_target(),
m_process(),
m_thread(),
m_listener(),
m_verbose(false)
{
<rdar://problem/13228487> A test case for the performance of some LLDB formatters Changes and improvements to the testing infrastructure itself llvm-svn: 177100
2013-03-15 03:00:42 +08:00
SBDebugger::Initialize();
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
SBHostOS::ThreadCreated ("<lldb-tester.app.main>");
m_debugger = SBDebugger::Create(false);
m_listener = m_debugger.GetListener();
}
<rdar://problem/13228487> A test case for the performance of some LLDB formatters Changes and improvements to the testing infrastructure itself llvm-svn: 177100
2013-03-15 03:00:42 +08:00
void
TestCase::Setup (int argc, const char** argv)
{}
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
bool
TestCase::Launch (const char** args, const char* cwd)
{
m_process = m_target.LaunchSimple(args,NULL,cwd);
m_process.GetBroadcaster().AddListener(m_listener, SBProcess::eBroadcastBitStateChanged | SBProcess::eBroadcastBitInterrupt);
return m_process.IsValid ();
}
void
TestCase::SetVerbose (bool b)
{
m_verbose = b;
}
bool
TestCase::GetVerbose ()
{
return m_verbose;
}
void
TestCase::Loop ()
{
int step = 0;
SBEvent evt;
while (true)
{
m_listener.WaitForEvent (UINT32_MAX,evt);
StateType state = SBProcess::GetStateFromEvent (evt);
if (m_verbose)
printf("event = %s\n",SBDebugger::StateAsCString(state));
if (SBProcess::GetRestartedFromEvent(evt))
continue;
switch (state)
{
case eStateInvalid:
case eStateDetached:
case eStateCrashed:
case eStateUnloaded:
break;
case eStateExited:
return;
case eStateConnected:
case eStateAttaching:
case eStateLaunching:
case eStateRunning:
case eStateStepping:
continue;
case eStateStopped:
case eStateSuspended:
{
bool fatal = false;
for (auto thread_index = 0; thread_index < m_process.GetNumThreads(); thread_index++)
{
SBThread thread(m_process.GetThreadAtIndex(thread_index));
SBFrame frame(thread.GetFrameAtIndex(0));
StopReason stop_reason = thread.GetStopReason();
if (m_verbose) printf("tid = 0x%llx pc = 0x%llx ",thread.GetThreadID(),frame.GetPC());
switch (stop_reason)
{
case eStopReasonNone:
if (m_verbose) printf("none\n");
break;
case eStopReasonTrace:
if (m_verbose) printf("trace\n");
break;
case eStopReasonPlanComplete:
if (m_verbose) printf("plan complete\n");
break;
case eStopReasonThreadExiting:
if (m_verbose) printf("thread exiting\n");
break;
case eStopReasonExec:
if (m_verbose) printf("exec\n");
break;
case eStopReasonInvalid:
if (m_verbose) printf("invalid\n");
break;
case eStopReasonException:
if (m_verbose) printf("exception\n");
fatal = true;
break;
case eStopReasonBreakpoint:
if (m_verbose) printf("breakpoint id = %lld.%lld\n",thread.GetStopReasonDataAtIndex(0),thread.GetStopReasonDataAtIndex(1));
break;
case eStopReasonWatchpoint:
if (m_verbose) printf("watchpoint id = %lld\n",thread.GetStopReasonDataAtIndex(0));
break;
case eStopReasonSignal:
if (m_verbose) printf("signal %d\n",(int)thread.GetStopReasonDataAtIndex(0));
break;
}
}
if (fatal)
{
if (m_verbose) Xcode::RunCommand(m_debugger,"bt all",true);
exit(1);
}
if (m_verbose)
printf("RUNNING STEP %d\n",step);
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
ActionWanted action;
TestStep(step, action);
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
step++;
switch (action.type)
{
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
case ActionWanted::Type::eContinue:
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
m_debugger.HandleCommand("continue");
break;
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
case ActionWanted::Type::eFinish:
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
if (action.thread.IsValid() == false)
{
if (m_verbose) Xcode::RunCommand(m_debugger,"bt all",true);
if (m_verbose) printf("[finish invalid] I am gonna die at step %d\n",step);
exit(501);
}
m_process.SetSelectedThread(action.thread);
m_debugger.HandleCommand("finish");
break;
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
case ActionWanted::Type::eNext:
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
if (action.thread.IsValid() == false)
{
if (m_verbose) Xcode::RunCommand(m_debugger,"bt all",true);
if (m_verbose) printf("[next invalid] I am gonna die at step %d\n",step);
exit(500);
}
m_process.SetSelectedThread(action.thread);
m_debugger.HandleCommand("next");
break;
Code cleanup: - don't use preprocessor macros - use switch statements - don't put anything in the lldb namespace, use "lldb_perf" namespace. - Pass the action struct into each TestStep() for each step fill in - Modify the ActionWanted class to have accessors to make the continue, next, finish, kill instead of using preproc macros llvm-svn: 177332
2013-03-19 06:34:00 +08:00
case ActionWanted::Type::eKill:
Initial checkin of a new project: LLDB Performance Testing Infrastructure This is a very basic implementation of a library that easily allows to drive LLDB.framework to write test cases for performance This is separate from the LLDB testsuite in test/ in that: a) this uses C++ instead of Python to avoid measures being affected by SWIG b) this is in very early development and needs lots of tweaking before it can be considered functionally complete c) this is not meant to test correctness but to help catch performance regressions There is a sample application built against the library (in darwin/sketch) that uses the famous sample app Sketch as an inferior to measure certain basic parameters of LLDB's behavior. The resulting output is a PLIST much like the following: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <dict> <key>fetch-frames</key> <real>0.13161715522222225</real> </dict> <dict> <key>file-line-bkpt</key> <real>0.029111678750000002</real> </dict> <dict> <key>fetch-modules</key> <real>0.00026376766666666668</real> </dict> <dict> <key>fetch-vars</key> <real>0.17820429311111111</real> </dict> <dict> <key>run-expr</key> <real>0.029676525769230768</real> </dict> </array> </plist> Areas for improvement: - code cleanups (I will be out of the office for a couple days this coming week, but please keep ideas coming!) - more metrics and test cases - better error checking This toolkit also comprises a simple event-loop-driven controller for LLDB, similar yet much simpler to what the Driver does to implement the lldb command-line tool. llvm-svn: 176715
2013-03-09 04:29:13 +08:00
if (m_verbose) printf("I want to die\n");
m_process.Kill();
return;
}
}
}
}
if (GetVerbose()) printf("I am gonna die at step %d\n",step);
}
void
TestCase::Run (TestCase& test, int argc, const char** argv)
{
test.Setup(argc, argv);
test.Loop();
test.Results();
}