llvm-project/lldb/source/Target/Statistics.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

260 lines
10 KiB
C++
Raw Normal View History

Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
//===-- Statistics.cpp ----------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "lldb/Target/Statistics.h"
#include "lldb/Core/Debugger.h"
#include "lldb/Core/Module.h"
#include "lldb/Symbol/SymbolFile.h"
#include "lldb/Target/Process.h"
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
#include "lldb/Target/Target.h"
#include "lldb/Target/UnixSignals.h"
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
using namespace lldb;
using namespace lldb_private;
using namespace llvm;
static void EmplaceSafeString(llvm::json::Object &obj, llvm::StringRef key,
const std::string &str) {
if (str.empty())
return;
if (LLVM_LIKELY(llvm::json::isUTF8(str)))
obj.try_emplace(key, str);
else
obj.try_emplace(key, llvm::json::fixUTF8(str));
}
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
json::Value StatsSuccessFail::ToJSON() const {
return json::Object{{"successes", successes}, {"failures", failures}};
}
static double elapsed(const StatsTimepoint &start, const StatsTimepoint &end) {
StatsDuration::Duration elapsed =
end.time_since_epoch() - start.time_since_epoch();
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
return elapsed.count();
}
void TargetStats::CollectStats(Target &target) {
m_module_identifiers.clear();
for (ModuleSP module_sp : target.GetImages().Modules())
m_module_identifiers.emplace_back((intptr_t)module_sp.get());
}
json::Value ModuleStats::ToJSON() const {
json::Object module;
EmplaceSafeString(module, "path", path);
EmplaceSafeString(module, "uuid", uuid);
EmplaceSafeString(module, "triple", triple);
module.try_emplace("identifier", identifier);
module.try_emplace("symbolTableParseTime", symtab_parse_time);
module.try_emplace("symbolTableIndexTime", symtab_index_time);
module.try_emplace("symbolTableLoadedFromCache", symtab_loaded_from_cache);
module.try_emplace("symbolTableSavedToCache", symtab_saved_to_cache);
module.try_emplace("debugInfoParseTime", debug_parse_time);
module.try_emplace("debugInfoIndexTime", debug_index_time);
module.try_emplace("debugInfoByteSize", (int64_t)debug_info_size);
module.try_emplace("debugInfoIndexLoadedFromCache",
debug_info_index_loaded_from_cache);
module.try_emplace("debugInfoIndexSavedToCache",
debug_info_index_saved_to_cache);
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-10 10:48:17 +08:00
if (!symfile_path.empty())
module.try_emplace("symbolFilePath", symfile_path);
if (!symfile_modules.empty()) {
json::Array symfile_ids;
for (const auto symfile_id: symfile_modules)
symfile_ids.emplace_back(symfile_id);
module.try_emplace("symbolFileModuleIdentifiers", std::move(symfile_ids));
}
return module;
}
llvm::json::Value ConstStringStats::ToJSON() const {
json::Object obj;
obj.try_emplace<int64_t>("bytesTotal", stats.GetBytesTotal());
obj.try_emplace<int64_t>("bytesUsed", stats.GetBytesUsed());
obj.try_emplace<int64_t>("bytesUnused", stats.GetBytesUnused());
return obj;
}
json::Value TargetStats::ToJSON(Target &target) {
CollectStats(target);
json::Array json_module_uuid_array;
for (auto module_identifier : m_module_identifiers)
json_module_uuid_array.emplace_back(module_identifier);
json::Object target_metrics_json{
{m_expr_eval.name, m_expr_eval.ToJSON()},
{m_frame_var.name, m_frame_var.ToJSON()},
{"moduleIdentifiers", std::move(json_module_uuid_array)}};
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
if (m_launch_or_attach_time && m_first_private_stop_time) {
double elapsed_time =
elapsed(*m_launch_or_attach_time, *m_first_private_stop_time);
target_metrics_json.try_emplace("launchOrAttachTime", elapsed_time);
}
if (m_launch_or_attach_time && m_first_public_stop_time) {
double elapsed_time =
elapsed(*m_launch_or_attach_time, *m_first_public_stop_time);
target_metrics_json.try_emplace("firstStopTime", elapsed_time);
}
target_metrics_json.try_emplace("targetCreateTime",
m_create_time.get().count());
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
json::Array breakpoints_array;
double totalBreakpointResolveTime = 0.0;
// Rport both the normal breakpoint list and the internal breakpoint list.
for (int i = 0; i < 2; ++i) {
BreakpointList &breakpoints = target.GetBreakpointList(i == 1);
std::unique_lock<std::recursive_mutex> lock;
breakpoints.GetListMutex(lock);
size_t num_breakpoints = breakpoints.GetSize();
for (size_t i = 0; i < num_breakpoints; i++) {
Breakpoint *bp = breakpoints.GetBreakpointAtIndex(i).get();
breakpoints_array.push_back(bp->GetStatistics());
totalBreakpointResolveTime += bp->GetResolveTime().count();
}
}
ProcessSP process_sp = target.GetProcessSP();
if (process_sp) {
UnixSignalsSP unix_signals_sp = process_sp->GetUnixSignals();
if (unix_signals_sp)
target_metrics_json.try_emplace("signals",
unix_signals_sp->GetHitCountStatistics());
uint32_t stop_id = process_sp->GetStopID();
target_metrics_json.try_emplace("stopCount", stop_id);
}
target_metrics_json.try_emplace("breakpoints", std::move(breakpoints_array));
target_metrics_json.try_emplace("totalBreakpointResolveTime",
totalBreakpointResolveTime);
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
return target_metrics_json;
}
void TargetStats::SetLaunchOrAttachTime() {
m_launch_or_attach_time = StatsClock::now();
m_first_private_stop_time = llvm::None;
}
void TargetStats::SetFirstPrivateStopTime() {
// Launching and attaching has many paths depending on if synchronous mode
// was used or if we are stopping at the entry point or not. Only set the
// first stop time if it hasn't already been set.
if (!m_first_private_stop_time)
m_first_private_stop_time = StatsClock::now();
}
void TargetStats::SetFirstPublicStopTime() {
// Launching and attaching has many paths depending on if synchronous mode
// was used or if we are stopping at the entry point or not. Only set the
// first stop time if it hasn't already been set.
if (!m_first_public_stop_time)
m_first_public_stop_time = StatsClock::now();
}
bool DebuggerStats::g_collecting_stats = false;
llvm::json::Value DebuggerStats::ReportStatistics(Debugger &debugger,
Target *target) {
json::Array json_targets;
json::Array json_modules;
double symtab_parse_time = 0.0;
double symtab_index_time = 0.0;
double debug_parse_time = 0.0;
double debug_index_time = 0.0;
uint32_t symtabs_loaded = 0;
uint32_t symtabs_saved = 0;
uint32_t debug_index_loaded = 0;
uint32_t debug_index_saved = 0;
uint64_t debug_info_size = 0;
if (target) {
json_targets.emplace_back(target->ReportStatistics());
} else {
for (const auto &target : debugger.GetTargetList().Targets())
json_targets.emplace_back(target->ReportStatistics());
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
}
std::vector<ModuleStats> modules;
std::lock_guard<std::recursive_mutex> guard(
Module::GetAllocationModuleCollectionMutex());
const size_t num_modules = Module::GetNumberAllocatedModules();
for (size_t image_idx = 0; image_idx < num_modules; ++image_idx) {
Module *module = Module::GetAllocatedModuleAtIndex(image_idx);
ModuleStats module_stat;
module_stat.identifier = (intptr_t)module;
module_stat.path = module->GetFileSpec().GetPath();
if (ConstString object_name = module->GetObjectName()) {
module_stat.path.append(1, '(');
module_stat.path.append(object_name.GetStringRef().str());
module_stat.path.append(1, ')');
}
module_stat.uuid = module->GetUUID().GetAsString();
module_stat.triple = module->GetArchitecture().GetTriple().str();
module_stat.symtab_parse_time = module->GetSymtabParseTime().get().count();
module_stat.symtab_index_time = module->GetSymtabIndexTime().get().count();
Symtab *symtab = module->GetSymtab();
if (symtab) {
module_stat.symtab_loaded_from_cache = symtab->GetWasLoadedFromCache();
if (module_stat.symtab_loaded_from_cache)
++symtabs_loaded;
module_stat.symtab_saved_to_cache = symtab->GetWasSavedToCache();
if (module_stat.symtab_saved_to_cache)
++symtabs_saved;
}
SymbolFile *sym_file = module->GetSymbolFile();
if (sym_file) {
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-10 10:48:17 +08:00
if (sym_file->GetObjectFile() != module->GetObjectFile())
module_stat.symfile_path =
sym_file->GetObjectFile()->GetFileSpec().GetPath();
module_stat.debug_index_time = sym_file->GetDebugInfoIndexTime().count();
module_stat.debug_parse_time = sym_file->GetDebugInfoParseTime().count();
module_stat.debug_info_size = sym_file->GetDebugInfoSize();
module_stat.debug_info_index_loaded_from_cache =
sym_file->GetDebugInfoIndexWasLoadedFromCache();
if (module_stat.debug_info_index_loaded_from_cache)
++debug_index_loaded;
module_stat.debug_info_index_saved_to_cache =
sym_file->GetDebugInfoIndexWasSavedToCache();
if (module_stat.debug_info_index_saved_to_cache)
++debug_index_saved;
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-10 10:48:17 +08:00
ModuleList symbol_modules = sym_file->GetDebugInfoModules();
for (const auto &symbol_module: symbol_modules.Modules())
module_stat.symfile_modules.push_back((intptr_t)symbol_module.get());
}
symtab_parse_time += module_stat.symtab_parse_time;
symtab_index_time += module_stat.symtab_index_time;
debug_parse_time += module_stat.debug_parse_time;
debug_index_time += module_stat.debug_index_time;
debug_info_size += module_stat.debug_info_size;
json_modules.emplace_back(module_stat.ToJSON());
}
ConstStringStats const_string_stats;
json::Object json_memory{
{"strings", const_string_stats.ToJSON()},
};
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
json::Object global_stats{
{"targets", std::move(json_targets)},
{"modules", std::move(json_modules)},
{"memory", std::move(json_memory)},
{"totalSymbolTableParseTime", symtab_parse_time},
{"totalSymbolTableIndexTime", symtab_index_time},
{"totalSymbolTablesLoadedFromCache", symtabs_loaded},
{"totalSymbolTablesSavedToCache", symtabs_saved},
{"totalDebugInfoParseTime", debug_parse_time},
{"totalDebugInfoIndexTime", debug_index_time},
{"totalDebugInfoIndexLoadedFromCache", debug_index_loaded},
{"totalDebugInfoIndexSavedToCache", debug_index_saved},
{"totalDebugInfoByteSize", debug_info_size},
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-21 05:49:09 +08:00
};
return std::move(global_stats);
}