We want to use `DexDeclareFile` to specify paths relative to a project root
directory. The option `--source-root-dir`, prior to this patch, causes dexter
to strip the path prefix from commands before passing them to a debugger, and
appends the prefix to file paths returned from a debugger. This patch changes
the behviour of `--source-root-dir`. Relative paths in commands, made possible
with `DexDeclareFile(relative/path)`, are appended to the `--source-root-dir`
directory.
A new option, `--debugger-use-relative-paths`, can be used alongside
`--source-root-dir` to reproduce the old behaviour: all paths passed to the
debugger will be made relative to `--source-root-dir`.
I've added a regression test source_root_dir.dex for this new behaviour, and
modified the existing `--source-root-dir` regression and unit tests to use
`--debugger-use-relative-paths`.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D100307
DexDeclareFile allows test producers to write test files with .dex extensions
that contain pure dexter commands.
.dex file commands do not need to be commented out like they do when written
inline within test source files.
DexDeclareFile commands are declarative in behaviour, they state that any
Dexter command seen from this point on will have its path attribute set to the
path declared in the DexDeclareFile command.
Differential Revision: https://reviews.llvm.org/D99651
The DexLimitSteps command leading breakpoint will be deleted after triggering
'hit_count' number of times if the argument is provided.
All the lit tests pass on linux (with lldb), and I've tested this on windows
(with vs2017) manually as the lit tests for DexLimitSteps are currently
unsupported on windows.
Reviewed By: StephenTozer, chrisjackson
Differential Revision: https://reviews.llvm.org/D101527
This patch changes how line labels are resolved in order to enable
label-relative offsets to be used in commands. This is a breaking change in
dexter. Instead of using label references directly as argument values, labels
will instead be referenced through a function `ref(str)`.
// No way to use offsets currently.
Currently: DexExpectWatchValue('x', '1', on_line='labled_line')
Patched: DexExpectWatchValue('x', '1', on_line=ref('labled_line'))
Patched: DexExpectWatchValue('x', '1', on_line=ref('labled_line') + 3)
A dexter command is "parsed" by finding the whole command and sending it off to
`eval`. This change adds a function called `ref` to the `eval` globals map that
simply looks up the name and returns an int. If the line name hasn't been
defined, or a name is defined more than once, an error is reported (see
err_bad_label_ref.cpp and err_duplicate_label.cpp). Label offsets can be
achieved by simply writing the desired expression.
The rationale behind removing the existing label referencing mechanic is for
consistency and to simplify the code required to make labels work.
I've separated the update to llvm's dexter tests into another patch for ease of
review here (D101148). Here is a small python script which can be used to
update tests to use the new syntax:
https://gist.github.com/OCHyams/8255efe7757cac266440ed2ba55f1442
If it helps anyone using dexter on downstream tests we can come up with a
deprecation plan instead out outright removing the existing syntax.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D101147
Currently the DexLimitSteps command requires at least one condition. This patch
lets users elide the condition to specify that the breakpoint range should
always be activated when the leading line is stepped on. This patch also
updates the terminology used in the `ConditionalController` class from the
terms 'conditional' and 'unconditional' to 'leading' and 'trailing' when
referring to the breakpoints in the DexLimitSteps range because the leading
breakpoint can now be unconditional.
Reviewed By: chrisjackson
Differential Revision: https://reviews.llvm.org/D101438
Remove the `ConditionalController._conditional_met` method. This was missed in
the recent ConditionalController refactor (D98699). We don't need to check that
the conditions for a conditional breakpoint have been met because
`DebuggerBase.get_triggered_breakpoint_ids` returns the set of ids for
breakpoints which have been triggered.
To get the "triggered breakpoints" from lldb we use `GetStopReasonDataCount`
and `GetStopReasonDataAtIndex`. It seems that these functions count all
breakpoints associated with the location which lldb has stopped at, regardless
of their condition. i.e. Even if we have two breakpoints at the same source
location that have mutually exclusive conditions, both will be found this way
when either condition is true. To get around this, we store a map of breakpoint
{id: condition} `_breakpoint_conditions` and evaluate the conditions of the
triggered breakpoints to filter the set down to those which are unconditional
or have a condition which evaluates to true.
Essentially we are just moving the condition double check from a general
debugger controller into the lldb specific wrapper. This tidy up will help make
upcoming patches simpler.
Reviewed By: chrisjackson
Differential Revision: https://reviews.llvm.org/D101431
Add optional keyword argument 'on_line' to DexLabel to label the specifed line
instead of the line the command is found on.
This will be helpful when used alongside DexDeclareFile (D99651).
Reviewed By: TWeaver
Differential Revision: https://reviews.llvm.org/D101055
Using a DexLimitSteps command forces dexter to use the ConditionalController
debugger controller. At each breakpoint the ConditionalController needs to
understand which one has been hit. Prior to this patch, upon hitting a
breakpoint, dexter used the current source location to look up which requested
breakpoint had been hit.
A breakpoint may not get set at the exact location that the user (dexter)
requests. For example, if the requested breakpoint location doesn't exist
in the line table then then debuggers will (usually, AFAICT) set the breakpoint
at the next available valid breakpoint location.
This meant that, occasionally in unoptimised programs and frequently in
optimised programs, the ConditionalController was failing to determine which
breakpoint had been hit.
This is the fix:
Change the DebuggerBase breakpoint interface to use opaque breakpoint ids
instead of using source location to identify breakpoints, and update the
ConditionalController to track breakpoints instead of locations.
These now return a breakpoint id:
add_breakpoint(self, file_, line)
_add_breakpoint(self, file_, line)
add_conditional_breakpoint(self, file_, line, condition)
_add_conditional_breakpoint(self, file_, line, condition)
Replace:
delete_conditional_breakpoint(self, file_, line, condition)
_delete_conditional_breakpoint(self, file_, line, condition)
with:
delete_breakpoint(self, id)
Add:
get_triggered_breakpoint_ids(self)
A breakpoint id is guaranteed to be unique for each requested breakpoint, even
for duplicate breakpoint requests. Identifying breakpoints like this, instead
of by location, removes the possibility of mixing up requested and bound
breakpoints.
This closely matches the LLDB debugger interface so little work was required in
LLDB.py, but some extra bookkeeping is required in VisualStudio.py to maintain
the new breakpoint id semantics. No implementation work has been done in
dbgeng.py as DexLimitSteps doesn't seem to support dbgeng at the moment.
Testing
Added:
dexter/feature_tests/commands/perfect/limit_steps/limit_steps_line_mismatch.cpp
There were no unexpected failures running the full debuginfo-tests suite.
The regression tests use dbgeng on windows by default, and as mentioned above
dbgeng isn't supported yet, so I have also manually tested (i.e. without lit)
that this specific test works as expected with clang and Visual Studio 2017 on
Windows.
Reviewed By: TWeaver
Differential Revision: https://reviews.llvm.org/D98699
The test optnone-simple-functions.cpp added in D97668 fails on macOS.
os.path.exists raises an exception because we pass it None. Guard against this.
Related revision: https://reviews.llvm.org/D97668
This method of the dbgeng debugger driver used to just "pass", as setting
dbgeng free running still leads to numerous errors. This wasn't a problem
in the past because, as it turns out, nothing called the go method.
However, a recent refactor uses it.
Rather than launch dbgeng free running, instead have it single step one
step forwards. This is slow, but it makes progress, where previously we
weren't.
Differential Revision: https://reviews.llvm.org/D91737
Adds visual studio debugger support to dexter via option --debugger vs2019
Differential Revision: https://reviews.llvm.org/D89803
Author: Nabeel Omer <nabeel.omer@sony.com>
Summary:
This allows to run dexter tests with separately compiled
binaries that are specified via --binary if the source file
location changed between compilation and dexter test run.
Reviewers: TWeaver, jmorse, probinson, #debug-info
Reviewed By: jmorse
Subscribers: #debug-info, cmtice, llvm-commits
Tags: #llvm, #debug-info
Differential Revision: https://reviews.llvm.org/D81319
When passing a test path, if the path points directly at a file, then
normcase would not be called on path.
This would change the expected lower case drive path, on windows, to be
uppercase. This patch simply calls normcase on the test path at the earliest
point possible to avoid this issue.
Reviewers: djtodoro, jmorse
Differential Revision: https://reviews.llvm.org/D78633
Fixes a mismatch in expected arguments passed to run_debugger_subprocess
Fix for:
https://reviews.llvm.org/D76926
rG9cf9710bb0d61cb5c27c6e780af6a182cb162bfb
Add DebuggerControllerBase and DefaultController to Dexter
implements a new architecture that supports new and novel ways of running
a debugger under dexter.
Current implementation adds the original default behaviour via the new
architecture via the DefaultController, this should have NFC.
Reviewers: Orlando
Differential Revision: https://reviews.llvm.org/D76926
Summary: It's possible for an instance of the visual studio debugger
to return a NoneType line number location when stepping during a
debugging session.
This patches teaches DexTer how to handle this particular case without
crashing out.
Reviewers: Orlando
Differential revision: https://reviews.llvm.org/D75992
On Windows, an error running the debugger typically leaves a process
hanging around in the working directory. When Dexter exits, it can't then
delete the working directory and produces an exception, masking the problem
in the debugger. (This can be worked around by specifying --save-temps).
Rather than hard-erroring, print a warning when we can't delete the working
directory instead.
It'd be much better to improve our error handling, and make the
WorkingDirectory class aware that something's wrong when it enters exit.
However, this is something that's going to mask genuine errors and make
everyones lives harder right now, so I think this non-ideal fix is
important to get in first.
Differential Revision: https://reviews.llvm.org/D74548
These are some minor things that I've run into on Windows, largely in
error handling paths:
* Giving --lldb-executable on Windows triggers a "useless option" code
path, which touches an attribute that only exists in the
list_debuggers tool. Switch this to use hasattr, which will work in
all subtools.
* We were over-decoding some text reporting errors, but only in an
exception path
* The path to lldb on Windows needs to be quoted (even though dexter
isn't making use of it).
Differential Revision: https://reviews.llvm.org/D74546
When writing the Windows dbgeng driver for Dexter, I couldn't work out why it
would either launch a process and leave it free running, or if I started the
process suspended, never do anything with it. The result was a hack to create
and attach processes manually. This has been flaking out on Reids Windows
buildbot, and clearly wasn't a good solution.
Digging into this, it turns out that the "normal" cdb / windbg behaviour of
breaking whenever we attach to a process is not the default: it has to be
explicitly requested from the debug engine. This patch does so (by setting
DEBUG_ENGOPT_INITIAL_BREAK in the engine options), after which we can simply
call "CreateProcessAndAttach2" and everything automagically works.
No test for this behaviour: everything was just broken before.
Differential Revision: https://reviews.llvm.org/D74409
First, add LLD as a dependency on Windows. The windows batch scripts
pass -fuse-ld=lld, so they need it.
Second, decode builder stdout/stderr even if the command fails.
Otherwise it gets printed as b'line 1\n\rline 2\n\r'.
Last, make the batch script one line less noisy. We might want to try to
do more here, though. It would be nice if we could get as close to
possible as lit, where you can literally copy & paste the failing
command to re-run it.
With the two changes above, now the feature tests that use clang++.bat
pass for me. The clang-cl_vs2015 ones still fail, and I'll fix them
separately.
Reviewers: jmorse
Differential Revision: https://reviews.llvm.org/D69725
When running a program, Dexter single steps if it's in one of the source
files under test, or free-runs if it isn't. Handle the circumstance where
the current source file simply isn't known.
I baked the "test" subcommand into the %dexter substituion, as that's
what all of the dexter tests use. However I forgot about the internal
tests for whether dexters features are working. Install a %dexter_base
command to allow those tests to call dexter.py directly, and un-xfail
the tests on darwin.
Update too the list of paths the unittests shouldn't try and cover, as it
tries to load dbgeng on unix machines. Ideally we wouldn't be using this
method of test discovery in the future.
This reverts commit cb935f3456.
Discussion in D68708 advises that green dragon is being briskly
refurbished, and it's good to have this patch up testing it.
Dexter (Debug Experience Tester) is a test-driver for our debug info
integration tests, reading a set of debug experience expectations and
comparing them with the actual behaviour of a program under a debugger.
More about Dexter can be found in the RFC:
http://lists.llvm.org/pipermail/llvm-dev/2019-October/135773.html
and the phab review in D68708. Not all the debuginfo tests have been
transformed into Dexter tests, and we look forwards to doing that
incrementally.
This commit mostly aims to flush out buildbots that are running
debuginfo-tests but don't have python 3 installed, possibly
green-dragon and some windows bots.
OCHyams