maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

[lldbsuite] Remove pre_kill_hook package 

This package was only used by dosep.py which has since been removed.
This commit is contained in:
Jonas Devlieghere 2019-10-29 16:29:12 -07:00
parent 5cc2e0651f
commit 8a82000e48
7 changed files with 0 additions and 418 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

55
lldb/packages/Python/lldbsuite/pre_kill_hook/README.md
View File

@ -1,55 +0,0 @@
# pre\_kill\_hook package
## Overview
The pre\_kill\_hook package provides a per-platform method for running code
after a test process times out but before the concurrent test runner kills the
timed-out process.
## Detailed Description of Usage
If a platform defines the hook, then the hook gets called right after a timeout
is detected in a test run, but before the process is killed.
The pre-kill-hook mechanism works as follows:
* When a timeout is detected in the process_control.ProcessDriver class that
runs the per-test lldb process, a new overridable on\_timeout\_pre\_kill() method
is called on the ProcessDriver instance.
* The concurrent test driver's derived ProcessDriver overrides this method. It
looks to see if a module called
"lldbsuite.pre\_kill\_hook.{platform-system-name}" module exists, where
platform-system-name is replaced with platform.system().lower(). (e.g.
"Darwin" becomes the darwin.py module).
* If that module doesn't exist, the rest of the new behavior is skipped.
* If that module does exist, it is loaded, and the method
"do\_pre\_kill(process\_id, context\_dict, output\_stream)" is called. If
that method throws an exception, we log it and we ignore further processing
of the pre-killed process.
* The process\_id argument of the do\_pre\_kill function is the process id as
returned by the ProcessDriver.pid property.
* The output\_stream argument of the do\_pre\_kill function takes a file-like
object. Output to be collected from doing any processing on the
process-to-be-killed should be written into the file-like object. The
current impl uses a six.StringIO and then writes this output to
{TestFilename}-{pid}.sample in the session directory.
* Platforms where platform.system() is "Darwin" will get a pre-kill action that
runs the 'sample' program on the lldb that has timed out. That data will be
collected on CI and analyzed to determine what is happening during timeouts.
(This has an advantage over a core in that it is much smaller and that it
clearly demonstrates any liveness of the process, if there is any).
## Running the tests
To run the tests in the pre\_kill\_hook package, open a console, change into
this directory and run the following:
```
python -m unittest discover
```

1
lldb/packages/Python/lldbsuite/pre_kill_hook/__init__.py
View File

@ -1 +0,0 @@
"""Initialize the package."""

46
lldb/packages/Python/lldbsuite/pre_kill_hook/darwin.py
View File

@ -1,46 +0,0 @@
"""Provides a pre-kill method to run on macOS."""
from __future__ import print_function
# system imports
import subprocess
import sys
# third-party module imports
import six
def do_pre_kill(process_id, runner_context, output_stream, sample_time=3):
"""Samples the given process id, and puts the output to output_stream.
@param process_id the local process to sample.
@param runner_context a dictionary of details about the architectures
and platform on which the given process is running. Expected keys are
archs (array of architectures), platform_name, platform_url, and
platform_working_dir.
@param output_stream file-like object that should be used to write the
results of sampling.
@param sample_time specifies the time in seconds that should be captured.
"""
# Validate args.
if runner_context is None:
raise Exception("runner_context argument is required")
if not isinstance(runner_context, dict):
raise Exception("runner_context argument must be a dictionary")
# We will try to run sample on the local host only if there is no URL
# to a remote.
if "platform_url" in runner_context and (
runner_context["platform_url"] is not None):
import pprint
sys.stderr.write(
"warning: skipping timeout pre-kill sample invocation because we "
"don't know how to run on a remote yet. runner_context={}\n"
.format(pprint.pformat(runner_context)))
output = subprocess.check_output(['sample', six.text_type(process_id),
str(sample_time)])
output_stream.write(output)

76
lldb/packages/Python/lldbsuite/pre_kill_hook/linux.py
View File

@ -1,76 +0,0 @@
"""Provides a pre-kill method to run on Linux.
This timeout pre-kill method relies on the Linux perf-tools
distribution. The appropriate way to obtain this set of tools
will depend on the Linux distribution.
For Ubuntu 16.04, the invoke the following command:
sudo apt-get install perf-tools-unstable
"""
from __future__ import print_function
# system imports
import os
import subprocess
import sys
import tempfile
def do_pre_kill(process_id, runner_context, output_stream, sample_time=3):
"""Samples the given process id, and puts the output to output_stream.
@param process_id the local process to sample.
@param runner_context a dictionary of details about the architectures
and platform on which the given process is running. Expected keys are
archs (array of architectures), platform_name, platform_url, and
platform_working_dir.
@param output_stream file-like object that should be used to write the
results of sampling.
@param sample_time specifies the time in seconds that should be captured.
"""
# Validate args.
if runner_context is None:
raise Exception("runner_context argument is required")
if not isinstance(runner_context, dict):
raise Exception("runner_context argument must be a dictionary")
# We will try to run sample on the local host only if there is no URL
# to a remote.
if "platform_url" in runner_context and (
runner_context["platform_url"] is not None):
import pprint
sys.stderr.write(
"warning: skipping timeout pre-kill sample invocation because we "
"don't know how to run on a remote yet. runner_context={}\n"
.format(pprint.pformat(runner_context)))
# We're going to create a temp file, and immediately overwrite it with the
# following command. This just ensures we don't have any races in
# creation of the temporary sample file.
fileno, filename = tempfile.mkstemp(suffix='perfdata')
os.close(fileno)
fileno = None
try:
with open(os.devnull, 'w') as devnull:
returncode = subprocess.call(['timeout', str(sample_time), 'perf',
'record', '-g', '-o', filename, '-p', str(process_id)],
stdout=devnull, stderr=devnull)
if returncode == 0 or returncode == 124:
# This is okay - this is the timeout return code, which is totally
# expected.
pass
else:
raise Exception("failed to call 'perf record .., error: {}".format(
returncode))
with open(os.devnull, 'w') as devnull:
output = subprocess.check_output(['perf', 'report', '--call-graph',
'--stdio', '-i', filename], stderr=devnull)
output_stream.write(output)
finally:
os.remove(filename)

0
lldb/packages/Python/lldbsuite/pre_kill_hook/tests/__init__.py
View File

107
lldb/packages/Python/lldbsuite/pre_kill_hook/tests/test_darwin.py
View File

@ -1,107 +0,0 @@
"""Test the pre-kill hook on Darwin."""
from __future__ import print_function
# system imports
from multiprocessing import Process, Queue
import platform
import re
from unittest import main, TestCase
# third party
from six import StringIO
def do_child_process(child_work_queue, parent_work_queue, verbose):
import os
pid = os.getpid()
if verbose:
print("child: pid {} started, sending to parent".format(pid))
parent_work_queue.put(pid)
if verbose:
print("child: waiting for shut-down request from parent")
child_work_queue.get()
if verbose:
print("child: received shut-down request. Child exiting.")
class DarwinPreKillTestCase(TestCase):
def __init__(self, methodName):
super(DarwinPreKillTestCase, self).__init__(methodName)
self.process = None
self.child_work_queue = None
self.verbose = False
def tearDown(self):
if self.verbose:
print("parent: sending shut-down request to child")
if self.process:
self.child_work_queue.put("hello, child")
self.process.join()
if self.verbose:
print("parent: child is fully shut down")
def test_sample(self):
# Ensure we're Darwin.
if platform.system() != 'Darwin':
self.skipTest("requires a Darwin-based OS")
# Start the child process.
self.child_work_queue = Queue()
parent_work_queue = Queue()
self.process = Process(target=do_child_process,
args=(self.child_work_queue, parent_work_queue,
self.verbose))
if self.verbose:
print("parent: starting child")
self.process.start()
# Wait for the child to report its pid. Then we know we're running.
if self.verbose:
print("parent: waiting for child to start")
child_pid = parent_work_queue.get()
# Sample the child process.
from darwin import do_pre_kill
context_dict = {
"archs": [platform.machine()],
"platform_name": None,
"platform_url": None,
"platform_working_dir": None
}
if self.verbose:
print("parent: running pre-kill action on child")
output_io = StringIO()
do_pre_kill(child_pid, context_dict, output_io)
output = output_io.getvalue()
if self.verbose:
print("parent: do_pre_kill() wrote the following output:", output)
self.assertIsNotNone(output)
# We should have a line with:
# Process: .* [{pid}]
process_re = re.compile(r"Process:[^[]+\[([^]]+)\]")
match = process_re.search(output)
self.assertIsNotNone(match, "should have found process id for "
"sampled process")
self.assertEqual(1, len(match.groups()))
self.assertEqual(child_pid, int(match.group(1)))
# We should see a Call graph: section.
callgraph_re = re.compile(r"Call graph:")
match = callgraph_re.search(output)
self.assertIsNotNone(match, "should have found the Call graph section"
"in sample output")
# We should see a Binary Images: section.
binary_images_re = re.compile(r"Binary Images:")
match = binary_images_re.search(output)
self.assertIsNotNone(match, "should have found the Binary Images "
"section in sample output")
if __name__ == "__main__":
main()

133
lldb/packages/Python/lldbsuite/pre_kill_hook/tests/test_linux.py
View File

@ -1,133 +0,0 @@
"""Test the pre-kill hook on Linux."""
from __future__ import print_function
# system imports
from multiprocessing import Process, Queue
import platform
import re
import subprocess
from unittest import main, TestCase
# third party
from six import StringIO
def do_child_thread():
import os
x = 0
while True:
x = x + 42 * os.getpid()
return x
def do_child_process(child_work_queue, parent_work_queue, verbose):
import os
pid = os.getpid()
if verbose:
print("child: pid {} started, sending to parent".format(pid))
parent_work_queue.put(pid)
# Spin up a daemon thread to do some "work", which will show
# up in a sample of this process.
import threading
worker = threading.Thread(target=do_child_thread)
worker.daemon = True
worker.start()
if verbose:
print("child: waiting for shut-down request from parent")
child_work_queue.get()
if verbose:
print("child: received shut-down request. Child exiting.")
class LinuxPreKillTestCase(TestCase):
def __init__(self, methodName):
super(LinuxPreKillTestCase, self).__init__(methodName)
self.process = None
self.child_work_queue = None
self.verbose = False
# self.verbose = True
def tearDown(self):
if self.verbose:
print("parent: sending shut-down request to child")
if self.process:
self.child_work_queue.put("hello, child")
self.process.join()
if self.verbose:
print("parent: child is fully shut down")
def test_sample(self):
# Ensure we're Darwin.
if platform.system() != 'Linux':
self.skipTest("requires a Linux-based OS")
# Ensure we have the 'perf' tool. If not, skip the test.
try:
perf_version = subprocess.check_output(["perf", "version"])
if perf_version is None or not (
perf_version.startswith("perf version")):
raise Exception("The perf executable doesn't appear"
" to be the Linux perf tools perf")
except Exception:
self.skipTest("requires the Linux perf tools 'perf' command")
# Start the child process.
self.child_work_queue = Queue()
parent_work_queue = Queue()
self.process = Process(target=do_child_process,
args=(self.child_work_queue, parent_work_queue,
self.verbose))
if self.verbose:
print("parent: starting child")
self.process.start()
# Wait for the child to report its pid. Then we know we're running.
if self.verbose:
print("parent: waiting for child to start")
child_pid = parent_work_queue.get()
# Sample the child process.
from linux import do_pre_kill
context_dict = {
"archs": [platform.machine()],
"platform_name": None,
"platform_url": None,
"platform_working_dir": None
}
if self.verbose:
print("parent: running pre-kill action on child")
output_io = StringIO()
do_pre_kill(child_pid, context_dict, output_io)
output = output_io.getvalue()
if self.verbose:
print("parent: do_pre_kill() wrote the following output:", output)
self.assertIsNotNone(output)
# We should have a samples count entry.
# Samples:
self.assertTrue("Samples:" in output, "should have found a 'Samples:' "
"field in the sampled process output")
# We should see an event count entry
event_count_re = re.compile(r"Event count[^:]+:\s+(\d+)")
match = event_count_re.search(output)
self.assertIsNotNone(match, "should have found the event count entry "
"in sample output")
if self.verbose:
print("cpu-clock events:", match.group(1))
# We should see some percentages in the file.
percentage_re = re.compile(r"\d+\.\d+%")
match = percentage_re.search(output)
self.assertIsNotNone(match, "should have found at least one percentage "
"in the sample output")
if __name__ == "__main__":
main()