llvm-project/lldb/test/redo.py

187 lines
6.3 KiB
Python
Raw Normal View History

#!/usr/bin/env python
"""
A simple utility to redo the failed/errored tests.
You need to specify the session directory in order for this script to locate the
tests which need to be re-run.
See also dotest.py, the test driver running the test suite.
Type:
./dotest.py -h
for help.
"""
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
import os, sys, datetime
import re
# If True, redo with no '-t' option for the test driver.
no_trace = False
# To be filled with the filterspecs found in the session logs.
redo_specs = []
# The filename components to match for. Only files with the contained component names
# will be considered for re-run. Examples: ['X86_64', 'clang'].
filename_components = []
# There is a known bug with respect to comp_specs and arch_specs, in that if we
# encountered "-C clang" and "-C gcc" when visiting the session files, both
# compilers will end up in the invocation of the test driver when rerunning.
# That is: ./dotest -v -C clang^gcc ... -f ...". Ditto for "-A" flags.
2011-08-17 04:57:05 +08:00
# The "-C compiler" for comp_specs.
comp_specs = set()
# The "-A arch" for arch_specs.
arch_specs = set()
def usage():
print"""\
Usage: redo.py [-F filename_component] [-n] [session_dir]
where options:
-F : only consider the test for re-run if the session filename conatins the filename component
for example: -F x86_64
-n : when running the tests, do not turn on trace mode, i.e, no '-t' option
is passed to the test driver (this will run the tests faster)
and session_dir specifies the session directory which contains previously
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
recorded session infos for all the test cases which either failed or errored.
If sessin_dir is left unspecified, this script uses the heuristic to find the
possible session directories with names starting with %Y-%m-%d- (for example,
2012-01-23-) and employs the one with the latest timestamp."""
sys.exit(0)
def where(session_dir, test_dir):
"""Returns the full path to the session directory; None if non-existent."""
abspath = os.path.abspath(session_dir)
if os.path.isdir(abspath):
return abspath
session_dir_path = os.path.join(test_dir, session_dir)
if os.path.isdir(session_dir_path):
return session_dir_path
return None
# This is the pattern for the line from the log file to redo a test.
# We want the filter spec.
filter_pattern = re.compile("^\./dotest\.py.*-f (.*)$")
comp_pattern = re.compile(" -C ([^ ]+) ")
arch_pattern = re.compile(" -A ([^ ]+) ")
def redo(suffix, dir, names):
"""Visitor function for os.path.walk(path, visit, arg)."""
global redo_specs
global comp_specs
global arch_specs
global filter_pattern
global comp_pattern
global arch_pattern
global filename_components
for name in names:
if name.endswith(suffix):
#print "Find a log file:", name
if name.startswith("Error") or name.startswith("Failure"):
if filename_components:
if not all([comp in name for comp in filename_components]):
continue
with open(os.path.join(dir, name), 'r') as log:
content = log.read()
for line in content.splitlines():
match = filter_pattern.match(line)
if match:
filterspec = match.group(1)
print "adding filterspec:", filterspec
redo_specs.append(filterspec)
comp = comp_pattern.search(line)
if comp:
comp_specs.add(comp.group(1))
arch = arch_pattern.search(line)
if arch:
arch_specs.add(arch.group(1))
else:
continue
def main():
"""Read the session directory and run the failed test cases one by one."""
global no_trace
global redo_specs
global filename_components
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
test_dir = sys.path[0]
if not test_dir:
test_dir = os.getcwd()
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
if not test_dir.endswith('test'):
print "This script expects to reside in lldb's test directory."
sys.exit(-1)
index = 1
while index < len(sys.argv):
if sys.argv[index].startswith('-h') or sys.argv[index].startswith('--help'):
usage()
if sys.argv[index].startswith('-'):
# We should continue processing...
pass
else:
# End of option processing.
break
if sys.argv[index] == '-F':
# Increment by 1 to fetch the filename component spec.
index += 1
if index >= len(sys.argv) or sys.argv[index].startswith('-'):
usage()
filename_components.append(sys.argv[index])
elif sys.argv[index] == '-n':
no_trace = True
index += 1
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
if index < len(sys.argv):
# Get the specified session directory.
session_dir = sys.argv[index]
else:
# Use heuristic to find the latest session directory.
name = datetime.datetime.now().strftime("%Y-%m-%d-")
dirs = [d for d in os.listdir(os.getcwd()) if d.startswith(name)]
if len(dirs) == 0:
print "No default session directory found, please specify it explicitly."
usage()
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
session_dir = max(dirs, key=os.path.getmtime)
if not session_dir or not os.path.exists(session_dir):
print "No default session directory found, please specify it explicitly."
usage()
#print "The test directory:", test_dir
session_dir_path = where(session_dir, test_dir)
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
print "Using session dir path:", session_dir_path
os.chdir(test_dir)
os.path.walk(session_dir_path, redo, ".log")
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
if not redo_specs:
print "No failures/errors recorded within the session directory, please specify a different session directory.\n"
Modify redo.py script so that if sessin_dir is left unspecified, it uses the heuristic to find the possible session directories with names starting with %Y-%m-%d- (for example, 2012-01-23-) and employs the one with the latest timestamp. For example: johnny:/Volumes/data/lldb/svn/latest/test $ ./redo.py Using session dir path: /Volumes/data/lldb/svn/latest/test/2012-01-23-11_28_30 adding filterspec: DisassembleRawDataTestCase.test_disassemble_raw_data Running ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data LLDB build dir: /Volumes/data/lldb/svn/latest/build/Debug LLDB-108 Path: /Volumes/data/lldb/svn/latest URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk Repository Root: https://johnny@llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 148710 Node Kind: directory Schedule: normal Last Changed Author: gclayton Last Changed Rev: 148650 Last Changed Date: 2012-01-21 18:55:08 -0800 (Sat, 21 Jan 2012) Session logs for test failures/errors/unexpected successes will go into directory '2012-01-23-17_04_48' Command invoked: python ./dotest.py -C clang -v -t -f DisassembleRawDataTestCase.test_disassemble_raw_data Configuration: compiler=clang ---------------------------------------------------------------------- Collected 1 test Change dir to: /Volumes/data/lldb/svn/latest/test/python_api/disassemble-raw-data 1: test_disassemble_raw_data (TestDisassembleRawData.DisassembleRawDataTestCase) Test disassembling raw bytes with the API. ... Raw bytes: ['0x48', '0x89', '0xe5'] Disassembled: movq %rsp, %rbp ok Restore dir to: /Volumes/data/lldb/svn/latest/test ---------------------------------------------------------------------- Ran 1 test in 0.233s OK llvm-svn: 148766
2012-01-24 09:53:02 +08:00
usage()
filters = " -f ".join(redo_specs)
compilers = ''
for comp in comp_specs:
compilers += " -C %s" % (comp)
archs = ''
for arch in arch_specs:
archs += "--arch %s " % (arch)
command = "./dotest.py %s %s -v %s -f " % (compilers, archs, "" if no_trace else "-t")
print "Running %s" % (command + filters)
os.system(command + filters)
if __name__ == '__main__':
main()