2015-09-23 00:29:15 +08:00
"""
The LLVM Compiler Infrastructure
This file is distributed under the University of Illinois Open Source
License . See LICENSE . TXT for details .
"""
Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.
When absolute imports are enabled, the import system changes in
a couple of ways:
1) The `import foo` syntax will *only* search sys.path. If `foo`
isn't in sys.path, it won't be found. Period. Without absolute
imports, the import system will also search the same directory
that the importing file resides in, so that you can easily
import from the same folder.
2) From inside a package, you can use a dot syntax to refer to higher
levels of the current package. For example, if you are in the
package lldbsuite.test.utility, then ..foo refers to
lldbsuite.test.foo. You can use this notation with the
`from X import Y` syntax to write intra-package references. For
example, using the previous locationa s a starting point, writing
`from ..support import seven` would import lldbsuite.support.seven
Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.
See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.
Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala
llvm-svn: 252191
2015-11-06 03:22:28 +08:00
from __future__ import absolute_import
2016-04-21 00:27:27 +08:00
from __future__ import print_function
2015-10-20 07:45:41 +08:00
Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.
When absolute imports are enabled, the import system changes in
a couple of ways:
1) The `import foo` syntax will *only* search sys.path. If `foo`
isn't in sys.path, it won't be found. Period. Without absolute
imports, the import system will also search the same directory
that the importing file resides in, so that you can easily
import from the same folder.
2) From inside a package, you can use a dot syntax to refer to higher
levels of the current package. For example, if you are in the
package lldbsuite.test.utility, then ..foo refers to
lldbsuite.test.foo. You can use this notation with the
`from X import Y` syntax to write intra-package references. For
example, using the previous locationa s a starting point, writing
`from ..support import seven` would import lldbsuite.support.seven
Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.
See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.
Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala
llvm-svn: 252191
2015-11-06 03:22:28 +08:00
# System modules
2015-09-23 00:29:15 +08:00
import curses
2015-09-24 08:19:42 +08:00
import datetime
import math
2015-09-23 00:29:15 +08:00
import sys
2015-09-24 08:19:42 +08:00
import time
2015-09-23 00:29:15 +08:00
Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.
When absolute imports are enabled, the import system changes in
a couple of ways:
1) The `import foo` syntax will *only* search sys.path. If `foo`
isn't in sys.path, it won't be found. Period. Without absolute
imports, the import system will also search the same directory
that the importing file resides in, so that you can easily
import from the same folder.
2) From inside a package, you can use a dot syntax to refer to higher
levels of the current package. For example, if you are in the
package lldbsuite.test.utility, then ..foo refers to
lldbsuite.test.foo. You can use this notation with the
`from X import Y` syntax to write intra-package references. For
example, using the previous locationa s a starting point, writing
`from ..support import seven` would import lldbsuite.support.seven
Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.
See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.
Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala
llvm-svn: 252191
2015-11-06 03:22:28 +08:00
# Third-party modules
# LLDB modules
2016-04-21 00:27:27 +08:00
from lldbsuite . test import lldbcurses
from . import results_formatter
from . . event_builder import EventBuilder
2015-12-03 02:48:38 +08:00
Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.
When absolute imports are enabled, the import system changes in
a couple of ways:
1) The `import foo` syntax will *only* search sys.path. If `foo`
isn't in sys.path, it won't be found. Period. Without absolute
imports, the import system will also search the same directory
that the importing file resides in, so that you can easily
import from the same folder.
2) From inside a package, you can use a dot syntax to refer to higher
levels of the current package. For example, if you are in the
package lldbsuite.test.utility, then ..foo refers to
lldbsuite.test.foo. You can use this notation with the
`from X import Y` syntax to write intra-package references. For
example, using the previous locationa s a starting point, writing
`from ..support import seven` would import lldbsuite.support.seven
Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.
See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.
Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala
llvm-svn: 252191
2015-11-06 03:22:28 +08:00
2016-04-21 00:27:27 +08:00
class Curses ( results_formatter . ResultsFormatter ) :
2015-09-23 00:29:15 +08:00
""" Receives live results from tests that are running and reports them to the terminal in a curses GUI """
test infra: catch bad decorators and import-time errors
Summary:
This change enhances the LLDB test infrastructure to convert
load-time exceptions in a given Python test module into errors.
Before this change, specifying a non-existent test decorator,
or otherwise having some load-time error in a python test module,
would not get flagged as an error.
With this change, typos and other load-time errors in a python
test file get converted to errors and reported by the
test runner.
This change also includes test infrastructure tests that include
covering the new work here. I'm going to wait until we have
these infrastructure tests runnable on the main platforms before
I try to work that into all the normal testing workflows.
The test infrastructure tests can be run by using the standard python module testing practice of doing the following:
cd packages/Python/lldbsuite/test_event
python -m unittest discover -s test/src -p 'Test*.py'
Those tests run the dotest inferior with a known broken test and verify that the errors are caught. These tests did not pass until I modified dotest.py to capture them properly.
@zturner, if you have the chance, if you could try those steps above (the python -m unittest ... line) on Windows, that would be great if we can address any python2/3/Windows bits there. I don't think there's anything fancy, but I didn't want to hook it into test flow until I know it works there.
I'll be slowly adding more tests that cover some of the other breakage I've occasionally seen that didn't get collected as part of the summarization. This is the biggest one I'm aware of.
Reviewers: zturner, labath
Subscribers: zturner, lldb-commits
Differential Revision: http://reviews.llvm.org/D20193
llvm-svn: 269489
2016-05-14 05:36:26 +08:00
def __init__ ( self , out_file , options , file_is_stream ) :
2015-09-23 00:29:15 +08:00
# Initialize the parent
test infra: catch bad decorators and import-time errors
Summary:
This change enhances the LLDB test infrastructure to convert
load-time exceptions in a given Python test module into errors.
Before this change, specifying a non-existent test decorator,
or otherwise having some load-time error in a python test module,
would not get flagged as an error.
With this change, typos and other load-time errors in a python
test file get converted to errors and reported by the
test runner.
This change also includes test infrastructure tests that include
covering the new work here. I'm going to wait until we have
these infrastructure tests runnable on the main platforms before
I try to work that into all the normal testing workflows.
The test infrastructure tests can be run by using the standard python module testing practice of doing the following:
cd packages/Python/lldbsuite/test_event
python -m unittest discover -s test/src -p 'Test*.py'
Those tests run the dotest inferior with a known broken test and verify that the errors are caught. These tests did not pass until I modified dotest.py to capture them properly.
@zturner, if you have the chance, if you could try those steps above (the python -m unittest ... line) on Windows, that would be great if we can address any python2/3/Windows bits there. I don't think there's anything fancy, but I didn't want to hook it into test flow until I know it works there.
I'll be slowly adding more tests that cover some of the other breakage I've occasionally seen that didn't get collected as part of the summarization. This is the biggest one I'm aware of.
Reviewers: zturner, labath
Subscribers: zturner, lldb-commits
Differential Revision: http://reviews.llvm.org/D20193
llvm-svn: 269489
2016-05-14 05:36:26 +08:00
super ( Curses , self ) . __init__ ( out_file , options , file_is_stream )
2015-09-23 00:29:15 +08:00
self . using_terminal = True
self . have_curses = True
self . initialize_event = None
self . jobs = [ None ] * 64
self . job_tests = [ None ] * 64
self . results = list ( )
try :
2015-12-03 02:48:38 +08:00
self . main_window = lldbcurses . intialize_curses ( )
2015-09-24 08:19:42 +08:00
self . main_window . add_key_action ( ' \t ' , self . main_window . select_next_first_responder , " Switch between views that can respond to keyboard input " )
2015-09-23 00:29:15 +08:00
self . main_window . refresh ( )
self . job_panel = None
self . results_panel = None
self . status_panel = None
self . info_panel = None
2015-09-23 01:18:15 +08:00
self . hide_status_list = list ( )
2015-09-24 08:19:42 +08:00
self . start_time = time . time ( )
2015-09-23 00:29:15 +08:00
except :
self . have_curses = False
lldbcurses . terminate_curses ( )
self . using_terminal = False
2015-10-20 07:45:41 +08:00
print ( " Unexpected error: " , sys . exc_info ( ) [ 0 ] )
2015-09-23 00:29:15 +08:00
raise
2015-12-03 02:48:38 +08:00
2015-09-23 00:29:15 +08:00
self . line_dict = dict ( )
2015-12-03 02:48:38 +08:00
# self.events_file = open("/tmp/events.txt", "w")
2015-09-23 00:29:15 +08:00
# self.formatters = list()
# if tee_results_formatter:
# self.formatters.append(tee_results_formatter)
2016-05-14 08:42:30 +08:00
def status_to_short_str ( self , status , test_event ) :
2015-12-03 02:48:38 +08:00
if status == EventBuilder . STATUS_SUCCESS :
2015-09-23 00:29:15 +08:00
return ' . '
2015-12-03 02:48:38 +08:00
elif status == EventBuilder . STATUS_FAILURE :
2015-09-23 00:29:15 +08:00
return ' F '
2015-12-03 02:48:38 +08:00
elif status == EventBuilder . STATUS_UNEXPECTED_SUCCESS :
2015-09-23 00:29:15 +08:00
return ' ? '
2015-12-03 02:48:38 +08:00
elif status == EventBuilder . STATUS_EXPECTED_FAILURE :
2015-09-23 00:29:15 +08:00
return ' X '
2015-12-03 02:48:38 +08:00
elif status == EventBuilder . STATUS_SKIP :
2015-09-23 00:29:15 +08:00
return ' S '
2015-12-03 02:48:38 +08:00
elif status == EventBuilder . STATUS_ERROR :
2016-05-14 08:42:30 +08:00
if test_event . get ( " issue_phase " , None ) == " build " :
# Build failure
return ' B '
else :
return ' E '
2016-04-21 00:27:27 +08:00
elif status == EventBuilder . STATUS_TIMEOUT :
return ' T '
elif status == EventBuilder . STATUS_EXPECTED_TIMEOUT :
return ' t '
2015-09-23 00:29:15 +08:00
else :
return status
2015-09-24 08:19:42 +08:00
def show_info_panel ( self ) :
selected_idx = self . results_panel . get_selected_idx ( )
if selected_idx > = 0 and selected_idx < len ( self . results ) :
if self . info_panel is None :
info_frame = self . results_panel . get_contained_rect ( top_inset = 10 , left_inset = 10 , right_inset = 10 , height = 30 )
self . info_panel = lldbcurses . BoxedPanel ( info_frame , " Result Details " )
# Add a key action for any key that will hide this panel when any key is pressed
self . info_panel . add_key_action ( - 1 , self . hide_info_panel , ' Hide the info panel ' )
self . info_panel . top ( )
else :
self . info_panel . show ( )
2015-12-03 02:48:38 +08:00
2015-10-08 04:00:28 +08:00
self . main_window . push_first_responder ( self . info_panel )
2015-09-24 08:19:42 +08:00
test_start = self . results [ selected_idx ] [ 0 ]
test_result = self . results [ selected_idx ] [ 1 ]
self . info_panel . set_line ( 0 , " File: %s " % ( test_start [ ' test_filename ' ] ) )
self . info_panel . set_line ( 1 , " Test: %s . %s " % ( test_start [ ' test_class ' ] , test_start [ ' test_name ' ] ) )
self . info_panel . set_line ( 2 , " Time: %s " % ( test_result [ ' elapsed_time ' ] ) )
self . info_panel . set_line ( 3 , " Status: %s " % ( test_result [ ' status ' ] ) )
2015-09-23 01:18:15 +08:00
2015-09-24 08:19:42 +08:00
def hide_info_panel ( self ) :
2015-10-08 04:00:28 +08:00
self . main_window . pop_first_responder ( self . info_panel )
2015-12-03 02:48:38 +08:00
self . info_panel . hide ( )
2015-09-23 00:29:15 +08:00
self . main_window . refresh ( )
2015-12-03 02:48:38 +08:00
2015-09-24 08:19:42 +08:00
def toggle_status ( self , status ) :
if status :
# Toggle showing and hiding results whose status matches "status" in "Results" window
if status in self . hide_status_list :
self . hide_status_list . remove ( status )
2015-09-23 01:18:15 +08:00
else :
2015-09-24 08:19:42 +08:00
self . hide_status_list . append ( status )
self . update_results ( )
2015-09-23 01:18:15 +08:00
def update_results ( self , update = True ) :
''' Called after a category of test have been show/hidden to update the results list with
what the user desires to see . '''
self . results_panel . clear ( update = False )
for result in self . results :
test_result = result [ 1 ]
status = test_result [ ' status ' ]
if status in self . hide_status_list :
continue
name = test_result [ ' test_class ' ] + ' . ' + test_result [ ' test_name ' ]
2016-05-14 08:42:30 +08:00
self . results_panel . append_line ( ' %s ( %6.2f sec) %s ' % ( self . status_to_short_str ( status , test_result ) , test_result [ ' elapsed_time ' ] , name ) )
2015-09-23 01:18:15 +08:00
if update :
2015-12-03 02:48:38 +08:00
self . main_window . refresh ( )
2015-09-23 00:29:15 +08:00
def handle_event ( self , test_event ) :
with self . lock :
super ( Curses , self ) . handle_event ( test_event )
# for formatter in self.formatters:
# formatter.process_event(test_event)
if self . have_curses :
worker_index = - 1
if ' worker_index ' in test_event :
worker_index = test_event [ ' worker_index ' ]
if ' event ' in test_event :
check_for_one_key = True
2015-10-24 01:04:29 +08:00
#print(str(test_event), file=self.events_file)
2015-12-03 02:48:38 +08:00
event = test_event [ ' event ' ]
2015-09-24 08:19:42 +08:00
if self . status_panel :
self . status_panel . update_status ( ' time ' , str ( datetime . timedelta ( seconds = math . floor ( time . time ( ) - self . start_time ) ) ) )
2015-09-23 00:29:15 +08:00
if event == ' test_start ' :
name = test_event [ ' test_class ' ] + ' . ' + test_event [ ' test_name ' ]
self . job_tests [ worker_index ] = test_event
if ' pid ' in test_event :
line = ' pid: %5d ' % ( test_event [ ' pid ' ] ) + name
else :
line = name
self . job_panel . set_line ( worker_index , line )
self . main_window . refresh ( )
elif event == ' test_result ' :
status = test_event [ ' status ' ]
self . status_panel . increment_status ( status )
if ' pid ' in test_event :
line = ' pid: %5d ' % ( test_event [ ' pid ' ] )
else :
line = ' '
self . job_panel . set_line ( worker_index , line )
name = test_event [ ' test_class ' ] + ' . ' + test_event [ ' test_name ' ]
elapsed_time = test_event [ ' event_time ' ] - self . job_tests [ worker_index ] [ ' event_time ' ]
2015-09-23 01:18:15 +08:00
if not status in self . hide_status_list :
2016-05-14 08:42:30 +08:00
self . results_panel . append_line ( ' %s ( %6.2f sec) %s ' % ( self . status_to_short_str ( status , test_event ) , elapsed_time , name ) )
2015-09-23 00:29:15 +08:00
self . main_window . refresh ( )
# Append the result pairs
test_event [ ' elapsed_time ' ] = elapsed_time
self . results . append ( [ self . job_tests [ worker_index ] , test_event ] )
self . job_tests [ worker_index ] = ' '
elif event == ' job_begin ' :
self . jobs [ worker_index ] = test_event
if ' pid ' in test_event :
line = ' pid: %5d ' % ( test_event [ ' pid ' ] )
else :
line = ' '
self . job_panel . set_line ( worker_index , line )
elif event == ' job_end ' :
self . jobs [ worker_index ] = ' '
self . job_panel . set_line ( worker_index , ' ' )
2015-12-03 02:48:38 +08:00
elif event == ' initialize ' :
2015-09-23 00:29:15 +08:00
self . initialize_event = test_event
num_jobs = test_event [ ' worker_count ' ]
job_frame = self . main_window . get_contained_rect ( height = num_jobs + 2 )
results_frame = self . main_window . get_contained_rect ( top_inset = num_jobs + 2 , bottom_inset = 1 )
status_frame = self . main_window . get_contained_rect ( height = 1 , top_inset = self . main_window . get_size ( ) . h - 1 )
2015-09-24 08:19:42 +08:00
self . job_panel = lldbcurses . BoxedPanel ( frame = job_frame , title = " Jobs " )
self . results_panel = lldbcurses . BoxedPanel ( frame = results_frame , title = " Results " )
2015-12-03 02:48:38 +08:00
2015-09-24 08:19:42 +08:00
self . results_panel . add_key_action ( curses . KEY_UP , self . results_panel . select_prev , " Select the previous list entry " )
self . results_panel . add_key_action ( curses . KEY_DOWN , self . results_panel . select_next , " Select the next list entry " )
self . results_panel . add_key_action ( curses . KEY_HOME , self . results_panel . scroll_begin , " Scroll to the start of the list " )
self . results_panel . add_key_action ( curses . KEY_END , self . results_panel . scroll_end , " Scroll to the end of the list " )
self . results_panel . add_key_action ( curses . KEY_ENTER , self . show_info_panel , " Display info for the selected result item " )
2015-12-03 02:48:38 +08:00
self . results_panel . add_key_action ( ' . ' , lambda : self . toggle_status ( EventBuilder . STATUS_SUCCESS ) , " Toggle showing/hiding tests whose status is ' success ' " )
self . results_panel . add_key_action ( ' e ' , lambda : self . toggle_status ( EventBuilder . STATUS_ERROR ) , " Toggle showing/hiding tests whose status is ' error ' " )
self . results_panel . add_key_action ( ' f ' , lambda : self . toggle_status ( EventBuilder . STATUS_FAILURE ) , " Toggle showing/hiding tests whose status is ' failure ' " )
self . results_panel . add_key_action ( ' s ' , lambda : self . toggle_status ( EventBuilder . STATUS_SKIP ) , " Toggle showing/hiding tests whose status is ' skip ' " )
self . results_panel . add_key_action ( ' x ' , lambda : self . toggle_status ( EventBuilder . STATUS_EXPECTED_FAILURE ) , " Toggle showing/hiding tests whose status is ' expected_failure ' " )
self . results_panel . add_key_action ( ' ? ' , lambda : self . toggle_status ( EventBuilder . STATUS_UNEXPECTED_SUCCESS ) , " Toggle showing/hiding tests whose status is ' unexpected_success ' " )
2015-09-23 00:29:15 +08:00
self . status_panel = lldbcurses . StatusPanel ( frame = status_frame )
2015-12-03 02:48:38 +08:00
2015-09-23 00:29:15 +08:00
self . main_window . add_child ( self . job_panel )
self . main_window . add_child ( self . results_panel )
self . main_window . add_child ( self . status_panel )
self . main_window . set_first_responder ( self . results_panel )
2015-12-03 02:48:38 +08:00
2015-09-24 08:19:42 +08:00
self . status_panel . add_status_item ( name = " time " , title = " Elapsed " , format = " %s " , width = 20 , value = " 0:00:00 " , update = False )
2015-12-03 02:48:38 +08:00
self . status_panel . add_status_item ( name = EventBuilder . STATUS_SUCCESS , title = " Success " , format = " %u " , width = 20 , value = 0 , update = False )
self . status_panel . add_status_item ( name = EventBuilder . STATUS_FAILURE , title = " Failure " , format = " %u " , width = 20 , value = 0 , update = False )
self . status_panel . add_status_item ( name = EventBuilder . STATUS_ERROR , title = " Error " , format = " %u " , width = 20 , value = 0 , update = False )
self . status_panel . add_status_item ( name = EventBuilder . STATUS_SKIP , title = " Skipped " , format = " %u " , width = 20 , value = 0 , update = True )
self . status_panel . add_status_item ( name = EventBuilder . STATUS_EXPECTED_FAILURE , title = " Expected Failure " , format = " %u " , width = 30 , value = 0 , update = False )
self . status_panel . add_status_item ( name = EventBuilder . STATUS_UNEXPECTED_SUCCESS , title = " Unexpected Success " , format = " %u " , width = 30 , value = 0 , update = False )
2015-09-23 00:29:15 +08:00
self . main_window . refresh ( )
elif event == ' terminate ' :
2015-09-25 00:37:41 +08:00
#self.main_window.key_event_loop()
2015-09-23 00:29:15 +08:00
lldbcurses . terminate_curses ( )
check_for_one_key = False
self . using_terminal = False
2015-12-03 02:48:38 +08:00
# Check for 1 keypress with no delay
2015-09-23 00:29:15 +08:00
# Check for 1 keypress with no delay
if check_for_one_key :
2015-12-03 02:48:38 +08:00
self . main_window . key_event_loop ( 0 , 1 )
