mirrors/eiffel-org
1
0
Fork 0
mirror of https://github.com/EiffelSoftware/eiffel-org.git synced 2025-12-08 07:42:33 +01:00
Code Issues Packages Projects Releases Wiki Activity

Author:halw

Browse Source 
Date:2010-01-19T20:15:45.000000Z


git-svn-id: https://svn.eiffel.com/eiffel-org/trunk@398 abb3cda0-5349-4a8f-a601-0c33ac3a8c38
This commit is contained in:
halw
2010-01-19 20:17:40 +00:00
parent 563c5218a2
commit db4202f903
5 changed files with 168 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
documentation/current/eiffelstudio/eiffelstudio-reference/autotest/autotest-interface.wiki
View File

@@ -1,19 +1,14 @@
[[Property:title|The Eiffel Testing Framework Interface]]
[[Property:link_title|The AutoTest Interface]]
[[Property:title|The AutoTest Interface]]
[[Property:weight|0]]
[[Property:uuid|6eec11df-9ea0-6834-d41b-a0c23b87c485]]
[[Image:AutoTest interface annotated 01]]
[[Image:Testing Framework interface annotated 01]]
The Eiffel Testing Framework interface consists of four primary components:
The AutoTest interface consists of four primary components:
# The toolbar
# The view filtering controls
# The tests view
# The tabbed status and results reporting pane
# The tests pane
# The test filtering box
# The tabbed creation and results reporting pane
==Toolbar==
@@ -21,65 +16,99 @@ The toolbar allows you to create new tests using the '''[[#The New Eiffel test w
The '''Create new tests''' command ( [[Image:Create new tests]] ) invokes the '''New Eiffel test wizard'''.
The '''Run all tests in background''' command ( [[Image:debug-run-icon]] ) will, by default, run all tests. But if only certain tests are selected in the '''Tests''' list, then '''Run all tests in background''' will run only those tests. The '''Run all tests in background''' allows other options for running tests through its drop-down icon ( [[Image:toolbar-dropdown-icon]] ).
The '''Run filtered tests''' command ( [[Image:debug-run-icon]] ) will, by default, run all tests that are visible in the Tests pane. Whether or not a test is visible depends upon any test [[#Filtering|filtering]] currently in effect. The '''Run filtered tests''' allows other options for running tests through its drop-down icon ( [[Image:toolbar-dropdown-icon]] ).
[[Image:Testing Framework run all drop down]]
[[Image:AutoTest run all drop down]]
The '''Debug all tests in EiffelStudio''' command ( [[Image:Testing Framework debug all icon]] ) functions in a similar manner to '''Run all tests in background''' with the exception that each test is paused at its starting point in the EiffelStudio debugger. '''Debug all tests in EiffelStudio''' has a drop-down icon providing the same options as '''Run all tests in background'''.
The '''Debug filtered tests''' command ( [[Image:AutoTest debug all icon]] ) functions in a similar manner to '''Run filtered tests''' with the exception that each test is paused at its starting point in the EiffelStudio debugger. '''Debug filtered tests''' has a drop-down icon providing the same options as '''Run filtered tests'''.
[[Image:Testing Framework debug all drop down]]
[[Image:AutoTest debug all drop down]]
The '''Stop all execution''' command ( [[Image:debug-stop-icon]] ) will halt all tests which were started by the '''Run all tests in background''' or '''Debug all tests in EiffelStudio''' commands.
==View Filtering==
{{note|The development of this documentation section is currently on hold pending changes to the interface expected in an upcoming release. }}
The '''Stop all test related tasks''' command ( [[Image:debug-stop-icon]] ) will halt all tests which were started by the '''Run filtered tests''' or '''Debug filtered tests''' commands.
==Tests==
The Tests pane presents tests in one or more expandable tree views. The structures of the trees depend upon the criteria that you have specified for organizing the view. For example, if you look at tests based on the test classes in which they exist, the structure of the tree view(s) will reflect the clusters in which the test classes exist.
The Tests pane presents tests in one or more expandable tree views. The structures of the trees depend upon the [[#Filtering|filter]] criteria that you have specified for organizing the view. For example, if you look at tests based on the test classes in which they exist (tag root "''class''"), the structure of the tree view(s) will reflect the clusters in which the test classes exist.
[[Image:Testing Framework interface test view - class]]
[[Image:AutoTest interface test view class]]
Likewise, if you look at tests based on their target classes, the structure of the display will be based on the target classes.
Likewise, if you look at tests based on their target classes (tag root "''covers''"), the structure of the display will be based on the target classes.
[[Image:Testing Framework interface test view - covers]]
[[Image:AutoTest interface test view covers]]
Tests and test classes are pickable in the Tests pane.
==Status and results==
==Filtering==
This pane supports a tabbed display that provides information about the execution and results of tests.
The '''Filter''' box allows you to limit which tests are visible in the Tests pane (and, by consequence, the tests run when '''Run filtered tests''' is clicked) by entering filter text in the box and pressing the Enter key. Filter text can be a string of characters that is contained in a test class or test routine name, or it can be a [[Create a manual test#About Tags|tag]] or a portion of a tag hierarchy.
After execution or debugging, the tests executed and their status are shown in the '''Execution''' tab or the '''Debugging''' tab.
In fact, filter text fully supports [http://www.regular-expressions.info/ regular expressions], so you can craft fine-grained filters.
Clicking the '''Clear filter''' icon ( [[Image:general-reset-icon]] ) to the right of the filter box will clear any filter contents and enable the viewing of all tests. The Tests pane is depicted below after clicking the '''Clear filter''' icon and expanding some of the items:
[[Image:Testing Framework interface results - Execution]]
[[Image:AutoTest tests pane null filter]]
Double clicking a test in the '''Tests''' pane will switch the '''Status and results''' pane to the '''Results''' tab. This tab shows an expandable history of the executions and results of that particular test.
The '''Filter''' box has a drop down icon on its right end. By default this icon provides views of the tests in the Tests pane based on tag criteria.
[[Image:Testing Framework interface results - Results]]
[[Image:AutoTest filter drop down]]
For failing executions, the failure code is shown, along with link to display the execution trace.
[[Image:Testing Framework interface results - trace]]
The criteria are associated with the roots of their respective tag trees. So for example, if '''Results''' is chosen the Tests pane shows tests grouped by their latest results.
[[Image:AutoTest filter result]]
{{note|The '''result''' tag hierarchy is, as you may have guessed, virtual and dynamic. A test falling into the '''fail''' subtree may later become part of the '''pass''' subtree without any change to the '''"testing:"''' notes in the test routine code, where tags are established. But for filtering purposes '''result''' tags work like other tags. }}
If the '''fail''' subtree name were appended to the '''result''' filter, only the failed tests would be visible in the Tests pane:
[[Image:AutoTest filter result fail]]
If the filter text is a character string that occurs in a test name, test class name, or a tag path, tests with any match will be visible:
[[Image:AutoTest filter withdraw]]
==Results==
This pane supports a tabbed display that provides information about the creation and execution of tests.
After creation of one or more tests, the creation is logged under the '''Creation''' tab.
After execution, the tests executed and the status of those tests are shown in the '''Execution''' tab.
[[Image:AutoTest interface results execution]]
After executing a set of tests, the Testing pane of the Outputs tool shows a chronology of the executions and the results of the tests executed.
[[Image:AutoTest testing pane execution results]]
You can view more detail about a test's execution by clicking the '''Information''' icon ( [[Image:AutoTest information icon]] ) to the right of the test's entry in '''Results''' when the '''Execution''' tab is selected.
[[Image:AutoTest test results details]]

42
documentation/current/eiffelstudio/eiffelstudio-reference/autotest/eiffel-test-wizard.wiki
View File

@@ -2,13 +2,11 @@
[[Property:link_title|The Eiffel Test Wizard]]
[[Property:weight|1]]
[[Property:uuid|305a8288-cb6a-df2a-1515-e23138e21566]]
==Introduction==
The '''New Eiffel test wizard''' creates new tests based on information entered by a developer on a series of wizard panes. Each of the three test types supported by the Eiffel Testing Framework requires certain unique information. Therefore, there is a unique sequence of wizard panes corresponding to each test type.
The '''New Eiffel test wizard''' creates new tests based on information entered by a developer on a series of wizard panes. Each of the three test types supported by AutoTest requires certain unique information. Therefore, there is a unique sequence of wizard panes corresponding to each test type.
The '''New Eiffel test wizard''' is invoked by clicking the '''Create new tests''' button ( [[Image:create new tests]] ) on the Eiffel Testing Framework interface toolbar.
The '''New Eiffel test wizard''' is invoked by clicking the '''Create new tests''' button ( [[Image:create new tests]] ) on the AutoTest interface toolbar.
@@ -17,7 +15,7 @@ The '''New Eiffel test wizard''' is invoked by clicking the '''Create new tests'
The first pane of the wizard allows you to choose which type of test(s) you wish to create.
[[Image: New test wizard screen 01 01]]
[[Image:AutoTest new test wizard 01 01]]
==The manual test wizard sequence==
@@ -32,7 +30,7 @@ Creating manual tests will involve using the following wizard panes. For a guide
In this pane you provide a name and cluster for the test class that will be created. You can also indicate that the created class should include redefined versions of features <eiffel>on_prepare</eiffel> (called during preparation for test execution) and/or <eiffel>on_clean</eiffel> (called during clean-up after test execution).
[[Image: New test wizard screen 02M 01]]
[[Image: AutoTest new test wizard 02M 01]]
===The "Test routine" pane===
@@ -40,7 +38,7 @@ In this pane you provide a name and cluster for the test class that will be crea
This pane gives you the ability to name the test, and to associate it with certain [[Create a manual test#About Tags|tags]].
[[Image: New test wizard screen 03M 01]]
[[Image: AutoTest new test wizard 03M 01]]
For tags, there are two entry boxes with '''Add''' buttons to their right. You can use the top entry box to create entries for a tag system that you have developed.
@@ -64,45 +62,43 @@ For extracted tests, the second pane to appear is the '''New test class''' pane.
===The "Extract application state" pane===
[[Image:New test wizard screen 03E 01]]
[[Image:AutoTest new test wizard 03E 01]]
This pane provides a depiction of the current call stack. Choose the routine or routines for which you want extracted tests to be created.
==The synthesized test wizard sequence==
==The generated test wizard sequence==
Creating synthesized tests will involve using the following wizard panes. For a guided tour of this process, see [[Using synthesized tests]].
Creating generated tests will involve using the following wizard panes. For a guided tour of this process, see [[Using generated tests]].
===The "New test class" pane===
[[Image:New test wizard screen 02S 01]]
[[Image:AutoTest new test wizard 02S 01]]
This pane appears slightly different for synthesized tests versus manual or extracted tests. The class name is actually a prefix upon which a numeric index will be appended. The index is depicted to the right of the entry box.
This pane appears slightly different for generated tests versus manual or extracted tests. The class name is actually a prefix upon which a numeric index will be appended. The index is depicted to the right of the entry box.
===The "Generate tests through AutoTest" pane===
[[Image:New test wizard screen 03S 01]]
[[Image:AutoTest new test wizard 03S 01]]
On this pane enter the information needed to produce synthesized tests. Use the '''Class or type name''' entry box in the lower left, along with the <code>+</code> and <code>-</code> buttons, to declare a list of one or more classes as target classes for synthesized tests.
On this pane enter the information needed to produce generated tests. Use the '''Class or type name''' entry box in the lower left, along with the <code>+</code> and <code>-</code> buttons, to declare a list of one or more classes as target classes for generated tests.
Other information necessary for creating synthesized tests is:
Other information necessary for creating generated tests is:
*'''Cutoff (minutes)''' -- How long the Eiffel Testing Framework will execute random invocations of the routines in the classes to be tested, specified in minutes.
*'''Cutoff (invocations)''' -- How long the Eiffel Testing Framework will execute random invocations of the routines in the classes to be tested, specified by invocation count.
*'''Routine timeout (seconds)''' -- How long the Framework will wait for completion of any invocation.
*'''Random number generation seed''' -- The seed used for random number generator used during the process. A value of zero instructs the Framework to use a seed derived from the system clock.
*'''Use slicing for minimization''' -- Designate '''slicing''' as the approach for minimizing the size of synthesized tests.
*'''Use ddmin for minimization''' -- Designate '''ddmin''' as the approach for minimizing the size of synthesized tests.
*'''Cutoff (minutes)''' -- How long AutoTest will execute random invocations of the routines in the classes to be tested, specified in minutes.
*'''Cutoff (invocations)''' -- How long AutoTest will execute random invocations of the routines in the classes to be tested, specified by invocation count.
*'''Routine timeout (seconds)''' -- How long AutoTest will wait for completion of any invocation.
*'''Random number generation seed''' -- The seed used for random number generator used during the process. A value of zero instructs AutoTest to use a seed derived from the system clock.
*'''Use slicing for minimization''' -- Designate '''slicing''' as the approach for minimizing the size of generated tests.
*'''Use ddmin for minimization''' -- Designate '''ddmin''' as the approach for minimizing the size of generated tests.
*'''Create html output''' -- Output history and statistics from the synthesizing process in html format.