eiffel-org/documentation/18.11/eiffelstudio/Tutorials/using-autotest/using-extracted-tests.wiki

[[Property:title|Using extracted tests]]
[[Property:weight|5]]
[[Property:uuid|bebd4f28-9818-80f0-a69a-e9ce867723f4]]
==About extracted tests==

At any time that you are running a system in EiffelStudio debugger and your system is paused, you can ask AutoTest to extract a new test class and test from the current executable context. Most often you would use this capability in the case in which you experienced an unexpected failure or exception in one of your routines. It is possible, though, to extract at any point at which the system is paused.

The value of extracted tests is that they provide a kind of a snapshot in testing form that will reproduce the unexpected failure. An extracted test attempts to reproduce the context in which the offending routine executed. So, extracted tests supplement your manual tests. They serve to cover situations which you just may not have written manual tests to cover.

Extracted tests are intended to supplement the suite of manual tests that you have created to do the bulk of your testing. So, usually when you create an extracted test, it happens as a result of your being surprised. You will notice that each time you create an extracted test, you get a new test class, too. This is in contrast to manual tests, in which you might use the wizard to create a new test class and one new test to cover a particular target class and target routine. Then you might manually create, in that same test class, many additional tests covering the routine behavior of the same or other target routines in the same target class. 


==Creating an extracted test==

Let's use the same test system we used for manual tests to demonstrate the creation of an extracted test. The example will be slightly contrived because it will find a problem that certainly we would already have discovered had we written a comprehensive set of manual tests against the <code>BANK_ACCOUNT</code> class. Still, the simplicity should help keep things clear.

If you remember, the root class for the example application was not very interesting, just a root procedure with a single instruction and a declaration <code>my_account</code> of type <code>BANK_ACCOUNT</code>:


<code>
    make
            -- Run application.
        do
            create my_account
        end

    my_account: BANK_ACCOUNT
</code>


Now, let's add some code into the <code>make</code> procedure that will make use of <code>my_account</code>:

<code>
    make
            -- Run application.
        do
            create my_account
            my_account.deposit (500)
            my_account.withdraw (100)
        end
</code>


If we run the application from EiffelStudio, we see that it stops when it incurs a postcondition violation in <code>{BANK_ACCOUNT}.withdraw</code>:


[[Image:AutoTest extracted 01]]


When we look at the feature pane, it's pretty easy to see where the problem is:


[[Image:AutoTest extracted 02]]


There is an error in the specification for <code>withdraw</code>. In the postcondition tagged <code>withdrawn</code>, the plus sign should have been a minus sign. Therefore, the assertion should read like this:

<code>
            withdrawn: balance = old balance - an_amount
</code>

Certainly we will fix this, but AutoTest gives us the opportunity to extract a test based on this particular failure. So, let's do that.

So, we go to the AutoTest tool and click triangle next to ''Create new tests'' button and select the '''Extract tests from debugger''' from the drop-down menu. Because we are paused in the debugger, the drop-down menu appears with the '''Extract tests from debugger''' choice enabled this time:


[[Image:AutoTest create new test 02]]


When we select '''Extract tests from debugger''', we are presented with the New Eiffel Test Wizard's '''Test Extraction''' pane. This wizard pane shows a depiction of the current call stack and asks us for which feature(s) on the stack we want to create the test:


[[Image:AutoTest test extraction pane|Test extraction pane]]


The choice for <code>withdraw</code> is the selection we want. We can deselect the stack frame for <code>make</code> if it is pre-selected. If we click '''Next''' at this point we would be taken to the '''Tags''' pane, and from there to the '''General''' pane. But we really don't need to do this. AutoTest will sense that we are extracting a test for <code>{BANK_ACCOUNT}.withdraw</code> and tag the test properly. It will use the same test class name from the '''General''' pane, but add a numerical suffix. So, all we need to do now is to click '''Launch''' from the '''Text Extraction''' pane.

AutoTest creates the new test and returns us to the debugger, where our system is still on hold. We can stop execution and compile to include the new test.

Now we see the new test class and test in the AutoTest tool windows. 


==Run the tests, fix a problem, run the tests==

We run our tests using '''Run all''', and we see that the test on <code>withdraw</code> is still failing:


[[Image:AutoTest tool after run]]


If we fix the error in the postcondition in <code>withdraw</code>, recompile, and then re-execute the test, we find that it is successful.


==A closer look at an extracted test==

Look at the code that was generated for the extracted test after the assertion violation occurred:

<code>
note
    description: "Regression tests reproducing application state of a previous execution."
    author: "Testing tool"

class
    TEST_BANK_ACCOUNT_EXTRACTED_WITHDRAW_01
	
inherit
    EQA_EXTRACTED_TEST_SET

feature -- Test routines

    test_withdraw
        note
            testing: "type/extracted"
            testing: "covers/{BANK_ACCOUNT}.withdraw"
        do
            run_extracted_test (agent {BANK_ACCOUNT}.withdraw, ["#1", {INTEGER_32} 100])
        end

feature {NONE} -- Access

    context: !ARRAY [!TUPLE [type: !TYPE [ANY]; attributes: !TUPLE; inv: BOOLEAN]]
            -- <Precursor>
        do
            Result := <<
                [{BANK_ACCOUNT}, [
                        "balance",  {INTEGER_32} 400
                    ], False]
            >>
        end

end
</code>


You probably noticed immediately that it doesn't look much like the code that we wrote for our manual test in the previous section. 

One reason for the difference is that the class does not inherit directly from <code>EQA_TEST_SET</code> as our manual test did. Instead, it inherits from <code>EQA_EXTRACTED_TEST_SET</code> which itself is a descendant of <code>EQA_TEST_SET</code>. <code>EQA_EXTRACTED_TEST_SET</code> provides additional functionality for extracted tests. 

Notice that the call to the target routine <code>{BANK_ACCOUNT}.withdraw</code> is effected in the routine <code>test_withdraw</code> which passes an agent representing <code>{BANK_ACCOUNT}.withdraw</code> to the procedure <code>run_extracted_test</code>. The second argument to <code>run_extracted_test</code> is a <code>TUPLE</code> with the argument values which were used in the call to <code>withdraw</code> which caused the original assertion violation.

Another thing worth noting is the function <code>context</code>. This is how AutoTest recreates the state of the instance of <code>BANK_ACCOUNT</code> at the time of the assertion violation. 

{{caution|The extracted test recreates the state at the point at which execution has halted. So, in the case of a postcondition or invariant violation, the values of the attributes will reflect any changes that have been made during the execution of the routine. (In the example, the value of balance is set to 400, rather than 500 as it would have been when routine <code>withdraw</code> began execution.) This could make a difference in whether the test extracted after an exception is a valid recreation of the original failure. One way of dealing with this, at least in simple cases like this, is to change the test class code to reflect the proper value. A safer way would be rather than extracting the test after the exception, restart the system and stop execution as it enters the failing routine, then extract the test at that point. }}