eiffel-org/documentation/current/eiffelstudio/eiffelstudio-guided-tour/using-autotest/create-manual-test.wiki

[[Property:title|Create a manual test]]
[[Property:weight|2]]
[[Property:uuid|e78f25e3-ed3a-f8fa-e71d-28a4dda1825f]]
==A system to test==

For developing our manual test, let's use a simple system that contains a class modeling bank accounts. Here are two classes that will make up our system. The first, <code>APPLICATION</code> will be the root class of our system. <code>APPLICATION</code> really only serves to declare an attribute of type <code>BANK_ACCOUNT</code>, which is the class we will write a test against. <code>APPLICATION</code> looks like this:

<code>
class
    APPLICATION

inherit
    ARGUMENTS

create
    make

feature {NONE} -- Initialization

    make
            -- Run application.
        do
        end

    my_account: BANK_ACCOUNT

end
</code>


And here's the class <code>BANK_ACCOUNT</code>:

<code>
class
    BANK_ACCOUNT
inherit
    ANY
        redefine
            default_create
        end
feature
    default_create
        do
            balance := 0
        end

    balance: INTEGER

    deposit (an_amount: INTEGER)
            -- Deposit `an_amount'.
        require
            amount_large_enough: an_amount > 0
        do
        ensure
            balance_increased: balance > old balance
            deposited: balance = old balance + an_amount
        end

    withdraw (an_amount: INTEGER)
            -- Withdraw `an_amount'.
        require
            amount_large_enough: an_amount > 0
            amount_valid: balance >= an_amount
        do
            balance := balance - an_amount
        ensure
            balance_decreased: balance < old balance
            withdrawn: balance = old balance + an_amount
        end

invariant
    balance_not_negative: balance >= 0
end
</code>

You shouldn't let it worry you if you've noticed that the class <code>BANK_ACCOUNT</code> contains some flaws. We'll deal with these later.

If you want to work along with this tutorial, you should be able to copy the text of each these classes from this page and paste it into the EiffelStudio editor pane. Build a system using these two classes, and <code>{APPLICATION}.make</code> as the root.


{{note|If you are using EiffelStudio version 6.3, there two things you will need to do to prepare your system for use with AutoTest. Both of these are done from the [[EiffelStudio: Project settings window]].<br/> 1) Set your project to be a console application in the [[Advanced options]].<br/>2) Set a value of <code>False</code> for the <code>Recursive</code> attribute of your project cluster in [[Group options]].}}

==Getting to the AutoTest interface==

If the AutoTest interface is not on a tab next to Clusters, Features, and Favorites, you can invoke it by following the menu path:
<code lang=text>
View --> Tools --> AutoTest
</code>
Depending upon your version and platform, the AutoTest interface should look about like this:


[[Image:AutoTest empty tool 01]]


==Creating a new test==

To begin the process of creating a new test, click the Create New Tests button ( [[Image:create new tests]] ) on the interface's tool bar. 

This will launch the New Eiffel Test Wizard which guides you through the test creation process.

If this is the first time you've used the testing tool for this project, it is likely that you will be presented with a dialog box asking if you want to add the testing library classes to your project and recompile:


[[Image:AutoTest add testing libraries dialog]]

You want EiffelStudio to do this before launching the wizard so, click "Yes". In a moment your system will have recompiled with the testing library classes available. Remember that you won't need to interact much with the testing classes, but AutoTest uses them, so they need to be available. As long as the testing classes stay available, you should not see this dialog again for the current project.


==The New Eiffel Test Wizard==

After the compile completes, then the first pane of the New Eiffel Test Wizard appears. It should look like this:


[[Image:AutoTest new test wizard 01 01]]


As you can see, there is a radio button for each of the three types of tests that were discussed in [[Testing: Background and basics]]. The buttons for '''Manual''' and '''Generated''' tests are enabled. The button for '''Extracted''' tests is not sensitive because the extraction of tests is only valid during the time in which a system is actually running. That's okay, we were going to choose '''Manual''' anyway.

If we select '''Manual''' and click '''Next''', then second pane of the wizard appears:


[[Image:AutoTest new test wizard 02M 01]]


We will use this wizard pane to name our test class and let AutoTest know where we want the test class to reside. You can give a test class any name you wish, as long as it doesn't conflict with another class name in your system. If you try to type in a class name that already exists, the wizard will let you know right away by changing the text color to red. There is a convention that has arisen around test class names. If possible make the test class name the name of the target class, prefixed with <code>TEST_</code>. So in our case, we want to build a test against a feature of the <code>BANK_ACCOUNT</code> class, so we will name our test class <code>TEST_BANK_ACCOUNT</code>.

Now, for the question of where the tests should be kept. Again, you can keep them anywhere you wish, but there are some advantages to keeping the test classes in a '''test cluster''' separate from your target classes. For example, it will be easier for you to deliver your application or library classes if the testing classes are mixed it. A '''test cluster''' is just a cluster of classes that EiffelStudio and AutoTest expect to contain test classes. So we want to create a new testing cluster as a subcluster of the cluster in which the classes <code>APPLICATION</code> and <code>BANK_ACCOUNT</code> reside. 

Notice the '''New cluster''' button ( [[Image:16x16--new-cluster-icon]] ) on the wizard pane above. We click that button to add a new test cluster. The '''Add Cluster''' dialog box appears:


[[Image:AutoTest new test wizard Add Cluster 01]]


As with test class names, there is an emerging convention for the name of a test cluster: <code>tests</code> . So we will name our test cluster <code>tests</code> and make it a subcluster to our root cluster <code>accounts</code>. Notice that there is a '''test cluster''' check box on the dialog. It is checked and disabled, so at this point in the wizard you would always create a test cluster. Once the test cluster is created, we're back to the second wizard pane which now looks like this: 


[[Image:AutoTest new test wizard 02M 02]]


We're ready to click '''Next''', but before we do, let's look at the check boxes on this wizard pane. The two check boxes labeled '''Redefine `on_prepare'''' and '''Redefine `on_clean'''' have to do with the way that tests are run. 

AutoTest runs each test as a three step process:
# Preparation
# Execution
# Clean up

There are features in class <code>EQA_TEST_SET</code> named <code>prepare</code> and <code>clean</code> which accomplish steps 1 and 3 above. These features are <code>frozen</code>, therefore you cannot redefine them in a test class (i.e., a descendant of <code>EQA_TEST_SET</code>) However the class does provide features that can be redefined so that you can include custom behavior before and/or after the execution of a test. These features are <code>on_prepare</code> and <code>on_clean</code>. So if you check one of these boxes, then the test class that is built for you will include a redefined feature ready for you to implement. In this simple example, we'll leave both boxes unchecked. 


{{note|The check box labeled '''System level test''' is displayed here as not sensitive. This box is reserved for future system level testing capability in AutoTest, so for versions including 6.5, you can ignore it. }}


After clicking '''Next''', we see the third pane of the wizard. In this pane, we will describe our manual test routine. It looks like this:


[[Image:AutoTest new test wizard 03M 01]]


Here we will name our test. Let's say that we plan to write this test against the feature <code>{BANK_ACCOUNT}.deposit</code>. Like the naming convention for test classes, we'll give this test the name <code>test_deposit_01</code>. The prefix <code>test_</code> before the feature name it will test, and the suffix <code>_01</code> so that we have a framework for adding more tests against <code>deposit</code>. Again, you can choose any naming scheme that makes sense to you. You may want to try to describe the test in its name. For example, <code>test_deposit_very_large_amount</code>.


==About Tags==

The other thing that can be done on this pane is to associate our test with any AutoTest '''tags''' that we feel are appropriate. 

'''Tags''' are simply names or otherwise meaningful strings of characters that are arranged hierarchically and can be associated with a test to help manage, maintain, execute, and monitor its results. Any one test can support many tags. It is quite likely that during the development process, your system may eventually accumulate a great number of tests. And you may want only to execute some selected portion of those tests at any particular time. '''Tags''' allow you do that with the help of AutoTest.

One of the most common types of tags specifies what class and feature a test covers. In our example, we will write our test against the <code>deposit</code> procedure of the class <code>BANK_ACCOUNT</code>. As you will see in a moment, the tag that will express this is:
<code>
covers/{BANK_ACCOUNT}.deposit
</code>
When we look at a tag in this notation, each hierarchical level is delimited by the forward slash. So the tag above specifies a root "covers" and its child "{BANK_ACCOUNT}.deposit". If this same test tested both <code>deposit</code> and <code>withdraw</code>, then its list of tags would be:
<code>
covers/{BANK_ACCOUNT}.deposit
covers/{BANK_ACCOUNT}.withdraw
</code>
So when ever you ask to view or run all the tests that <code>covers</code> either <code>deposit</code> or <code>withdraw</code>, this test would show up in that set. 

The "covers" tags, as you will see, can be generated by AutoTest's New Eiffel Test Wizard when you create a new test. But you could enter the tag manually, as well. For example if you had written a high level test that exercised all or most of the functionality of the class <code>BANK_ACCOUNT</code>, you could manually add a tag that expresses that, i.e., a "covers" tag for <code>BANK_ACCOUNT</code> that does not specify a particular routine:
<code>
covers/{BANK_ACCOUNT}
</code>

Tags can be completely arbitrary, too. So, for example if you were building software that you expected to run on multiple platforms, in the test suite, you might have a test with the following tags:
<code>
platform/os/linux
platform/architecture/i386
</code>
So this test would be specifically for Linux running on Intel architecture. When you were testing on that platform combination, you could select the appropriate tests to run using tags.


===Associating tags with a new test===

Now that we've named our new test, let's associate a tag with it that indicates that it <code>covers</code> the <code>deposit</code> procedure of class <code>BANK_ACCOUNT</code>. Looking again at the New Eiffel test wizard pane, you will see that there are three boxes under the label '''Tags for new tests'''. The first is just a display of the list of tags that you have added to the new test. The next box down allows you to add an arbitrary tag sequence like:
<code>
platform/os/linux
</code>
And the third allows you to add certain commonly used or predefined tag types. This is the box we'll use. So first we will select '''Add class/feature under test tag''', then click the '''Add''' button to the right of the box. This will cause the appearance of a dialog that allows us to pick a target class and routine from our system. So we navigate to <code>{BANK_ACCOUNT}.deposit</code>. The dialog will look like this:


[[Image: AutoTest new test wizard 03M dialog 01]]


Now we click '''OK'''. The dialog disappears and the <code>covers</code> tag:
<code>
covers/{BANK_ACCOUNT}.deposit
</code>
is now visible in the list of tags for the new test we are creating. So, next click '''Create''' and the wizard pane disappears and AutoTest will create our test class and display it in the edit window.


===Other predefined tags===

In addition to '''Add class/feature under test tag''', choices for other predefined tags are shown in the drop-down box. For example, the choices for '''execution''' tags are '''Run test in private evaluator''' and '''Run test serially'''. 

Selecting '''Run test in private evaluator''' will insert the tag:
<code>
execution/isolated
</code>

When tests are executed, they do so within the context of '''evaluator processes'''. Normally, evaluator processes are reused for multiple test executions. But if you select '''Run in private evaluator''', the tag added to your test guarantees that this test will be run in a fresh evaluator process, that terminates when the test completes. This can be helpful, for example, when you don't want your test to enter or leave the evaluator process with the effects of "once" routines or any other action that might effect the efficacy of other tests. For example, if your test executes external routines which might have a damaging effect on memory, you should run the test in a private evaluator.

If you select '''Run test serially''', the following tag will be inserted:
<code>
execution/serial
</code>

Tests tagged with this tag will not run concurrently with any other similarly tagged test is running.

You can extend the serial execution tag with arbitrary terms that will differentiate groups of tagged tests. For example, if some of your tests  are tagged like this:
<code>
execution/serial/group_1
</code>
and some are tagged:
<code>
execution/serial/group_2
</code>
then AutoTest will not run any <code>group_1</code> tagged test concurrently with any other <code>group_1</code> test, and likewise for tests tagged <code>group_2</code>.


==Writing a test==

Let's look at the class <code>TEST_BANK_ACCOUNT</code>:

<code>
note
    description: "[
        Eiffel tests that can be executed by testing tool.
    ]"
    author: "EiffelStudio test wizard"
    date: "$Date$"
    revision: "$Revision$"
    testing: "type/manual"

class
    TEST_BANK_ACCOUNT

inherit
    EQA_TEST_SET

feature -- Test routines

    test_deposit_01
            -- New test routine
        note
            testing:  "covers/{BANK_ACCOUNT}.deposit"
        do
            assert ("not_implemented", False)
        end

end
</code>

We can see that the feature <code>test_deposit_01</code> exists, but doesn't really test anything. So, let's change that. We'll alter <code>test_deposit_01</code> so that it creates an instance of <code>BANK_ACCOUNT</code> and then makes a deposit to that account. 

So, <code>test_deposit_01</code> now looks like this:

<code>
    test_deposit_01
            -- New test routine
        note
            testing:  "covers/{BANK_ACCOUNT}.deposit"
        local
            l_ba: BANK_ACCOUNT
        do
            create l_ba
            l_ba.deposit (500)
        end 
</code>

Now we have created and written a manual test using AutoTest.

Next let's see what it takes to execute a test.