User Documentation for MakeTestScript

Subject Infrastructure Repository (SIR)
May 04, 2006


Overview
--------------------

MakeTestScript is a Java implementation of the old 'mts' and 'javamts' tools
for creating executable test scripts from STImpL files (previously known as
universe files). A STImpL file contains information describing how test cases
in a test suite are to be implemented through scripting commands. Full details
regarding the STImpL file format can be found on the SIR website. This Java
implementation provides many enhancements, enabling considerably more control
over the script generation process and the way in which test case
implementations are encoded in script files.


Installation
--------------------

1. Add the jar file 'mts.jar' found in the 'bin' directory of this distribution
   to your CLASSPATH.

2. Add the jar file 'antlr-2.7.6.jar' found in the 'lib' directory of this
   distribution to your CLASSPATH. You may ignore this step if you already have
   a valid path to an ANTLR distribution on your CLASSPATH.
 
3. If you intend to use the wrapper scripts (see below), add the appropriate
   directory to your PATH. Ignore this step otherwise.

You are now ready to use MakeTestScript.


Using MakeTestScript
--------------------

The command to invoke MakeTestScript is as follows:

java sir.mts.MakeTestScript --stimple-file|-sf <file> --script-name|-sn <name>
    --exe-name|-en <name> [--experiment-dir|-ed <path>
    [--compare-outputs|-c <comp_dir> [comp_cmd>]] [--compare-type|-ct D|d]
    [--trace|-t] [--trace-source-dir|-ts <path>] [--trace-name|-tn <name>]
    [--invoke-prefix|-ip <prefix>] [--invoke-suffix|-is <suffix>]
    [--exe-prefix|-ep <prefix>] [--exe-suffix|-es <suffix> [D]] [--java|-j]
    [--target|-tg <script_gen_class>] [--no-escapes|-nesc] [--legacy-macros|-lm]
    [--print-config|-pc] [--version]

Parameters in square brackets ('[', ']') are optional. Items in angle brackets
('<', '>') are required data for associated parameters. A bar ('|') indicates
a choice among options. The function of each parameter is described below.

    --stimple-file | -sf <file> :
        Specifies the STImpL (universe) <file> from which to generate a test
        script. This parameter must be provided.

    --script-name | -sn <name> :
        Specifies the <name> of the test script file to be generated. This
        parameter must be provided.

    --exe-name | -en <name> :
        Specifies the <name> of the executable program under test in the
        generated script. This parameter must be provided.

    --experiment-dir | --ed <path> :
        Specifies the <path> to the directory containing the subject
        infrastructure for the program under test. If absent, defaults to "..",
        which enables scripts to execute correctly if run from the
        "scripts" directory of an SIR subject.

    --compare-outputs | -c <comp_dir> [comp_cmd] :
        Specifies that the generated script will be an output comparing script.
        Such a script will compare the output files recorded from stdout and
        stderr during program execution against the same output files from
        a previous run at a specified location. Files that are moved using
        STImpL '-F' commands are also compared.
            <comp_dir> is required and specifies the path to the directory
                containing the old files against which current test case
                outputs should be compared by the script.
            [comp_cmd] is an optional parameter specifying the command to be
                used for file comparison. Options to the command can be
                provided; parameters will be consumed from the command line
                until another valid MakeTestScript parameter is encountered
                or the end of the parameter list is reached. Alternately,
                it can be provided as a single quoted parameter.

    --trace | -t :
        Specifies that the generated script will collect traces.

    --trace-source-dir | -ts <path> :
        Specifies the <path> to the directory from which the generated script
        will attempt to copy trace files. Ignored if the script is not to be a
        trace collecting script. If "--trace | -t" is specified but this
        parameter is absent, MakeTestScript will attempt to guess an appropriate
        trace source directory based on other parameters and historical
        assumptions from 'mts' and 'javamts'.

    --trace-name | -tn <name> :
        Specifies the <name> of the trace file that will be copied from the
        trace source directory if this script is a trace collecting script,
        ignored otherwise.

    --invoke-prefix | -ip <prefix> :
        Specifies a <prefix> to be inserted at the front of test case
        invocations in the generated script. This is the preferred way of
        specifying the virtual machine to be used for Java subjects, by
        specifying a prefix such as "java" or "jikes". It may also be used in
        conjunction with "--invoke-suffix | -is" to achieve other effects.
        For example, in many shells, specifying an invoke prefix of "(" and
        invoke suffix of ")" will cause the test case to be run in a subshell.

    --invoke-suffix | -is <suffix> :
        Specifies a <suffix> to be appended at the end of test case
        invocations in the generated script.

    --exe-prefix | -ep <prefix> :
        Specifies a <prefix> to be concatenated to the start of the
        executable program name. The prefix will be directly concatenated
        without whitespace.

    --exe-suffix | -es <suffix> [D] :
        Specifies a <suffix> to be concatenated to the end of the executable
        program name. The suffix will directly concatenated without whitespace.
            [D] An optional parameter that instructs MakeTestScript to only
                append the suffix to the executable name for test cases that
                specify a test driver (the '-D' STImpL command). This is
                provided primarily to achieve backward compatibility with the
                implementation of the <inst_option> parameter to the old
                'javamts' tool.

    --java | -j :
        Indicates to MakeTestScript that the subject program under test is
        a Java subject. MakeTestScript will set defaults for a number of
        parameters to emulate the old 'javamts' tool. Using this parameter
        is a shorthand for the specifying the following to MakeTestScript:
            --invoke-prefix "java"
            --trace-source-dir '${SOFYA_DB_DIR}'
            --trace-name "instout.tr"

    --target | -tg <bsh|csh|<script_gen_class>> :
        Specifies the target language of the script to be generated.
            "bsh" instructs MakeTestScript to generate a Bourne shell script.
            "csh" instructs MakeTestScript to generate a C shell script.
            <script_gen_class> instructs MakeTestScript to attempt to load the
                specified class as a plugin and use it to generate the output
                script. The specified class must implement the
                'sir.mts.TestScriptGenerator' interface, and provide a public
                constructor that accepts a single argument of type
                'sir.mts.Configuration'.

    --no-escapes | -nesc :
        Disables the recognition of backslash character escapes by the
        MakeTestScript lexer. This is intended only to provide backward
        compatibility with existing universe files, when necessary.

    --legacy-macros | -lm :
        Enables recognition of a legacy substitution macro '%n%' that was used
        to insert the current test case number at the given location. Handling
        of this substitution is deferred to the script generator. Very few
        should require use of this parameter.

    --print-config | -pc :
        Causes MakeTestScript to print the configuration after all parameters
        have been parsed and before beginning processing of the STImpL file.

    --version :
        Displays the MakeTestScript version number.


Wrapper Scripts
--------------------

The 'bin' directory contains subdirectories 'bsh' and 'csh'. These directories
contain bash shell scripts that exactly emulate the old 'mts' and 'javamts'
interfaces. You should update old invocations of 'mts' and 'javamts' to use
the new MakeTestScript interface as soon as possible. However, in the mean
time you may add one of these directories to your PATH and any existing
scripts or programs that call 'mts' or 'javamts' will continue to work.

IMPORTANT: There is one configuration step you must perform before attempting
to use the wrapper scripts. Edit the scripts and set variables marked
'SET THIS VARIABLE' to correct values for your system before attempting to
use the scripts.

The 'bsh' directory contains wrapper scripts to emulate the behavior of
mts/javamts when compiled to produce Bourne shell scripts. The 'csh' directory
contains wrapper scripts to emulate the behavior of mts/javamts when
compiled to produce C shell or TCsh scripts.
