Java Object Directory Specification

Each Java object we create has its own object directory, named to indicate the program involved.

An object directory is organized into subdirectories (which in turn may contain subdirectories) according to a specific format. Doing this enables us to write generic tools for experimentation that know where to find things and know what they're named, and work across all of our objects.

This document describes the object directory organization in terms of its subdirectories, and what they contain.

This organization is used for objects where we have a sequence v0, v1, v2... vK of versions, where each vK+1 is a derivative of the previous version vK.

In general, any directory or subdirectory that contains files whose function may not be obvious, or that depart from naming conventions, should contain a CONTENTS file describing those files.

The subdirectories of an object are as follows:

  1. source

    An empty directory. During experiments, a version of a program that we're working with is placed in here, then deleted after we're finished with it.

  2. versions.alt

    Contains "variants" of the source code of the versions of the program. Variants are needed when, say, we have to modify the code to accomodate some tool.

    The basic variant, that any object will have, is contained in the subdirectory of versions.alt called:

    orig

    This directory contains subdirectories

    v0, v1, v2, v3 ... vK

    where v0 is the earliest version,and each directory vJ contain the next sequential versions in turn.

    A second variant may exist when we have seeded faults, this is called:

    seeded

    This directory's organization is the same as that of the .orig directory, except that it contains additional files required to build the object with seeded faults activated. Please see Section 5 of the Java object handbook for detailed information regarding the activation of seeded faults.

    If an object consists of a component and an application, then the versions directory will contain additional subdirectories following the pattern "component[K]" and "application[K]" where K may or may not be present and K >= 1. Each subdirectory contains .orig and .seeded subdirectories as explained above.

    CONTENTS files should describe the reasons for other variants.

  3. versions

    An empty directory, used during some experiments to hold either a version, or a directory of versions, used in an experiment.

  4. testplans.alt

    Testing information for objects.

    This directory contains subdirectories v0, v1, ... vK, each of which contains testing information for that version. The testplans.alt directory may also contain a testscript subdirectory.

    To avoid keeping multiple copies of identical files, we have organized the vJ subdirectories into three types of files:

    • General Files - .tsl, .frame and .universe files as follows:
      • tsl specification files (.tsl extension)

        These files are named vK_n.tsl or vK.tsl where K=0,1,2... for different versions and n=1,2,3... for the different tsl specs. For example, if we are in version v0 and there are two tsl specification files then these files will be named v0_1.tsl and v0_2.tsl.

      • sets of test frames generated from .tsl files (.frame extension)

        These files are named vK_n.frame or vK.frame, generated from vK_n.tsl or vK.tsl files respectively. Here K=0,1,2... for different versions and n=1,2,3... for tsl specs. For example, the frame file generated from vK_1.tsl will be named vK_1.frame.

      • STImpL files (.tsl.universe, .cov.universe, or .junit.universe extension)

        A STImpL file is a file listing your test cases. A program called "make-test-script" (mts) transforms STImpL files into various types of scripts that can be used to automatically execute the tests or gather traces for the tests. There are two types of STImpL files:

        • vK_n.tsl.universe or vK.tsl.universe where K=0,1,2,... for different versions and n=1,2,3,... for different tsl specification files. These files contain test cases generated from tsl test frames.
        • vK_n.cov.universe or vK.cov.universe where K=0,1,2.... for different versions and n=1,2,3... for different coverage files. These files contain additional tests that provide some additional code coverage.
        • vK.class.junit.universe where K=0,1,2,... for different versions for JUnit test cases.

        Depending on the object, a STImpL file consists of a set of test case files, not a list of test cases. That is, each test case exists as a separate file. An example can be found in the siena object.

    • test suite directories (containing multiple test suites, typically these just contain a subset of the lines in some STImpL files).

      These may be tarred and gzipped due to size. A CONTENTS file should identify the meaning of these directories.

    • testscript subdirectory

      If there are any startup or exit scripts present in the STImpL file (defined below) then they are stored in this directory.

    CONTENTS files should be provided that will distinguish the different tsl specifications and STImpL files.

  5. testplans

    An empty directory, used during some experiments to hold test suites needed by that experiment.

  6. traces.alt

    Contains subdirectories v0, v1, ... vK for the versions of the object; these subdirectories may hold trace files.

  7. traces

    An empty directory, used during some experiments to hold traces.

  8. inputs

    Files containing inputs or startup directories used by programs in various tests. These could be grouped into subdirectories or distinguished by naming conventions if helpful.

  9. outputs.alt

    Provides a place to permanently store the outputs of test runs. This is useful for regression testing where new outputs are compared ('diffed') against previous outputs.

    This directory could contain subdirectories v0, v1, ... vK to differentiate between versions.

  10. outputs

    An empty directory, used during some experiments to hold test outputs.

  11. docs

    Documentation about the object. Use any subdirectories that are useful. Possibly, v0, v1, ... vK directories will help.

  12. originals

    Original materials obtained on the objects. Subdirectories are used if necessary (e.g. v0, v1, ... vK), and a CONTENTS file will describe the purpose of the subdirectories.

  13. scripts

    The directory from which experiments are run. This directory may also contain saved scripts, such as runall.sh and gettraces.sh scripts that run all tests in the STImpL specification. Subdirectories can be used if helpful (e.g. v0, v1, ... vK containing scripts for each version).

  14. info

    Other information about the program, especially information gathered by analysis tools and requiring saving for experiments. In particular, we have found it useful to include subdirectories v1, ... vK and "common" to store version-specific and object-specific information, respectively.

    The principle information to be found in the version-specific directories is the fault matrix data, as follows:

    Each vK subdirectory in info may contain fault matrix information for that version. The fault matrices have the following naming convention:

    1. If a fault matrix file is generated from a single STImpL file then it is named fault-matrix.<universe-file>. For example the fault matrix file generated by running on the v0_1.tsl.universe file will be named fault-matrix.v0_1.tsl.universe.
    2. If a fault matrix file is generated from the concatenation of two or more STImpL files then it is named fault-matrix.<universe-file-1>.<universe-file-2>.---.<universe-file-n> where <universe-file-1> is the name of the 1st STImpL file <universe-file-2> is the name of the 2nd STImpL file and so on.

    Further information on fault matrices and file names is provided in the main Java handbook.

  15. testdrivers

    Contains test programs which make the application run.