Software-artifact Infrastructure Repository: SPL Object Handbook: Object Directory Specification

SPL Object Directory Specification

Each SPL object we create has its own object directory, named to indicate the SPL packaged within.

An object directory is organized into subdirectories (which in turn may contain subdirectories) according to a specific format. Doing this enables us to write generic tools for experimentation that know where to find things and know what they're named, and work across all of our objects.

This document describes the object directory organization in terms of its subdirectories, and what they contain.

This organization is used for objects where we have a sequence v0, v1, v2... vK of versions, where each vK+1 is a derivative of the previous version vK. Because an SPL may have components provided by different development groups, the preparation of the source versions may be further segmented beneath each of these vK versions to reflect the relevant version of each component used in the SPL.

In general, any directory or subdirectory that contains files whose function may not be obvious, or that depart from naming conventions, should contain a CONTENTS file describing those files.

The subdirectories of an object are as follows:

source
An empty directory. During experiments, a version of a program that we're working with is placed in here, then deleted after we're finished with it.
versions.alt
Contains "variants" of the source code of that comprises the SPL components. Variants are needed to differentiate between the original SPL component set and the SPL components containing modifications for fault insertion or analysis tool support. The use of the term "variants" in this context should not be confused with the SPL specific terminology usage which refers to a feature that can be realized by a variation point in the SPL.

The basic "variant", that any object will have, is contained in the subdirectory of versions.alt called:

orig

This directory contains subdirectories

v0, v1, v2, v3 ... vK

where v0 is the earliest version of the SPL components used to construct the products, and each numerically sequential directory vJ contain the subsequent versions of SPL components in turn. This representation supports the variability in time property of an industrial SPL where subsequent versions have increased functionality or capability not available in previous versions.

Pictorially, the directory tree of the originals directory would be generally be arranged like this:
```
object_experiment_root_directory
|_______verisions.alt
        |________orig
                 |_______ v0
                 |        |_______component1_name
                 |        |       |_______ src
                 |        |                |______sub_componentA
                 |        |                |      |__________A.java
                 |        |                |
                 |        |                |______sub_componentB
                 |        |                       |__________B.java
                 |        \______application1_name
                 |               |________ src
                 |                         |______sub_componentA
                 |                         |      |__________A.java
                 |                         |
                 |                         |______sub_componentB
                 |                                |__________B.java
                 |_______ v1
                 |        |_______component1_name
                 |        |
                ...      ...
```
This is a generalized representation and other subdirectories may be located under each vK and src. The actual directory tree contained beneath each vK should be identical to the directory scheme provided by the original authors of the SPL and tailored to conform to the vK partitioning to separate the evolving SPL versions.

A second variant of the original SPL source will usually exist containing seeded or real faults. To be consistent with the format applied to the existing SIR Java and C objects this subdirectory is named:

seeded

This directory's organization is the same as that of the versions.alt/orig directory, except that it contains additional or modified files containing the faults to be activated. There may be additional subdirectories found under the seeded path to facilitate storing faulted source files by object version and fault number, depending upon how the faults are applied and activated. A generalized tree for the seeded directory would have the arrangement shown below:
```
object_experiment_root_directory
|_______verisions.alt
        |________seeded
                 |_______ v0   The v0 directory contains no faults
                 |        |    and reflects the original object
                 |       ...   directory layout
                 |
                 |_______ v1
                 |        |_______component1_name
                 |        |       |____component.c    Original v1 source
                 |        |       |____component.cpp  Faulted v1 source
                 |        |       |
                ...      ...     ...
```
The generalized tree assumes that the preprocessor method of fault application (seeding) is used. A short description of the directory tree used for the fault overlay method can be found here.

Additional information on fault activation procedures is given in Section 5 of the SPL object handbook.

In all cases, a CONTENTS files should describe the reasons for other variants.
versions
An empty directory, used during some experiments to hold either a version, or a directory of versions, used in an experiment.
configurations
A directory containing the files that define the composition of the products of the SPL.

Since an SPL can instantiate several products (variants) a means is needed to guide the build process in combining the appropriate features into the final packaged product. All SPLs we have experience with to date use a configuration file that describes the desired feature content, e.g. a flat text file, an XML file, a file expressing a domain specific language, or an Ant properties script. To allow composition of all products (variants) in an automated manner this directory stores the configuration files needed for each version of the SPL.

The configurations directory supports the evolution of SPL versions (variability in time) by providing subdirectories of the form:

v0, v1, v2, v3 ... vK

where each subdirectory will contain the configuration files for the products (variants) that can be built in that version of the SPL.
```
configurations
    |____________v0
    |            |_________ product-1_config_file
    |            |_________ product-2_config_file
    |            |_________ product-3_config_file
    |            |
    |           ...
    |
    |____________v1
    |            |_________ product-1_config_file
    |            |_________ product-2_config_file
    |            |_________ product-3_config_file
    |            |_________ product-4_config_file
    |            |
    |           ...
   ...
```
In use, the build script will copy the SPL source from the versions.alt/orig/vK or versions.alt/seeded/vK directory into the versions or source directory then copy the configuration file from the configurations directory into the proper location within either versions or source directory. The product can then be built in the versions or source directory for experimentation.

Since variability in products lines can be expressed through inheritance, extension, paremeterization, configuration and generation, the method employed by the SPL author should be reflected in the configurations/vK directories. This implies that an SPL having multiple components each using a configuration file to define the capabilites of that component would need to support a "sparse" directory tree populated with these configuration files in a directory hierarchy used in the original SPL directory. In using this approach, an overlay technique can be applied and these configuration specifying files can be automatically applied by scripting. Refer to the SPL Fault Overlay document for more on this "sparse" directory structure.
testplans.alt
Testing information for objects.

This directory contains the tests applicable to the SPL and may be futher divided into subdirectories v0, v1, ... vK, each of which contains testing information for that version. The testplans.alt directory may also contain a testscript subdirectory that contains scripts used to facilitate testing. Again the contents of this directory should be annotated with a CONTENTS to indicate the arrangement of testing artifacts and any special instructions on usage.

To avoid keeping multiple copies of identical files, we have organized the vJ subdirectories into three types of files:
- General Files - .tsl, .frame and .universe files as follows:
  - tsl specification files (.tsl extension)
    These files are named vK_n.tsl or vK.tsl where K=0,1,2... for different versions and n=1,2,3... for the different tsl specs. For example, if we are in version v0 and there are two tsl specification files then these files will be named v0_1.tsl and v0_2.tsl.
  - sets of test frames generated from .tsl files (.frame extension)
    These files are named vK_n.frame or vK.frame, generated from vK_n.tsl or vK.tsl files respectively. Here K=0,1,2... for different versions and n=1,2,3... for tsl specs. For example, the frame file generated from vK_1.tsl will be named vK_1.frame.
  - STImpL files (.tsl.universe, .cov.universe, or .junit.universe extension)
    A STImpL file is a file listing your test cases. A program called "make-test-script" (mts) transforms STImpL files into various types of scripts that can be used to automatically execute the tests or gather traces for the tests. There are two types of STImpL files:
    - vK_n.tsl.universe or vK.tsl.universe where K=0,1,2,... for different versions and n=1,2,3,... for different product (variants). These files contain test cases generated from tsl test frames.
    - vK_n.cov.universe or vK.cov.universe where K=0,1,2.... for different versions and n=1,2,3... for different products. These files contain additional tests that provide some additional code coverage.
    - vK_n.class.junit.universe where K=0,1,2,... for different versions and n=1,2,3... for different products. This provides support for SPLs that implement test suites using JUnit test cases.
    Depending on the object, a STImpL file consists of a set of test case files, not a list of test cases. That is, each test case exists as a separate file. An example can be found in the siena object.
- test suite directories (containing multiple test suites, typically these just contain a subset of the lines in some STImpL files).
  These may be tarred and gzipped due to size. A CONTENTS file should identify the meaning of these directories.
- testscript subdirectory (optional)
  If there are any startup or exit scripts present in the STImpL file (defined below) then they are stored in this directory.
CONTENTS files should be provided that will distinguish the different tsl specifications and STImpL files.
testplans
An empty directory, used during some experiments to hold test suites needed by that experiment.
traces.alt
Contains subdirectories v0, v1, ... vK for the versions of the object; these subdirectories is intended to hold test execution trace files.
traces
An empty directory, used during some experiments to temporarily hold trace files created during testing of an instrumented object.
inputs
Files containing inputs or startup directories used by programs in various tests. These could be grouped into subdirectories or distinguished by naming conventions if helpful. A CONTENTS file should be provided describing the input files and/or their usage for testing.
outputs.alt
Provides a place to permanently store the outputs of test runs. This is useful for regression testing and fault matrix building where new outputs are compared ('diffed') against previous outputs.

This directory could contain subdirectories v0, v1, ... vK to differentiate between versions.
outputs
An empty directory, used during some experiments to hold test outputs.
docs
Documentation about the SPL object. Appropriate subdirectories may be used if the evolution of the SPL documentation artifacts is available. Since an SPL requires (at minimum) a feature model or a similar meta-model to represent variability in the product family, this directory is intended to be the location this information will be stored. In addition to a feature model, an SPL may provide use cases, UML diagrams, viewtypes such as the architectural view and decision model, and component and connector diagrams. These documents should be provided in portable document format (pdf) or flat ASCII text (txt) files. Alternate formats (e.g. doc files) can be provided but these must also be provided as a pdf file for maximal portability.
originals
Original source materials obtained on the objects. Subdirectories are used if necessary (e.g. v0, v1, ... vK) to separate version evolution. A CONTENTS file will describe the purpose of the subdirectories found under this directory path.
scripts
The directory from which experiments are run. This directory will contain scripts that allow building both the orig and seeded versions of all SPL products. The directory may also contain saved scripts, such as runall.sh and gettraces.sh scripts that run all tests in the STImpL specification. Subdirectories can be used if helpful (e.g. v0, v1, ... vK containing scripts for each version). This directory can also be used as an alternate location for storing support tools that are used to enable or facilitate testing. A CONTENTS or README file will describe the scripts and support tools delivered in this directory and provide instructions on their use in building and testing the SPL versions.
info
Other information about the program, especially information gathered by analysis tools and requiring saving for experiments. In particular, we have found it useful to include subdirectories v1, ... vK and "common" to store version-specific and object-specific information, respectively.

The principle information to be found in the version-specific directories is the fault matrix data, as follows:

Each vK subdirectory in info may contain fault matrix information for that version. The fault matrices have the following naming convention:
1. If a fault matrix file is generated from a single STImpL file then it is named fault-matrix.<universe-file>. For example the fault matrix file generated by running on the v0_1.tsl.universe file will be named fault-matrix.v0_1.tsl.universe.
2. If a fault matrix file is generated from the concatenation of two or more STImpL files then it is named fault-matrix.<universe-file-1>.<universe-file-2>.---.<universe-file-n> where <universe-file-1> is the name of the 1st STImpL file <universe-file-2> is the name of the 2nd STImpL file and so on.
3. If an STImpL file is not provided with the SPL, the fault matrix data will reflect the test ordering of the test script used for that version and product.
Further information on fault matrices and file names is provided in the main Java handbook.
testdrivers
Intended to contain test programs needed to perform tests upon the SPL. If the SPL requires a particular program or framework to execute the tests this directory is a convenient place to put these. An alternate location for these can also be the scripts directory, particularly if this required program is used for basic operation of the SPL (e.g. an emulator platform.)