Resolving Nondeterminism Problems

Here are some problems with nondeterminism and stdout/stderr that we've run into, and solutions we've used to address them.

  1. If the program contains printf statements to stderr, we make them write to stdout instead. One way to do this is add:

    #define STDERR STDOUT

    to the top of the program.

  2. If the program uses libraries or macros that write to stderr, that's a problem that needs to be addressed case by case.

    One example is assert. Replace all assert(expr) with myassert(expr) and add: 

    int assert_fn(char * file, int line)
{
    printf("%s: %i\n", file, line);
    exit(0);
}
#define myassert(EX) {(EX) || assert_fn(__FILE__, __LINE__);}

    to the program.

  3. If the program uses a random number generator, fixing the seed can make it deterministic.
  4. If the program uses dates or terminal ids or other information that differs from run to run (e.g. date stamps), we replace printf statements of such information with printf statements that print out deterministic placeholder strings. An example of doing this with a Java subject can be found in the non-deterministic Ant and JUnit test runner, sir_junit.tar.gz, provided under the Download Tools link.
  5. If the program uses the executable file path or executable file name to output messages then there can be problems when the two executable files reside in different directories or when the names of executable file differ. For example, you can have non-determinism when comparing the outputs generated by the instrumented executable and non-instrumented executable. This is because the name of the instrumented executable file (foo.int.exe) differs from that of the non-instrumented executable (foo.exe).

    One way to remove this problem is to change the argv[0] inside the printf statements to some common name across all the versions like "Executable".