Locating Real Faults in Object Programs

Where possible, we would like to locate existing faults in object programs and versions.

There are several potential sources of information on these.

  1. Look for any change logs (like RCS change logs, or anything else) that might have come with the distributions, or that are available on the web sites where the code was obtained. These can list all sorts of things, but some of what they list may be faults.
  2. Carefully examine the differences (using diff or cmp) between versions, looking for code changes that may be for fault correction.
  3. Check outputs. It probably won't be feasible (or even very productive) to look at all test outputs, but certainly if there are core dumps, these suggest the presence of faults, and these should be found.

For a fault to be useful to us, we need to know a few things about it, which you need to determine:

  1. where and what the fault is -- that is, the code associated with the fault (might be code in a few places)
  2. a test input that reveals the fault
  3. what versions the fault is in. Even if it is in every version it is useful to us. (This is another reason to look for change logs. If you found some more recent versions of grep, they might have change logs that describe errors that exist in all of our versions.)
  4. a judgement about the fault's severity. Do this on a scale of 1-3.
    1. catastrophic (crashes the system)
    2. normal (anything else not cosmetic)
    3. cosmetic (causes only a small problem, perhaps an output problem, spelling error or some such)
  5. You should also keep track of where you learned about the fault; sometime we might need to look back at change logs.

All of this assumes that you'll keep some sort of written document detailing this information about each fault you find. Since we have had no success in this with our newer objects, we haven't yet defined the format or location for such a document.