View article

While it is now relatively easy to produce large sets of software behaviors, eg, via automated test sequence or input generation, or by automatically executing and monitoring the executions of deployed software, it is hard to use the generated behaviors for finding business logic bugs. This is due to the fact that deciding, for a given system, if its executions correspond to correct/desired software behavior as opposed to defective or in some sense anomalous behavior, is an inherently manual task. This situation has caused the so-called oracle problem, ie the problem of effectively distinguishing correct from incorrect software behavior, to receive significant attention by the software engineering community [5]. Still, with the current state-of-the-art, precise software oracles largely depend on manually produced specifications, or when automatically synthesized, often involve implicit oracles that only check very general properties, such as deadlock freedom and no null dereference.

To better deal with the oracle problem, metamorphic testing has been proposed [32, 43, 44]. Instead of manually describing the expected behavior for every single execution scenario, a weaker form of oracle that applies to a large family of executions is provided at once. This oracle is known as a metamorphic relation, and describes an expected property that relates two or more elements of the software under test (SUT)[43, 44]. The resulting “metamorphic” oracle is in general weaker than a precise and complete specification of the software under analysis, but at the same time is also easier to define and provide. Moreover, metamorphic properties capture properties that are …