Private API Access and Functional Mocking in Automated Unit Test Generation

Not all object oriented code is easily testable:
Dependency objects might be difficult or even impossible to
instantiate, and object-oriented encapsulation makes testing potentially
simple code difficult if it cannot easily be accessed.
When this happens, then developers can resort to mock objects
that simulate the complex dependencies, or circumvent objectoriented
encapsulation and access private APIs directly through
the use of, for example, Java reflection. Can automated unit test
generation benefit from these techniques as well? In this paper
we investigate this question by extending the EvoSuite unit test
generation tool with the ability to directly access private APIs
and to create mock objects using the popular Mockito framework.
However, care needs to be taken that this does not impact the
usefulness of the generated tests: For example, a test accessing a
private field could later fail if that field is renamed, even if that
renaming is part of a semantics-preserving refactoring. Such a
failure would not be revealing a true regression bug, but is a
false positive, which wastes the developer’s time for investigating
and fixing the test. Our experiments on the SF110 and Defects4J
benchmarks confirm the anticipated improvements in terms of
code coverage and bug finding, but also confirm the existence of
false positives. However, by ensuring the test generator only uses
mocking and reflection if there is no other way to reach some
part of the code, their number remains small.