MockDetector: Detecting and tracking mock objects in unit tests

Abstract

Unit testing is a widely used tool in modern software development processes. A well-known issue in writing tests is handling dependencies: creating usable objects for dependencies is often complicated. Developers must therefore often introduce mock objects to stand in for dependencies during testing. Test suites are an increasingly important component of the source code of a software system. We believe that the static analysis of test suites, alongside the systems under test, can enable developers to better characterize the behaviours of existing test suites, thus guiding further test suite analysis and manipulation. However, because mock objects are created using reflection, they confound existing static analysis techniques. At present, it is impossible to statically distinguish methods invoked on mock objects from methods invoked on real objects. Static analysis tools therefore currently cannot determine which dependencies' methods are actually tested, versus mock methods being called. In this thesis, we introduce MockDetector, a technique to identify mock objects and track method invocations on mock objects. We first built a Soot-based imperative dataflow analysis implementation of MockDetector. Then, to quickly prototype new analysis features and to explore declarative program analysis, we created a Doop-based declarative analysis, added features to it, and ported them back to the Soot-based analysis. Both analyses handle common Java mock libraries' APIs for creating mock objects and propagate the mock objects information through test cases. Following our observations of tests in the wild, we have added special-case support for arrays and collections holding mock objects. On our suite of 8 open-source benchmarks, our imperative dataflow analysis approach reported 2,095 invocations on mock objects intraprocedurally, whereas our declarative dataflow approach reported 2,130 invocations on mock objects (under context-insensitive base analyses in intraprocedural mode), out of a total number of 63,017 method invocations in test suites; across benchmarks, mock invocations accounted for a range from 0.086% to 16.4% of the total invocations. Removing confounding mock invocations from consideration as focal methods can improve the precision of focal method analysis, a key prerequisite to further analysis of test cases.