A Look at the Design of JUnit
See also: A Look at the Evolution of JUnit

Hayden Melton
http://www.cs.auckland.ac.nz/~hayden
Department of Computer Science
University of Auckland
New Zealand

Overview

On another page I argued that it is useful to view a software system as a directed graph, the nodes of which are source files and the edges of which are compilation dependencies. I argued that such graphs should be acyclic, and "flatter" rather than "taller". I argued that it is preferable for a source file to depend only on externally-declared types (e.g. types out of the standard Java API), so long as this does not comprise encapsulation, abstractions, type-safeness and other important design principles.

In this document I look at the design of the framework package (i.e., the classes in junit.framework.*) of JUnit-3.8.2. I suggest that the existing design of the (classes in) framework package is difficult to understand because of the cycles among the classes in it. I also present a refactored version of these classes that I believe is easier to understand. The refactored version breaks the compatibility the framework package has with existing classes in other packages of JUnit, but (I believe that) none of the important functionality is compromised in the refactored version.

1.0 Existing Design

Figure 1 shows the relationships between the source files in the framework package of JUnit-3.8.2. These classes do not depend on any other source files outside the framework package.

Figure 1: Dependencies between classes in framework package of JUnit-3.8.2

Clearly there are cycles among the classes in the existing design of the framework package. I think this is bad, because it makes the package harder to understand incrementally. In terms of particular cycles, there is a 3-cycle involving Assert, ComparisonFailure and ComparisonCompactor; there is a 2-cycle among Test and TestResult; another 2-cycle among TestCase and TestResult; and so on. In the past I have found it useful to characterise cycles in terms of Strongly Connected Components (SCCs) (i.e. classes that are all mutually reachable from one another). Table 1 shows the classes involved in SCCs in the framework package.

Table 1: Classes from the framework package of JUnit-3.8.2 involved in SCCs

2.0 Modified Design

I have refactored the framework package so that it does not contain any cycles. The dependency graph of the refactored version is shown in Figure 2.

Figure 2: Dependencies between classes in the refactored framework package

The least controversial refactoring was to move the static method format from Assert to ComparisonCompactor. This broke the smaller of the SCCs in the original design. Now ComparisonCompactor can be understood, tested and reused completely independently from any other class in the package.

The more controversial refactorings were as follows:

• A single, logical "test" --- which I view as being a public method, with no parameters and a void return type, whose name starts with the string "test" --- is now represented by the class MethodClassPair. I thought I could represent it simply as a method (java.lang.reflect.Method) but quickly realised this wouldn't work. It won't work because a method can potentially have different effects depending on the runtime type it is invoked on. So I needed to store the runtime type (java.lang.Class) and method pair in a class in order to represent a single "test".
• I also removed the composite pattern among the original Test, TestCase and TestSuite [1]. I think that this pattern unnecessarily complicates things. I only ever run tests by adding them to a test suite. I seldom run a single logical test (as I have described above as being a single method prefixed with the string "test"), but with the refactored code this is still possible using a method on the refactored TestSuite.
• I thought that I could remove Test because I found myself with an interface that declared no methods after removing the composite pattern. Again I was wrong about this because when you recurse up an inheritance hierarchy using reflection to find methods that start with the string "test" you need to know where to stop. Otherwise you might inadvertently add a method "testFoo()" that's declared in a future version of, say, java.lang.Object. So the refactored version of Test is a "tagging" interface, much like java.io.Serializable.
• I refactored TestResult, which was involved in the "parameter collector" pattern [1], to be an listener. I renamed it  TestResultCollector because it collects a bunch of test results.Again I think this simplified the design. As far as I can tell TestResult was just some kind of cache for the output of a test --- this data could alternatively be stored in the user interface, if it is needed by the user interface at all.
• I eliminated the classes Protectable and TestFailure because I couldn't see the point of them. Protectable probably has something to do with concurrency, which in my refactored version I haven't bothered to support. I think I could change the refactored version to support concurrency relatively easily but I'm just too lazy to do so.

I think that pretty much sums up the changes I made. To run some tests with refactored JUnit framework you go something like this:

All that happens when you run the refactored version is that a bunch of (very unpretty) text gets spat out to stdout describing what methods are getting called, what tests are being invoked and what failures and errors are being thrown. There are possibly even bugs in the refactored version, but hey, it's an experiment in design not correctness.

I've made the original and refactored versions available for download here. The original version is in the package junit.framework and my refactored version is in the package junit.framework_refact. You can collect the data I have presented here using the tool Jepends-BCEL. The directed graphs shown on this page were drawn using the "dotty" layout algorithm of GraphViz.

3.0 Concluding Remarks

I'm not advocating that JUnit actually be refactored in the way that I have described on this page. Firstly, it has to deal with compatibility issues --- most users of JUnit in the world would be unwilling to adopt a version that will break all their existing test code. Secondly there might be features latent in the code that as a casual JUnit user (and not developer) I have overlooked. For instance, I know I've deliberately overlooked the concurrency stuff and the fact that JUnit-3.8.1 is meant to be compatible with Java 1.1.7. 

Also, in all fairness, the overall structure of JUnit's is very good compared to many software systems out there (see "An Empirical Study of Cycles among Classes in Java"). I just wanted to look at the structure of a real application w.r.t. cycles, and JUnit was small enough, and I believe I understood it well enough to do so. I may use the code provided here for a controlled experiment on the effect cycles have on understandability in the near future.

References

[1] A Cook's Tour of JUnit-3.8.x

Back to my research page