Back to Thoughts On Technology Page | Back to Peter's Home Page | Back to Family Home Page



Basics of Test First
Using Junit and Jmock

Peter Rose - 11/2006


ABSTRACT

SUMMARY

DEFINITIONS

BENEFITS OF PROCESS ADOPTION

SOME OTHER USEFUL TOOLS

SETTING UP

TEST FIRST

SOME STANDARDS

MOCK OBJECTS

THE PROCESS

EXAMPLE CASE

REFACTORING CURRENT CODE FOR USING JUNIT AND JMOCK

HOW MUCH TESTING IS ENOUGH?

APPENDIX: COMPLETED CODE THAT PASSES ALL TESTS





Abstract
Using Junit in conjunction with a test first and/or test driven development (tdd) provides enormous gains for not only the developer but also the company.

This document will outline what these benefits are and how they can be achieved.

Summary
It is a fairly well established statistic that as much as 80% of the lifetime cost of a software project is in it’s bug work, maintenance, and enhancement phases. Proper use of testing with Junit can almost eliminate this overhead.

By taking just a little more time upfront - approximately 33% for new development and perhaps 25% for enhancement work - the strength and structure of the overall code base becomes such that massive bug hunts, lengthy end of project testing, complex systems integration testing, and later enhancement requests are smooth processes rather than the more difficult and complex ordeals that the developers - and thus indirectly the company - face.

As even moderate experinece in the process is gained, the upfront time costs can drop significantly. The reason for this is that the developer simply becomes comfortable in letting the test guide how they develop the method or business process under test. Nevertheless, there is still always going to be an upfront cost, even if it’s an expert because the test methods and setups and mocks and all that have to be created and/or just typed into the file. But this cost is more than made up for in the integrity of the resulting code and the benefits that results in.

Definitions

Benefits of Process Adoption
The major benefits in adopting a test first approach with Junit can be summarized as follows:

Some Other Useful Tools

  • Cobertura - Cobertura (http://cobertura.sourceforge.net/) is an open source tool that shows which methods in all of your classes have been tested and which have not and gives a percentage of methods covered by tests as well as a percentage of branching logic that has been tested. The results of this product can be set in an ant script and break the build if not met by the code, i.e. 80% code coverage and/or 75% conditional branching logic tested.
  • Simian - Another useful product is called Simian (https://simian.dev.java.net/) which identifies duplication in Java, C#, C, CPP, COBOL, JSP, HTML source code and even plain text files. Simian can be used as part of the build process during development or as a guide when re-factoring. Think of Simian as an independent pair of eyes that will assist in raising the quality of your software.
  • JsUnit - An open source testing tool for JavaScript, JsUnit (http://www.jsunit.net/). JsUnit is a Unit Testing framework for client-side (in-browser) JavaScript. It is essentially a port of JUnit to JavaScript.
  • Selenium - Selenium (http://www.openqa.org/selenium/) is an open-source integration testing tool for web applications written by Thoughtworks developers. You can run Selenium IDE tests in any supported browser using Selenium Core (http://www.openqa.org/selenium-core/)
  • DbUnit - DbUnit (http://dbunit.sourceforge.net) is a JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts your database into a known state between test runs. This is an excellent way to avoid the myriad of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. DbUnit has the ability to export and import your database data to and from XML datasets.

Setting Up
To use Junit and Jmock, download both jars, and drop them into the common lib with the rest of your application’s jars (or in some local lib making sure that they can be found on the general classpath).

At the same level that your src directory is located, create a test directory with a com.this.that parallel structure to the src directory hierarchy. Thus, if you have src/com/myapp/dao/MyDAO.java then you are going to end up with a test/com/myapp/dao/MyDAOTest.java.

This will allow any protected method in the src treee to be tested seamlessly by the corresponding test class (and note that the standard is for the test class to be the same name as the class under test but with "Test" appended to it. Just makes life soooo much easier to find these things...

If you are running your tests from build.xml, then you may find yourself with the following really poor error message:

BUILD FAILED
/blah.../build.xml:51:
Could not create task or type of type: junit.

Ant could not find the task or a class this task relies upon.

Go to $ANT_HOME/lib and either make a symbolic link to your JUnit's junit.jar or copy it there. Ant apparently does not find JUnit in its classpath, resulting in the error message above. When JUnit is linked to Ant's library dir however, it can find JUnit and will call the unit tests as expected.

Test First
The following from
http://www.extremeprogramming.org/rules/testfirst.html is a great discussion of what the test first process involves and how you go about structuring a test.

     "When you create your tests first, before the code, you will find it much easier and faster to create your code. The combined time it takes to create a unit test and create some code to make it pass is about the same as just coding it up straight away. But, if you already have the unit tests you don't need to create them after the code saving you some time now and lots later.
     Creating a unit test helps a developer to really consider what needs to be done. Requirements are nailed down firmly by tests. There can be no misunderstanding a specification written in the form of executable code.
     You also have immediate feedback while you work. It is often not clear when a developer has finished all the necessary functionality. Scope creep can occur as extensions and error conditions are considered. If we create our unit tests first then we know when we are done; the unit tests all run.
     There is a rhythm to developing software unit test first. You create one test to define some small aspect of the problem at hand. Then you create the simplest code that will make that test pass. Then you create a second test. Now you add to the code you just created to make this new test pass, but no more! Not until you have yet a third test. You continue until there is nothing left to test."
A very good summary article of this process can be found at http://junit.sourceforge.net/doc/testinfected/testing.htm

Some Standards

Naming Test Classes
The test class should be the same name as the class under test but with “Test” appended to it. Thus, if I have a class EmployeeClassificationMgr.java then my test class will be EmployeeClassificationMgrTest.java.

Naming Test Methods
If the class I am testing has methods


protected Qualification findQualifications(IEmployee emp)
protected boolean validateQualifications(IQualification qual)

then I will have methods in my test class


public void testFindQualificationsWithEmployeeReturnsQualification()
public void testValidateQualificationsWithQualificationReturnsTrueWhereOkSkills()

For a method to be recognized by Junit to execute a test on, it must begin with the word "test". Thus, if you need helper methods in your test class, don’t start their name with "test"!

The accepted naming convention for test methods after the word test is to identify the name of the method under test, what parameters are passed in (because you may be testing other overloaded methods of the same name), what the return of the method should be (even if "void"), again due to methods of different signatures, and finally - if significant - what conditions must be met for the test to be considered a success.

Note that this naming convention works with open source frameworks you can use that will document all of the tests in your suite by parsing through the name of the test so you know exactly what the test does and which methods in the class are tested.

Be Aware of SetUp and TearDown Methods
Junit provides a great feature to make sure that each test is starting with the objects it uses in the same state.

The setUp() method, if you write it, is executed prior to Junit running each test in the class. And if you write a tearDown() method, it will be executed after each test method executes, sort of similar to a finally statement in a try/catch would.

The benefit of this is that say you need a new mock of an Employee object for each test and that this object must be in the same state for each test to make sure if there is a failure it is not because of the Employee object but because of some defect in the code operating on that Employee object. The test class itself would carry an instance property of Mock employee which is then created with its associated expectations in the setUp() method thus ensurring this continuity throughout all of the test methods in the test class.

These methods are very useful when testing calls to the database as you can run a script to drop all tables, recreate them, and then add a fresh dataset from a text or Excel file (perhaps using a tool like DbUnit).

Mock Objects

Mock vs. Concrete Object
A mock object is used in place of a real concrete object. You mock the Interface the class implements as follows:

Mock accts = mock(IAccounts.class); You mock objects that you are not testing, i.e. other objects that the method under test is using. Let’s say you have the following method in an AccountsManagerService class


class AccountsManagerService {
	public Money payEmployee(IEmployee emp, IAccounts accts) {
		if( accts.employeeNotOnFile(emp) ) {
			accts.setUpNewEmployee(emp);
		}
		Money payment = accts.issuePay(emp);
		return payment;
	}
}

While nothing is being done here to the Employee object, it is evident that the Accounts object will be doing something, i.e. those methods being called are going to ask questions of and perhaps even change that Employee object. But we don’t care about the Accounts object or what it does - or for that matter anything about the Employee object either. All we want to do is test the behavior of the payEmployee() method.

If we pass in a "real" Accounts object, then we can’t isolate the payEmployee() method’s activities. However, if we pass in a mock of an Accounts object, we can set expectations on the mock to do whatever we want it to do. We tell the mock what methods will be called on it, if those methods will expect certain input parameters, and what sort of return we are expecting from those method calls; all this without having the Accounts object really "do" anything.

Defining Expectations On Mocks
From the above code, we see that we are going to "expect" the Accounts object to call its employeeNotOnFile() method. Depending on what that method does, it will return a boolean result. But we don’t really care how it does what it does; all we want to know is that if we give it an Employee object it’s going to reply either true or false. Our mock would thus look like this for this one method expectation:


Mock accts = mock(IAccounts.class);
accts.
	expects(once()).
	method("employeeNotOnFile").
	with( eq((IEmployee)emp.proxy()) ).
	will( returnValue(true) );

Here, we’re telling the mock to return true because when it does, we will then test the conditional call setUpNewEmployee() method. Thus, we test both conditional branches. In this case, it’s pretty simple to do; in a more complex example you might have to write an additional expectation returning a false value.

Note the recepie for the construction of a mock object: we "expect" just "once" that the "method" employeeNotOnFile will be called and that it will be called "with" a parameter of type IEmployee, and that it "will" return a value ("returnValue") of true.

Of confusion to those new to mock objects is the ‘with( eq(xxx) )’ construct. Let’s say I have a method that takes three parameters: one of type IEmployee, one Boss, and a boolean. Our parameter definitions and the resulting statement might look like this:


Mock accts = mock(IAccounts.class);

Boss theBoss = new Boss();
theBoss.setName("Big Guy");
theBoss.setFireAuthority(true);

boolean shouldIFireSomeone = true;

accts.
- - - other expectations - - -
with(  eq((IAccounts)accts.proxy()),
		eq(theBoss),
		eq(shouldIFireSomeone)  )
- - - other expectations - - -

We use the strange looking cast (IAccounts)accts.proxy() when passing a mock as a parameter. It’s just something you have to do in comparisson to the simple concrete theBoss object we are passing in. What this signature is saying is that we expect the Accounts object will be doing some sort of work, possibly instantiating or using other objects etc so we must mock it and pass a proxy of it in the manner shown. However, all we are going to use the Boss object for is to call some accessor methods; we don’t have to mock it, we can just use it as is. It’s confusing at first, but comes intuitively later...

The Process

Using Test First
Write the test first, even if that means instantiating an object that does not yet exist to instantiate. Though seeming trite and nonproductive, it puts you into a pattern of habit that will pay off in the long run.

Writing a test implies writing a test against a single method in a class. We are not interested in where objects used in the method come from or what they do, we are only interested in that one method. Thus, any object other than a simple data object is simply mocked. The important thing to understand about testing a method is that you are testing only 2 things:

You test void methods and methods with no signature simply by calling them and not having an unexpected Exception or other unwanted trauma occur.

Limiting Tests To Boundary Conditions Of The Method
What this means is that we are only testing the boundary conditions of that method. Thus, a method like the following is going to be very difficult to test because we have to test the Accounts object as well as the business logic inside of the payEmployee() method:


public Money payEmployee(IEmployee emp) {
	Accounts accts = new Accounts();

	if( accts.employeeNotOnFile(emp) ) {
		accts.setUpNewEmployee(emp);
	}

	Money payment = accts.issuePay(emp);

	return payment;
}

While it might be ok to instantiate data objects inside a method, it is never appropriate to instantiate agent type objects because then you have to test them as well as the class that is supposed to be the focus of the testing! Not good... Better would be to form the method like this:


public Money payEmployee(IEmployee emp, IAccounts accts) {
	if( accts.employeeNotOnFile(emp) ) {
		accts.setUpNewEmployee(emp);
	}

	Money payment = accts.issuePay(emp);

	return payment;
}

Now, with the mock you don’t have to worry about testing the Accounts object.

Example Case

Establish The Problem Domain
Let’s say we have the following simple interface and class


interface IAccounts {
	boolean employeeNotOnFile(IEmployee emp);
	void setUpNewEmployee(IEmployee emp);
	Money issuePay(IEmployee emp);
}

class Accounts implements IAccounts {
	public boolean employeeNotOnFile(IEmployee emp) {
		return false;
	}
	public void setUpNewEmployee(IEmployee emp) {
		/*whatever...*/
	}
	public Money issuePay(IEmployee emp) {
		return new Money();
	}
    }

Our assignment is to write a method that will pay an employee and if it is discovered that the employee is not yet on the accounting files to add him on, and then pay him.

This defination of the problem to solve is called the problem domain. We will not concern ourselves with anything outside of this small finite issue.

Create The Solution Stub
Ok, we’ll create a new service class AccountsManagerService with a method payEmployee() on it that looks like the following (skipping several of the test first steps so as not to put anyone to sleep):


class AccountsManagerService {
	public Money payEmployee(IEmployee emp, IAccounts accts) {
		return null;
	}
}

This is just a stub or place holder. It’s a starting point in creating a solution to our problem domain.

Write The Test
Now, we write our test.


public class AccountsManagerServiceTest
			extends MockObjectTestCase {
	public void
			testPayEmployeeWithEmployeeAndAccountsReturnsMoney () {
		Money payment = new Money();
		payment.setGrossPay(2000.00);
		Mock emp = mock(IEmployee.class);
		Mock accts = mock(IAccounts.class);

		AccountsManagerService svc =
				new AccountsManagerService();
		Money cash = svc.payEmployee(
				(IEmployee)emp.proxy(),
				(IAccounts)accts.proxy());

		assertNotNull(cash);
		assertEquals(
				cash.getGrossPay(),
				payment.getGrossPay(),
				0.0);
}

Notice the simplicity of the expectation of the first test, i.e. the assertNotNull test. I just stubbed in the next test because I wanted to stress how important it was not to test what you might normally think is "the" test, i.e. that the amounts match, but rather that you don’t get a null value back.

Working Through The Test

Fix And Test Again
Of course, the test will fail because the method is written to return null. So fix it:


class AccountsManagerService {
	public Money payEmployee(
			IEmployee emp,
			IAccounts accts) {
		Money payment = new Money().setValue(42.80);
		return payment;
	}
}

The Next Test Iteration
Guess what: the first test passes, but now the second one fails (because 42.80 does not equal 2000.00). So, let’s add a call to the IAccounts object to return the Money value:


class AccountsManagerService {
	public Money payEmployee(
			IEmployee emp,
			IAccounts accts) {
		Money payment = accts.issuePay(emp);
		return payment;
	}
}

The whole thing now blows up and JUnit tells us that the IAccounts object is being asked to do something and that something is not given as an expectation on the mock. So, we have to rewrite the test to account for this:


public class AccountsManagerServiceTest extends MockObjectTestCase {
	public void testPayEmployeeWithEmployee
				AndAccountsReturnsMoney () {
		Money payment = new Money();
		payment.setGrossPay(2000.00);
		Mock emp = mock(IEmployee.class);
		Mock accts = mock(IAccounts.class);
		accts.
			expects(once()).
			method("issuePay").
			with( eq((IEmployee)emp.proxy()) ).
			will( returnValue(payment) );

		AccountsManagerService svc =
				new AccountsManagerService();
		Money cash = svc.payEmployee(
				(IEmployee)emp.proxy(),
				(IAccounts)accts.proxy());

		assertNotNull(cash);
		assertEquals(
				cash.getGrossPay(),
				payment.getGrossPay(),
				0.0);
	}

Yeahhhhhhh! Both tests pass. Success.

Development Through The Remaining Problem Domain
However, we have an additional business requirement for this method. We need to make sure the employee is registered, and if not to register him. So, we add this logic into the method as follows:


class AccountsManagerService {
	public Money payEmployee(
			IEmployee emp,
			IAccounts accts) {
		if( accts.employeeNotOnFile(emp) ) {
			accts.setUpNewEmployee(emp);
		}
		Money payment = accts.issuePay(emp);
		return payment;
	}
}

And of course, when we run the test it complains that expectations need to be added to the IAccounts mock for employeeNotOnFile() and setUpNewEmployee(). So, the final form of the test would look like this:


public class AccountsManagerServiceTest extends MockObjectTestCase {
	public void testPayEmployeeWithEmployee
				AndAccountsReturnsMoney() {
		Money payment = new Money();
		payment.setGrossPay(2000.00);

		Mock emp = mock(IEmployee.class);
		Mock accts = mock(IAccounts.class);
		accts.
			expects(once()).
			method("employeeNotOnFile").
			with( eq((IEmployee)emp.proxy()) ).
			will( returnValue(true) );
		accts.
			expects(once()).
			method("setUpNewEmployee").
			with( eq((IEmployee)emp.proxy()) );
		accts.
			expects(once()).
			method("issuePay").
			with( eq((IEmployee)emp.proxy()) ).
			will( returnValue(payment) );

		AccountsManagerService svc =
				new AccountsManagerService();
		Money cash = svc.payEmployee(
				(IEmployee)emp.proxy(),
				(IAccounts)accts.proxy());

		assertNotNull(cash);
		assertEquals(
				cash.getGrossPay(),
				payment.getGrossPay(),
				0.0);
	}
}

Summing Up
And the critical thing to note from this test is that we don’t need the real Accounts object to test this method. We don’t care... We put the expectations we needed on the mock and that’s all that needed to be done.

If we would have passed the real Accounts object in, the internal workings of its employeeNotOnFile() method may have entailed, for example, a call out to the database as well as potentially instantiating other helper objects (maybe an AccountingDept object) to make its determination as to if the employee was registered or not. How do you write a simple test for that? You don’t and that’s why understanding mocks are so important.

And most important to the whole process is that the architecture of the application is designed through Interfaces that encapsulate discrete business logic and make the whole process of testing so much cleaner.

Refactoring Current Code For Using Junit and Jmock
Coming into a code base where objects are instantiated within the method you want to test makes mocking those objects impossible, particularly if they do not implement some interface. And if you can’t mock the object then you can’t test the method in isolation.

This is the situation that was demonstrated in the discussion on testing the boundary conditions of a method rather than the objects in the method. Here is the method that was given as an example:


public Money payEmployee(IEmployee emp) {
	Accounts accts = new Accounts();

	if( accts.employeeNotOnFile(emp) ) {
		accts.setUpNewEmployee(emp);
	}

	Money payment = accts.issuePay(emp);

	return payment;
}

Though we can mock the IEmployee object emp, we can’t mock the Accounts object. To complicate things, let’s assume that the signature of the payEmployee() method is defined in an interface so that we can’t just turn around and pass the Accounts object in as an interface.

We find this a lot in existing code bases. The solution could be as simple as creating a new protected method payEmployeeDelegate() that does take an IAccount object and calling that from the public method (note changing the object type from Accounts to its interface type IAccounts).


public Money payEmployee(IEmployee emp) {
	IAccounts accts = new Accounts();
	return payEmployeeDelegate(emp, accts);
}


protected Money payEmployeeDelegate(IEmployee emp, IAccounts accts) {
	if( accts.employeeNotOnFile(emp) ) {
		accts.setUpNewEmployee(emp);
	}

	Money payment = accts.issuePay(emp);

	return payment;
}

Now, the full responsibility of the method can be tested through the newly refactored protected delegate method.

Even if Accounts does not have an interface that it implements, nothing adverse happens by creating one for it with any public methods it uses as part of the that interface.

How Much Testing Is Enough?
One of the major decisions that a development group needs to make in regard to all of this is: how much testing is enough? There is no simple answer. The only approach that I know of to address this is to discuss it in context with if the project is new design or if the project is enhancement of an existing code base.

New Development
When designing a new application, the world is wide open for using test first and all of the advantages of the test process. It is a relatively simple thing to install a product like Cobertura, for example, and set high test coverage thresholds. I participated in a project where we started the threshold at 90% and worked it up to 94%. That’s outstanding, but at the same time probably a little on the overkill side.

But the point is that in such an environment, high thresholds are possible. From that high coverage comes total confidence in the code base and its ability to be deployed to production with little concern. I guess it all depends on how much pre-deployment testing you want to have to do in comparison to the time you want to allocate to developing test coverage.

My view is that when the world is open I want the most coverage I can get in the places where most of the problems are going to occur. I’m not going to be too concerned with writing what we kindly refer to as "coverage tests", i.e. tests written against every method in every object to get the coverage level as high as possible. Why write tests against accessor methods in data objects, for example? What’s the point? What are you testing; that the setter method won’t barf if you send it a null? I mean, come on...

What I do want to test is any method that provides a service or makes a business process decision. This does not include façade or delegation methods, or methods that simply act as pass throughs (as some persistence layer objects sometimes do). Service methods solve the problem domain and so those are the methods that need to be tested.

And we might be surprised at how few of these types of methods there are as a percentage of the entire code base when the application is designed properly. What percentage I couldn’t speculate on, but far less than might at first be guessed. Of course, the individual developer has to be trusted to make mature independent decisions then as to what tests are written and which tests are not written. But that’s an organizational issue that is out of scope for this discussion.

The long and the short of writing tests for new development is that it’s a lot easier to do because it is being done in a test first process and so coverage can be consistent, predictable, and dependable.

Existing Code Base
We can spend our whole life going through an existing application writing tests that mean nothing. Most code bases which have not had a good test policy (or any test policy) applied to them are generally plagued with coupled objects. Many times this is because the design was not done by Interface but rather by abstract and concrete classes using too much inheritance instead of aggregation and applying strategy design patterns and taking full use of polymorphism.

That said: such an application runs and is not falling apart or bleeding exceptions with every user submit from the GUI. Under the sheets things may be a mess, but it works. So, to go in and start creating Interfaces and refactoring code to write tests is to beg for trouble. If the application wasn’t broken before this kind of silly effort is started, it surely will be in a very short time under this kind of activity.

The Agile mantra of "anytime you touch code, leave it in a little better state" is a great thought to guide us as we poke around in an existing code base. It is pointless to refactor and test relatively straightforward methods that have little chance of being passed crummy parameters (particularly if the "good neighbor policy" appears to be followed in the code) or that stand little chance of blindly returning what might or could be a crummy result. Just leave it alone!

Rather, refactor the method that controls some critical aspect of the business process where these types of maladies may be lurking. Pick your battles carefully because many times making such a refactor will unveil the lack of Interfaces on certain key objects the method uses. And that’s where things can get sticky in a coupled code base: it’s not so easy to just create that interface now and apply it across the breadth of concrete objects that must implement it.

Selecting these complex business process methods and testing them may only provide a 20% code base coverage. However, we can casually - though confidently - say that 80% of the problems come from 20% of the code base, and so we can thus conclude without much of a leap of faith that we thus have an 80% confidence level over the entire code base by only tightening up 20% of the code. How’s that for logic?

Appendix: Completed Code That passes All Tests
Here’s the production code needed to support all of this:


public class Money {
	double grossPay = 0.0;
	public void setGrossPay(double grossPay) {
		this.grossPay = grossPay;
	}
	public double getGrossPay() {
		return grossPay;
	}
}

public class AccountsManagerService {
	public Money payEmployee(
			IEmployee emp,
			IAccounts accts) {
		if( accts.employeeNotOnFile(emp) ) {
			accts.setUpNewEmployee(emp);
		}
		Money payment = accts.issuePay(emp);
		return payment;
	}
}

public interface IEmployee { }
class Employee implements IEmployee { }

public interface IAccounts {
	boolean employeeNotOnFile(IEmployee emp);
	void setUpNewEmployee(IEmployee emp);
	Money issuePay(IEmployee emp);
}

public class Accounts implements IAccounts {
	public boolean employeeNotOnFile(IEmployee emp) {
		return false;
	}
	public void setUpNewEmployee(IEmployee emp) {
		/*whatever...*/
	}
	public Money issuePay(IEmployee emp) {
		return new Money();
	}
}


Back to Thoughts On Technology Page | Back to Peter's Home Page | Back to Family Home Page