Test Before You Code
Test Before You Code
html
For example, let's say you are a software architect who has always described project functional requirements as a list of unrelated "shall
have" items in a Software Requirement Specification (SRS). But then you attend a workshop on use cases and realize that they more
accurately describe why systems will be valuable to stakeholders. Although you would unquestionably be adopting a better technique,
switching to use cases on your current project would lead to chaos — unless you can ensure that everyone on the team has made the
switch as well.
Of course, if you are working on a small project within a larger one, you might be able to adopt different techniques to optimize your work,
as long as they will not interfere with more widespread organizational practices. For example, if a large project requires specialized 2-D
and 3-D graphics for its user interface, the subproject that implements the libraries for rendering these graphics might adopt special
methods for describing requirements. This would have no negative impact on other process activities. Another subproject that is suitable
for model-driven development might employ UML models to generate a significant portion of the subproject's code — without interfering
with the overall project's code generation procedures.
The explanation below is based on how I apply TFP. Although not an exhaustive description, it perhaps will provide enough information to
whet your appetite and encourage you to learn more about TFP. I include several sources for this purpose in the References section.
z Run the unit test. If it succeeds, you're done. Go to step 1, or if you are completely finished, go home.
z Fix the immediate problem: maybe it's the fact that you didn't write the new method yet. Maybe the method doesn't quite work. Fix
whatever it is. Go to step 3.2
This seems straightforward enough, but you must fulfill two requirements to adopt TFP successfully. First, you must know how to write a
good test. Second, you must discipline yourself to write the tests before coding. This runs counter to everything you may have learned in
your training.
For me, a good test focuses on one thing. However, how I define that "one thing" may vary depending upon the type of code I am writing.
This is similar to the principle for designing methods and classes. You want high cohesion throughout your design, and your tests should
be cohesive as well.
The typical supporting tools for TFP are good integrated development environments (IDEs) and unit test frameworks. I use the Eclipse
IDE, with the JUnit plug-in, for all of my Java development.
JUnit is a unit testing framework designed by Kent Beck and Erich Gamma for testing Java programs.3 Variations of it support the testing
of programs in other languages, such as cppUnit for C++. I used JUnit to create the example in the next section of this article.
At this point I must make a couple of decisions. First, I need to decide what type of number to use for arguments. I choose double-
precision real numbers, which are the most general type. I also need to decide how to represent the triangle type. After some thought I
create a separate type called TriangleType that indicates properties of the triangle.
Since there is no TriangleType class, I need to create it. This class, with just a constructor, is shown in Code Sample 2.
Now I'm ready to write the first test. Eclipse can create a test class for the TriangleTester class. By default it will be named
TriangleTesterTest, but you can change the name to anything you want. Since I already have a method in my TriangleTester class
(kindOfTriangle), I can have the Eclipse JUnit integration create a test method, testKindOfTriangle, that corresponds to it. Note that
all JUnit test methods must begin with the word "test." Case is not important.
Many test cases are possible for testing kindOfTriangle. I need to decide if I will have just one test method or many test methods for the
different test cases. My preference is to have many small test methods. So, instead of letting the tool create the initial test method, I will
write each method as I go.
What should I test first? As I noted above, we can classify triangle types by angles or sides, and in some instances the three numbers we
supply may not form a valid triangle. It doesn't matter which possibility I choose first, so I pick a right triangle.
I create the method called testRightTriangle and write a simple test, shown in Code Sample 3.
The Pythagorean Theorem tells us that a 3-4-5 triangle is a right triangle, so we can simply test for sides of 3, 4, and 5.5 Even after I
import the junit.framework.Assert class, my test class will not compile. That is because the TriangleType class does not have a
method called isRightTriangle; I need to create one.
When practicing TFP, you add only enough code to make your tests pass. In this case, I need a true value; the simplest way I can make
this happen is by adding the code in Code Sample 4.
You might wonder whether I should write a test class for the TriangleType class. At this point I choose not to do so because everything I
write in the TriangleType class will be tested in the TriangleTesterTest class.
Now I execute my first test. This is very simple in Eclipse. I select the TriangleTesterTest class and tell Eclipse to run that class as a
JUnit test. When I run the test, it fails. Figure 1 shows the JUnit view I get in Eclipse. The red bar tells me that there was a failure, and the
rest of the information in the view helps me determine what went wrong.
Click to enlarge
Ah, now I see the problem. I'm returning null from the kindOfTriangle method. I change the method to return an instance of
TriangleType and rerun the test. Now the bar turns green, as shown in Figure 3.
What's next? What about triangles that are not right triangles? What if I put in different numbers and ask if they represent a right triangle?
I add a test for that to the test method I've already written, as shown in Code Sample 5.
Now, when I run the test it fails at the assertion I just inserted. The reason is simple. I'm just returning true from the isRightTriangle
method (Code Sample 4). I have to fix this before continuing with new functionality. Never add new code until the existing code passes all
previous tests. First, I add code to check whether the triangle is a right triangle as shown in Code Sample 6.
Now, of course, I have to write the code for isRightTriangle and setRightTriangle. I add the method isRightTriangle to
TriangleTester.java and setRightTriangle to TriangleType.java, as shown in Code Samples 7 and 8, respectively.
To keep the right triangle property I add a private variable and a setter method.
Now I get a green bar again in JUnit, so I can continue to add functionality. Should I go on to tests for other types of triangles? Perhaps,
but at some point, I must deal with the fact that real number arithmetic is not exact on digital computers. I know this from experience. So
now I'd like to see if the code I've written so far handles real numbers correctly. (I'm pretty sure that it doesn't, but I need a test to make
sure.) I add the test in Code Sample 9 to my right triangle tests. Sure enough, JUnit displays a red bar: The test failed.
Now I have a design decision to make. How do I want to handle this precision problem? All solutions I can think of would require
refactoring and rework. Experience tells me that some rework is inevitable. At least I now have a set of tests the code must pass, and I
haven't gotten so far into coding that the amount of rework is excessive.6 I need a delta — an acceptable amount for arithmetic error. I'd
like to make my client tests remain as they are, so I add a default delta, remove the right triangle test return statement, and add the code
shown in Code Sample 10 to TriangleTester.java.
Then, I continue to add tests, each time modifying the code by adding new code, changing existing code, and refactoring as I go.
Eventually I have implemented a complete class in which I have confidence. I also have tests that I can run anytime. In the future, if I
change the code, or someone else does, these tests can tell us if we have broken any existing functionality.
At this point, I'll stop giving you all of the details. Before moving on, however, let's look at Table 1, which shows the tests I wrote and the
order in which I wrote them. We'll also look at some decisions I had to make as the coding progressed.
Test Description
Positive right triangle Check to make sure that a 3-4-5 triangle is recognized as a right triangle.
Negative right triangle Check to make sure that a 3-3-5 triangle is not recognized as a right triangle.
Real number precision Add a real number default delta to the program.
Variable default delta Allow the client to change the default delta. This caused me to make the TriangleTester
methods non-static, which required a change to the test class.
Test for delta as part of the Let the caller supply a delta value as part of the call to the kindOfTriangle method.
method call
Test for invalid triangles Check to see if the values given to kindOfTriangle can actually represent a triangle. If they can't,
return a special TriangleType.
Test to ensure NOT_A_TRIANGLE NOT_A_TRIANGLE is meant to be a constant, but it is not protected from change. To bulletproof the
cannot be modified code that implements this special TriangleType, I created TriangleTypeTest.java.
Test combinations of sides on Make sure that the program finds a right triangle, regardless of which side is the hypotenuse.
right triangles
Isosceles triangle tests Make sure that an isosceles right triangle is recognized as both isosceles and right. This is similar
to the right triangle test.
There you have it. I implemented the TriangleTester and TriangleType classes with about 100 lines of Java code and 70 lines of test
code.
Benefits of TFP
Adopting TFP can have several benefits, some of which I have experienced personally. Your work style and context will determine how
much value you get from the practice.
The fact is, everyone is responsible for quality — regardless of how you define it. It is reasonable to expect that programmers will unit test
their code before adding it to the rest of the project. If programmers use TFP, they have to test their code. There is no way of getting
around it. Students who are just learning how to test code often protest that TFP does not necessarily result in good tests. I tell them they
must decide whether "bad" tests are better than no tests at all. Perhaps we'll explore this issue in a future column.
The triangle tester example has a total of seven test methods, but it has twenty-three distinct tests. And this is not a complete set. I could
add many more, but I'm confident that the tests I have developed are pretty solid. This is confidence I would not have if I had not written
any tests.
High-percentage coverage
You can measure code coverage in several ways: by assessing line or statement coverage, condition coverage, branch coverage, and so
on. When you adopt TFP, you can achieve 100 percent code coverage. Although my particular method for using TFP does not provide
this (see the How much is enough? section below), it does provide an acceptable level of coverage for the code segments most likely to
contain defects.
Testers agree that an acceptable level of code coverage is around 80 percent. Trying to achieve full coverage is often a waste of time; if
you have to spend hours trying to force tests to exercise error conditions and exceptional cases, you get diminishing returns.
Theoretically, if you write code only to satisfy an existing test, then by definition you can achieve coverage for every line of code you write.
However, this may not be so in practice. I am not aware of any empirical studies that demonstrate a 100 percent coverage benefit.
The important point is that adopting TFP guarantees that you will achieve a significant amount of code coverage in your unit tests —
probably far more than what you achieve today.
Test-driven design
Designs evolve. We know this to be true from experience. When business conditions change or stakeholders develop new requirements,
we need ways to modify existing code. That is why software should be soft and flexible.
When you use TFP to drive your design, in effect you have adopted a test-driven design (TDD) approach to building software that
naturally leads to simpler, more flexible architectures.
In the triangle tester example above, I revised my design based upon the tests I wrote. I added the constant NOT_A_TRIANGLE object in
response to my test. Based on the final result, I might decide to remove that constant and add a field to the TriangleType class that
indicates an invalid triangle. The design will evolve.
As I implemented the tests for equilateral and isosceles triangles, I realized that I was using only the delta value for calculations in right
triangles. Perhaps I will want to add a "fuzz factor" with deltas for determining whether I have an isosceles or equilateral triangle,
respectively. However, since I don't need that now, I can put it off to another day — when I might really need it.
TFP/TDD is a practice that I will use to supplement my other design tools and techniques. As I gain more experience, I will find more
ways of applying it and understand better when and where it is appropriate.
Tests = specs
When you finish using TFP, the tests you have created represent a set of executable specifications. As long as you keep your tests and
code synchronized (and that's the whole point of the practice), if a programmer wants to know what the system does, he or she can look
at the tests.
Tests are not the only type of specification you need. It is not reasonable to ask clients to look at the tests to see if your specifications are
correct. As software engineers, we know there are many types of requirements that we can represent in different ways. It is foolish to
think that one type of requirement will satisfy every stakeholder or that it will be easy to keep requirements synchronized with the code.
However, TFP does provide support in this area.
If we use tests to bound our increments, we can get immediate satisfaction by writing our first test and then implementing code to make it
work. I have found that adopting TFP helps prevent analysis paralysis, that condition that makes us afraid to write code because we don't
yet know all of the constraints and possible problems we might face.
As the old Chinese proverb says, "A journey of a thousand miles begins with the first step." TFP helps us take that first step, and then the
next, and so on.
Easy to learn
TFP is an easy practice to learn. I introduce it in one hour-long class during the term, providing references and walking students through
an example such as the triangle tester program. At the end of this hour, they are ready to go out and try their luck.
Several students take to TFP quickly and report that it has changed the way they create programs. Others are uncomfortable with it, and I
don't force them to adopt the practice. For now, I want them to experience many software development practices and use the ones that
match their style. However, if somewhere along their career paths they need TFP, at least they will know how to use it. If you are a project
manager, you might want to use a similar approach and spend a modest amount of time training your team to use TFP.
There are different opinions on this. I take the middle ground and test most of the methods I implement. However, I do not bother writing
tests for methods that the IDE generates for me.
How much testing you do depends upon your project goals and available time. Although it might be nice to create extensive tests for
every method you write, you must decide how much you can actually do and whether the return will warrant the time you put in.
A versatile practice
TFP is a useful practice, whether or not you use it as a design tool. It provides extensive coverage and is a relatively easy way to prevent
coding disasters. If we think the code we're about to write is simple, we tend to cut corners. But if you've ever seen a system fail because
of one line change, then you can appreciate the need to test as much as possible.
You can apply TFP within almost any project context, whether or not the organization or project process dictates its use. When they see
the quality of your code, maybe your teammates will ask what you've done to improve it. You can easily share your "secret" and try to
make it part of the team's process. Even if you don't succeed at this, you can still be proud of the tests and code you deliver.
References
z http://www.junit.org/index.htm: the home page for the JUnit project offers code and articles about how to apply TFP. Be sure to
read "Test Infected — Programmers Love Writing Tests," an article in the documentation section.
z http://c2.com/cgi/wiki?CodeUnitTestFirst: the XP Wiki Web page devoted to TFP.
z Kent Beck, Test Driven Development by Example. Addison-Wesley, 2002. Describes Beck's approach to TDD.
z David Astels, Test-Driven Development: A Practical Guide. Prentice Hall, 2003. Provides learning via examples.
z Andrew Hunt and David Thomas, Pragmatic Unit Testing in Java with JUnit. The Pragmatic Programmers, LLC, 2003. Instruction
for experienced TFP users.
Notes
1 Readers unfamiliar with XP can consult http://c2.com/cgi/wiki?ExtremeProgrammingRoadmap.
2 See http://c2.com/cgi/wiki?CodeUnitTestFirst.
3 For information about JUnit, visit http://www.junit.org. Read the Test Infected paper at this site for a good overview of the framework and
the process. Also, the References section at the end of my article lists some of the many books and papers that describe how to use
JUnit in the context of TFP.
4 We won't show a complete program, just a class that will do the job. It's easy to test this class with Eclipse and JUnit. You can download
all the code for the class by clicking on the link at the end of this article.
6 Clearly, if I had waited until later in the development cycle to add this test, I'd have to do more rework. Experience counts.
Killer! (5) Good stuff (4) So-so; not bad (3) Needs work (2) Lame! (1)
Comments?
Submit feedback