I have Written Defect-Free Code Without Tests

The question “If you’re not doing TDD, what are you doing?!” arose again in the Twitterverse. It reminded me of this old post, which I’ve updated a bit. (It originally appeared on my Essential Test-Driven Development blog, dated 20 Oct 2010.)

The Arrogance of Youth

I used to write bug-free software without TDD! (Look at me: I am so smart...)

My boss and I would play this game called "Who Wrote the Bug?" in which he would claim that a bug was in my code, and I would come back (after painstakingly debugging the whole product) and point out where it was in his code. (Hmmm...who was smarter?)

Humility Sets In

The software I was writing was a protocol stack that needed to convert between 8-bit and 9-bit bytes, or 16-bit and 36-bit words. I used the command-line/printf approach (System.out.println for the Java readers; Console.WriteLine for C# readers) to test stuff. The debugger, COFF (the “C Obscure Feature Finder”), was okay, but testing through a GCOS8 debugger was such a pain. Instead, I’d pepper the code with something akin to the following abomination:

#ifdef DEBUG
printf("foo=%s", foo);
#endif

(If you haven't figured it out from the clues I've dropped: this was over 30 years ago.)

In hindsight, this domain was pretty tame stuff. I could mostly keep track of what was going on, in my head.

One can assume that software projects these days are more complex than that. Perhaps too complex for anyone to keep track of all the little moving parts. It's enough for the team to have a grasp of the overall concepts, goals, and metaphors.

Even on a team with only three or four developers, eventually they're going to bump into each other's changes. They need a suite of fast tests in order to maintain the quality of their code, while they add features.

The test frameworks like JUnit, MSTest, RSpec, Jasmine, & Cucumber all give you a way to record your assumptions, each independently. Rather than manually checking console output each time, you let the computer check each assumption whenever you run the tests. You can then easily let those assumptions grow upon each other.

Two Years of Investment

In 2002 I worked on a team of four J2EE developers on a life-critical (i.e., "You break it, someone may die") application. After two years, we had built some pretty sophisticated reports and heuristics that helped hospitals determine who was most likely to survive an organ transplant.

We had about 20,000 tests. They all ran in about 12 minutes.

Imagine: All aspects of a complex, life-critical, team-built application could be thoroughly checked in 12 minutes. We could (we did) bring in new developers, sit down to pair-program, and say "today we will make whatever changes we need, even mere hours before the next release, as long as we run all the tests and see a 100% pass-rate!"

Twelve minutes, and we knew we hadn't broken any of our careful, 8-(plus-)person-year investment.

Rise to the Occasion

If you're sitting alone in your basement writing a game to sell to a VC, you may not care if it's maintainable. By all means, please continue using only manual testing techniques to verify your app. I'm not being snarky: This may be the optimal win-win approach for you and the VC. (The team that eventually maintains your app will curse your name and throw darts at your picture, but do you care?)

On the other hand, if you're part of a team, writing mission-critical software for a large or complex domain, and it needs to get beyond version 1.0; then please make sure you can always test everything, and quickly. These test-first approaches, TDD & BDD, are the best and fastest ways to do this.

Which also makes it the professional thing to do.