How Test-First Differs from Test-After
I’ve been asked many times if test-first methods of software development—Test-Driven Development (TDD) and Behavior Driven Development (BDD)—are “better” than similar test-after methods (unit-testing or automated end-to-end testing applied after the code has been written).
Often there’s the “all things being equal” qualifier in the question: if we could assure that the quality and fit-for-purpose of the product would be similar with either method, then is there still a reason to use test-first?
Despite what could be considered an error in logic, there’s a very human concern expressed therein: “Have we been doing it the hard way all along?”
Yes, test-first is much easier, because it’s natural. (Okay, we’ll get there…just stay with me.)
A Flaw in Terminology
Both terms, “test-first” and “test-after,” only allude to the mechanics of the methods. But the methods go beyond mechanics in significant ways. We are not mere machines; we have to look at how each method affects the practitioners.
And there’s that term “test.” Even though the BDD community has dropped the word in favor of “behaviors", “specifications,” and/or “scenarios,” many people still reflexively use “test” in conversation.
And it’s not wrong. When these methods are used correctly, the team grows a strong, comprehensive “safety net” of automated regression tests.
The trouble with “test” is that what we write isn’t a test until it passes for the first time. We run tests, we do not write tests. We write examples.
A Way of Thinking
I’ve lost count of the number of times teams have been struggling with these practices, and they sit down together to have an informal conversation as a team (product, BAs, testers, developers), and while discussing a feature, they start naturally speaking in terms of examples.
E.g., “If Rob is on the Electric Vehicle plan, and it’s 1am in his time zone, then he gets the SuperSaver electric rate…”
With test-first methods, we take those examples and record them in a framework like Cucumber, Jasmine, or JUnit.
One Step at a Time
Test-first doesn’t just change the order of tasks: it makes writing the test an integral part of building the system.
With TDD and BDD, we write one example at a time, and immediately build just enough of the system so it can answer “yes, I can do that for you.”
Why immediately? Well, for one thing, an example that doesn’t have any code to call is going to fail in a messy way: with a compiler error, a null reference exception, or worse. (There’s an intermediate step here that I’m not going to detail in this article: we next want to see the test “fail cleanly”: failing due to a logical assertion about behavioral outcomes.) But there are numerous team-centric, product-centric benefits:
It’s fresh in everyone’s mind, and that’s the best time to clarify vagueness and avoid misinterpretations. If, instead, we wrote a bunch of examples all at once, we will likely have to change them all when we find a better way to say what we mean, or a better way to interface with our system. The shape of each passing test feeds into the shape of the next example.
It’s done! We’re not building up a huge backlog of tests for some later time.
We know that one small part of our system works as expected.
Our investment of time and effort is secured. When we add the next small bit of behavior to the system, we are confident that we will not break any of the previous behaviors.
Getting one example working will inspire more and different examples. The conversation continues, the examples continue, the brainstorming and clarifications and troubleshooting all continue in real-time, rather than at some later time when a developer is seated alone in a cubicle with no one else to talk to.
So while it’s true that—if your code is designed to be testable—you can write the tests afterward and get the later benefits from having tests. But you’ll be missing the benefits of that early ease and confidence.
Shaping the Solution
Tests from examples represent the desired behaviors of the system. Conceptually, they act together as a container that shapes the semi-solid system within. Trying to build the container afterwards is likely to result in a shape that matches the lumpy shapeless blob of code. I.e., you end up with a suite of tests that support the behaviors that were built, not necessarily the behaviors that were intended.
There’s also likely to be numerous unseen pinholes where the fluid will slowly leak out. In software terms: defects.
Reality Beats Theory
All things are not equal.
Whenever a team has reassured me that they’ve been adding tests after the fact, we (the team and I) often discover they have perhaps 20-30% behavioral coverage. That leaves a lot of untested behavior that could break without anyone knowing!
Let’s face it, writing tests (the chore) after the implementation (the fun part) is akin to eating cold vegetables after dessert. No one—except possibly the family dog—is up for that.
With a disciplined test-first approach, the team will not add untested behaviors. Period.
