Is unit testing doomed?

Most organisations that try unit testing usually find the return is well worth the cost, so why is the approach looking like running out of steam?


The agile revolution that began in software development in the 1990s has been inexorably making its way into mainstream IT organisations.

Today, one of the most adopted agile practices is unit testing, where developers write hundreds of small tests for exercising their own code. Although the benefits of unit testing are widely recognised, there's growing evidence that unit testing might have reached its high-water mark and be entering a period of stagnation or even decline.

Signs that unit testing has hit a wall include the high-profile collapse of Agitar, which was known widely for a good, if expensive unit-testing product. CEO Jerry Rudisin told the SD Times that Agitar's collapse (it was bought by McCabe Software in June was due simply to the fact that "unit testing hasn't taken off as a mainstream practice."

Joe Ponczak, who heads up Condign Software, the makers of automated unit test generation tools and metrics dashboards, says he believes the Agitar failure was brought on by slower-than-desired growth, "not at the level expected from a Silicon Valley startup."

Andrew Glover, president of the agile-development consulting firm Stelligent, blamed Agitar's high price. "Only very large enterprises could afford it. And it was hard to justify paying those prices for a product that simply automated what developers could already do for themselves with free tools," he says.

Why unit testing may be losing its allure

Of course, Agitar is just one company, and its experience is hard to translate as an overall trend -- especially because the major tools used in unit testing are available as free open source, such as JUnit and TestNG for Java and the xUnit frameworks for other languages, as well as the free code-coverage tools Cobertura and Emma. Most of the commercial products focus on automatic generation of unit tests and coverage analysis.

But consider the typical experience of Stelligent's Glover when he consults on development approaches with clients: "When I go into sites, I would love to turn them on to the benefits of the TestNG test framework," he says, "but most organisations do so little unit testing, if any, that I stick with the simple, well-known xUnit tools, so as not to create confusion. Most organisations are just not ready to learn anything more than basic unit testing -- if that."

While languages such as Python and Ruby have active unit-testing cultures, there are signs that, in the Java community at least, resistance has built up against the unit testing idea. I posted the question on unit testing's future in my blog recently, and got several hundred responses. Among those, I identified two major threads of concerns.
One issue is that unit testing seems to offer little value.

"The problem with unit testing is that it increases the cost of development before you know whether what you developed is what you needed. Afterwards when you have a working solution, there's little interest in writing unit tests as what you have works, and your management wants you to move on," wrPeterB.ote one anonymous poster. "Many developers and managers mistakenly think that by cutting testing, they well get feature X out faster. What they don't realise is that you are robbing Peter to pay Paul," wrote Peter B.

The other issue was annoyance over evangelism, which is at the core of much of the agile movement (which began with a signed manifesto and a formulated creed). "Please, please, please, no more evangelism!!" wrote Gabriel C. A segment of the agile development community has adopted unit testing in a radical way, insisting that tests be written before the code.

This approach, called test-driven development, is intended to clarify design before committing to code. Its proponents have been zealous in promoting test-driven development as the best way to program, but for many developers that evangelism has become so overbearing that it makes unit testing as a whole unappealing.

Why you should give unit testing a second look

Unfortunately, these reasons conspire to obscure the benefits of unit testing, so it is frequently rejected at sites in favor of traditional testing, rather than being viewed as an important adjunct to it. That's a mistake. Unit testing has four significant benefits that, in my experience, all organisations that adopt it recognise almost immediately: Defects are identified nearly immediately, and because unit testing isolates their cause and effect, they are also resolved more quickly. As a result, you can be more confident in the schedule. You also get better code-level documentation.

But the first and principal result of unit testing is better code. In the traditional manner of development, code is written and quickly checked by the developer against a narrow set of criteria. Once it works, the developer moves on to other coding. Whether that routine works later on is a question that the developer faces only when examining the larger software package or during the testing cycle. At that point, the developer might easily spend hours with a debugger figuring out why the routine is unexpectedly balking.

These long debug sessions are terribly expensive to projects on several levels. As there is no method for predicting how long it will take to locate and resolve a bug, nor what effects that bug fix will have, there is no way to accurately project schedules.

"Recommended For You"

Software quality is a C-Level responsibility 7 steps to improve software development quality