“ISO 29119: Why it is Dangerous to the Software Testing Community” (from uTest)

This article originally appeared on the uTest blog on September 2nd 2014.

Two weeks ago, I gave a talk at CAST 2014 (the conference of the Association for Software Testing) in New York, titled “Standards: Promoting quality or restricting competition?”


It was mainly about the new ISO 29119 software testing standard (according to ISO, “an internationally agreed set of standards for software testing that can be used within any software development life cycle or organization”), though I also wove in arguments about ISTQB certification.

My argument was based on an economic analysis of how ISO (the International Organization for Standardization) has gone about developing and promoting the standard. ISO’s behavior is consistent with the economic concept of rent seeking. This is where factions use power and influence to acquire wealth by taking it from others — rigging the market — rather than by creating new wealth.

I argued that ISO has not achieved consensus, or has even attempted to gain consensus, from the whole testing profession. Those who disagree with the need for ISO 29119 and its underlying approach have been ignored. The opponents have been defined as irrelevant.

If ISO 29119 were expanding the market, and if it merely provided another alternative — a fresh option for testers, their employers and the buyers of testing services — then there could be little objection to it. However, it is being pushed as the responsible, professional way to test — it is an ISO standard, and therefore, by implication, the only responsible and professional way.

What is Wrong With ISO 29119?

Well, it embodies a dated, flawed and discredited approach to testing. It requires a commitment to heavy, advanced documentation. In practice, this documentation effort is largely wasted and serves as a distraction from useful preparation for testing.

Such an approach blithely ignores developments in both testing and management thinking over the last couple of decades. ISO 29119 attempts to update a mid-20th century worldview by smothering it in a veneer of 21st century terminology. It pays lip service to iteration, context and Agile, but the beast beneath is unchanged.

The danger is that buyers and lawyers will insist on compliance as a contractual requirement. Companies that would otherwise have ignored the standard will feel compelled to comply in order to win business. If the contract requires compliance, then the whole development process could be shaped by a damaging testing standard. ISO 29119 could affect anyone involved in software development, and not just testers.

Testing will be forced down to a common, low standard, a service that can be easily bought and sold as a commodity. It will be low quality, low status work. Good testers will continue to do excellent testing. But it will be non-compliant, and the testers who insist on doing the best work that they can will be excluded from many companies and many opportunities. Poor testers who are content to follow a dysfunctional standard and unhelpful processes will have better career opportunities. That is a deeply worrying vision of the future for testing.

The response to my talk

I was astonished at the response to my talk. I was hoping that it would provoke some interest and discussion. It certainly did that, but it was immediately clear that there was a mood for action. Two petitions were launched. One was targeted at ISO to call for the withdrawal of ISO 29119 on the grounds that it lacked consensus. This was launched by the International Society for Software Testing.

The other petition was a more general manifesto that Karen Johnson organized for professional testers to sign. It allows testers to register their opposition to ISTQB certification and attempts to standardize testing.

A group of us also started to set up a special interest group within the Association for Software Testing so that we could review the standard, monitor progress, raise awareness and campaign.

Since CAST 2014, there has been a blizzard of activity on social media that has caught the attention of many serious commentators on testing. Nobody pretends that a flurry of Tweets will change the world and persuade ISO to change course. However, this publicity will alert people to the dangers of ISO 29119 and, I hope, persuade them to join the campaign.

This is not a problem that testers can simply ignore in the hope that it will go away. It is important that everyone who will be affected knows about the problem and speaks out. We must ensure that the rest of the world understands that ISO is not speaking for the whole testing profession, and that ISO 29119 does not enjoy the support of the profession.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s