The longer my career lasts the more aware I am of the importance of Gerald Weinberg’s Second Law of Consulting (from his book “The Secrets of Consulting”), “No matter what the problem is, it’s always a people problem.”
The first glimmer of light that illuminated this truth was when I came across the term “goal displacement” and reflected on how many times I had seen it in action. People are given goals that aren’t quite aligned with what their work should deliver. They focus on the goals, not the real work. This isn’t just an incidental feature of working life, however. It is deeply engrained in our psychological make-up. There is a long history of academic work to explain this phenomenon.
Focal and subsidiary awareness
I’ll start with Michael Polanyi. In his book “Personal Knowledge”, Polanyi makes a distinction between focal and subsidiary awareness. Focal awareness is what we consciously think about. Subsidiary awareness is like tacit knowledge. We don’t think about the mechanics of holding a hammer to drive in a nail. We think about what we are trying to achieve. If we try to focus on the mechanics of holding the hammer correctly, and consciously aim for the nail then we are far more likely to make a painful mess of things. Focal and subsidiary awareness are therefore, in a sense, mutually exclusive. As Polanyi puts it.
“If a pianist shifts his attention from the piece he is playing to the observation of what he is doing with his fingers while he is playing it, he gets confused and may have to stop. This happens generally if we switch our focal attention to particulars of which we had previously been aware only in their subsidiary role.
Our attention can hold only one focus at a time… it would be hence contradictory to be both subsidiarily and focally aware of the same particulars at the same time.”
Does this apply in organisational life too, as well as to musicians and carpenters performing skilled physical activities? I think it does. We have often focused too closely on the process of software development and of testing and lost sight of the end we are trying to reach. Formal processes, prescriptive methods and standards encourage exactly that sort of misplaced focus.
Thorstein Veblen and trained incapacity
This problem of misplaced focus has long been observed by organisational psychologists and sociologists. A full century ago, in 1914, Thorstein Veblen identified the problem of trained incapacity.
People who are trained in specific skills tend to lose the ability to adapt. Their response has worked in the past, and they apply it regardless thereafter. They focus on responding in the way they have been trained, and cannot see that the circumstances require a different response. Their training has rendered them incapable of doing the job effectively unless it fits their mental framework. This is “trained incapacity”. They have been trained to be useless. The phenomenon applies to all workers, but especially to managers.
However, the problem that Veblen identified was worse than that. Highly specialised training and education meant that people were increasingly becoming expert in narrower fields and their areas of ignorance were increasing. When they entered the active workforce their jobs required wider skills and knowledge than their education had given them, but they were unable to contribute effectively to those other areas. They focussed on what they knew. Veblen was especially concerned about business school graduates.
“[These schools’] specialization on commerce is like other specializations in that it draws off attention and interest from other lines than those in which the specialization falls, thereby widening the candidate’s field of ignorance while it intensifies his effectiveness within his specialty. The effect, as touches the community’s interest in the matter, should be an enhancement of the candidate’s proficiency in all the futile ways and means of salesmanship and “conspiracy in restraint of trade” together with a heightened incapacity and ignorance bearing on such work as is of material use.”
A way of not seeing
In 1935 Kenneth Burke built on Veblen’s work, arguing that trained incapacity was;
“that state of affairs whereby one’s very abilities can function as blindnesses.”
People can focus on the means or the ends, not both, and their specific training in prescriptive methods or processes leads them to focus on the means. They do not even see what they are missing.
“A way of seeing is also a way of not seeing- a focus on object ‘A’ involves a neglect of object ‘B’.”
Robert Merton Robert Merton made the point more explicitly in 1957 when he introduced the concept of goal displacement.
“Adherence to the rules… becomes an end in itself… Formalism, even ritualism, ensues with an unchallenged insistence upon punctilious adherence to formalised procedures. This may be exaggerated to the point where primary concern with conformity to the rules interferes with the achievement of the purposes of the organization.”
Why do we think we are different?
So the problem had been recognised before software development was even in its infancy. How did it come to be such a pervasive problem in our profession? What possible reason could there be for thinking that we are different in software developing and testing? Why would we think we are immune from these problems?
I explored some of the reasons earlier this year in Teddy Bear Methods. Software development is difficult and stressful. It is tempting to seek refuge in neat, ordered structures. In that article I talked about social defences, transitional objects and how slavishly following prescriptive processes and methods can be become a fetish.
However, there is an over-arching explanation; functional stupidity. This was identified by Alvesson and Spicer in a fascinating paper in the Journal of Management Studies in 2012, ”A Stupidity-Based Theory of Organizations” (PDF, opens in new tab).
The concept is rather more nuanced than the headline grabbing name suggests. It is no glib piece of cod psychology; it is soundly rooted in organisational psychology and sociology and in management theory.
Organisations can function more smoothly if employees suspend their critical thinking faculties. It can actually be beneficial if they do not question the validity of management directives, if they don’t think about whether the actions they have to take are justified, and if they don’t waste cognitive effort thinking about whether their work is aligned with the objectives of the organisation.
In large organisations the goal towards which many employees are working is effectively the smooth running of the bureaucracy. Functional stupidity does help things run smoothly. It can be beneficial for compliant employees too. The people who thrive are those who play the game by the rules and don’t question whether the “game” is actually aligned with the objectives of the organisation.
However, where functional stupidity is beneficial it is in organisations operating in a fast moving, relatively well understood environment. In these cases fast and efficient action and reaction may be more important than reflective analysis, though it still carries serious dangers.
On the other hand if the environment is less well understood and there is a need to reflect and learn, then functional stupidity can be disastrous. Apart from a failure to learn from, or even detect mistakes, functional stupidity can commit the organisation to damaging initiatives, while corroding employee morale and effectiveness. Even if the organisation as a whole might be suited to functional stupdity there are roles where it is entirely inappropriate.
Software testing is exactly such a role. Testers must question, analyse, reflect and learn. These are all activities that functional stupidity discourages.
Management fads, lack of evidence and ISO 29119
Alvesson and Spicer refer to a further, damaging effect of functional stupidity that has particular relevance to the debate about ISO 29119. They argue that managers are prone to getting caught up in enthusiasm for unproven initiatives.
”Most managerial practices are adopted on the basis of faulty reasoning, accepted wisdom, and complete lack of evidence.
…organizations will often adopt new practices with few robust reasons beyond the fact that they make the company ‘look good’ or that ‘others are doing it’… Refraining from asking for justification beyond managerial edict, tradition or fashion, is a key aspect of functional stupidity.”
Does ISO 29119 fall into this category? Dr Stuart Reid, convener of the ISO 29119 Working Group is a surprising source of compelling evidence to support the claim.
He has conceded that there is no evidence of the standard’s efficacy and that the people who buy testing services do not understand what they are buying (see the slides from his presentation at ExpoQA14 in Madrid in May, with my added emphasis).
Yet he hopes that they will nevertheless write contracts that mandate the use of the standard (PDF, opens in new tab).
This standard will impose on testers working practices that are only loosely aligned with the real objective of testing. It will provide fertile breeding grounds for goal displacement. Will functional stupidity ease the way for ISO 29119? I fear the worst.
I asked why we think we are different in software development and testing. The question is poorly framed. It’s not that we think we are different. The problem is that we, as a global testing community, are not thinking enough. Far too many of us are simply going with the flow. Thousands have unthinkingly adopted functional stupidity as a career move. ISO 29119? That will do nicely.
“No matter what the problem is, it’s always a people problem.”
Any organisational initiative, or new methodology, or new standard that ignores that rule will not work. The lessons have been there for decades. We only have to look for them.