Why do organisations do things that appear to be downright irrational? The question is of vital importance to testing because many corporations undertake testing in a way that is ineffective and expensive, following techniques (and standards) that lack any basis in evidence, and indeed which are proven to be lousy. If we are to try and understand what often seems to be senseless we have to think about why organisations behave the way that they do. It’s pointless to dismiss such behaviour as stupid. Big organisations are usually run by very intelligent people, and senior managers are generally acting rationally given the incentives and constraints that they see facing them.
In order to make sense of their behaviour we have to understand how they see the world in which they are competing. Unless you are working for a specialist testing services firm you are engaged in a niche activity as far as senior management is concerned. If they think about testing at all it’s an offshoot of software development, which itself is probably not a core activity.
Whatever wider trends, economic forces or fads are affecting corporations, as perceived by their management, will have a knock-on impact all the way through the organisation, including testing. If these wider factors are making an organisation think about adopting testing standards then it is extremely difficult to argue against them using evidence and logic. It’s hard to win an argument that way if you’re dealing with people who are starting from a completely different set of premises.
I’ve looked at this sort of topic before, concentrating more on individuals and the way they behave in organisations. See “Why do we think we’re different?” and “Teddy bear methods”. Here I want to develop an argument that looks at a bigger picture, at the way big corporations are structured and how managers see their role.
Bureaupathology and the denigration of competence
I recently came across this fascinating paper, “Bureaupathology: the denigration of competence”, by Edward Giblin from way back in 1981. As soon as I saw the title I couldn’t wait to read it. I wasn’t disappointed. Giblin identified problems that I’m sure many of you will recognise. His argument was that big corporations often act in ways that are inconsistent with their supposed goals, and that a major reason is the way the hierarchy is structured and managers are rewarded.
Giblin drew on the work of the German sociologist Klaus Offee, who argued against the widely accepted belief that managers are rewarded according to their performance. The more senior the manager the less relevant is specific job knowledge and the more important are social skills. These social skills and the way they are applied are harder to assess than straightforward task skills.
Offee distinguished between “task-continuous” organisations and those which are “task-discontinuous”. In a task-continuous organisation the hierarchy and therefore seniority is aligned with the core task. As you look further up the hierarchy you find deeper expertise. In a task discontinuous organisation you find quite distinct, separate skills at the higher levels.
Good examples of task continuous organisations would be small family businesses, or business, run by experienced experts, with skilled workers and apprentices underneath them. Task discontinuous organisations are far more common nowadays. Modern corporations are almost invariably run by specialist managers who have only a loose grip on the core activity, if they have any proficiency at all. I think this distinction between the two types of organisation has big implications for software testing, and I’ll return to that in my next post.
Social skills trump technical skills
The perceived ability of managers in large corporations is mainly dependent on their social skills, how good they are at persuading other people to do what they want. Such abilities are hard to measure, and managers are more likely to be rewarded for their ability to play the corporate game than for driving towards corporate objectives. Conventional performance evaluation and compensation techniques lack credibility; they don’t effectively link performance with rewards in a way that helps the business. Back in the early 1960s Victor Thompson coined the term bureaupathology to describe organisations in which the smooth running of the bureaucracy took precedence over its real work. In such organisations the bureaucracy is no longer a means to an end. It has become the end. In bureaupathic organisations managers have the freedom and incentive to pursue personal goals that are at variance with the official corporate goals.
With senior managers pursuing their own agenda Giblin argued that twin systems of authority emerge in large organisations. There is the official system, exercised by the generalist managers, and there is an unofficial system utilised by experienced staff relying on “surreptitious behaviour to accomplish the work of the organisation”. With much of the real work being carried out informally, even invisibly, the senior managers’ perspective naturally becomes warped.
“As many generalists either didn’t have the technical knowledge to begin with, or allowed it to atrophy over time, they depend solely on the power of their position to maintain their status. Their insecurity… leads to needs that seem irrational from the standpoint of organisational goals, because these needs do not advance these goals. This also leads them, defensively, to denigrate the importance of the technical competence they don’t have.”
Giblin cited the example of a consultancy firm in which the senior partners spent their time on administration and supervision of the professional consultants, with minimal client contact.
“(It) became fashionable among the administrators to refer to outstanding consultants as ‘mere technicians’. This despite the fact that such ‘mere technicians’ were the firm’s only revenue-producing asset.”
Even though these observations and arguments were being made several decades ago the results will be familiar to many today. I certainly recognise these problems; a lack of respect for key practitioners, and the informal networks and processes that are deployed to make the organisation work in spite of itself.
Big corporations charge further down the wrong road
Giblin had several suggestions about how organisations could improve matters. These focused on simplifying the management hierarchy and communication, re-establishing the link between managers and the real work and pushing more decision making further down the hierarchy. In particular, he advocated career development for managers that ensured they acquired a solid grounding in the corporation’s business before they moved into management.
The reason I found Giblin’s paper so interesting is that it described and explained damaging behaviour with which I was very familiar and it came up with practical, relevant solutions that have been totally ignored. Over the following decades the problems identified by Giblin grew worse and corporations did the exact opposite of what he recommended. This didn’t happen by accident. It was the result of global trends that swamped any attempts individuals might have made to adopt a more rational approach. These trends and their repercussions have had a significant impact on testing, and they will continue to do so. I will develop this in my next post, “Digital Taylorism & the drive for standardisation“.
James Christie's Blog 