Why do you need the report?

Have you ever wondered what the purpose of a report was, whether it was a status report that you had to complete, or a report generated by an application? You may have wondered if there was any real need for the report, and whether anyone would miss it if no-one bothered to produce it.

I have come across countless examples of reports that seemed pointless. What was worse, their existence shaped the job we had to do. The reports did not help people to do the job. They dictated how we worked; production, checking and filing of the reports for future inspection were a fundamental part of the job. In any review of the project, or of our our performance, they were key evidence.

My concern, and cynicism, were sharpened by an experience as an auditor when I saw at first hand how a set of reports were defined for a large insurance company. To misquote Otto von Bismarck’s comment on the creation of laws; reports are like sausages, it is best not to see them being made.

The company was developing a new access controls system, to allow managers to assign access rights and privileges to staff who were using the various underwriting, claims and accounts applications. As an auditor I was a stakeholder, helping to shape the requirements and advising on the controls that might be needed and on possible weaknesses that should be avoided.

One day I was approached by the project manager and a user from the department that defined the working practices at the hundred or so branch offices around the UK and Republic of Ireland. “What control reports should the access control system provide?” was their question.

I said that was not my decision. The reports could not be treated as a bolt on addition to the system. They should not be specified by auditors. The application should provide managers with the information they needed to do their jobs, and if it wasn’t feasible to do that in real time, then reports should be run off to help them. It all depended on what managers needed, and that depended on their responsibilities for managing access. The others were unconvinced by my answer.

A few weeks later the request for me to specify a suite of reports was repeated. Again I declined. This time the matter was escalated. The manager of the branch operations department sat in on the meeting. He made it clear that a suite of reports must be defined and coded by the end of the month, ready for the application to go live.

He was incredulous that I, as an auditor, would not specify the reports. His reasoning was that when auditors visited branches they would presumably check to see whether the reports had been signed and filed. I explained that it was the job of his department to define the jobs and responsibilities of the branch managers, and to decide what reports these managers would need in order to fulfill their responsibilities and do their job.

The manager said that was easy; it was the responsibility of the branch managers to look at the reports, take action if necessary, then sign the reports and file them. That was absurd. I tried to explain that this was all back to front. At the risk of stating the obvious, I pointed out that reports were required only if there was a need for them. That need had to be identified so that the right reports could be produced.

I was dismissed as a troublesome timewaster. The project manager was ordered to produce a suite of reports, “whatever you think would be useful”. The resulting reports were simply clones of the reports that came out from an older access control system, designed for a different technical and office environment, with quite different working practices.

The branch managers were then ordered to check them and file them. The branch operations manager had taken decisive action. The deadline was met. Everyone was happy, except of course the poor branch managers who had to wade through useless reports, and the auditors of course. We were dismayed at the inefficiency and sheer pointlessness of producing reports without any thought about what their purpose was.

That highlighted one of the weaknesses of auditors. People invariably listened to us if we pointed out that something important wasn’t being done. When we said that something pointless was being done there was usually reluctance to stop it.

Anything that people have got used to doing, even if it is wasteful, ineffective and inefficient, acquires its own justification over time. The corporate mindset can be “this is what we do, this is how we do it”. The purpose of the corporate bureaucracy becomes the smooth running of the bureaucracy. Checking reports was a part of a branch manager’s job. It required a mental leap to shift to a position where you have to think whether reports are required, and what useful reporting might comprise. It’s so much easier to snap, “just give us something useful” and move on. That’s decisive management. That’s what’s rewarded. Thinking? Sadly, that can be regard as a self-indulgent, waste of time.

However, few things are more genuinely wasteful of the valuable time of well paid employees than reporting that has no intrinsic value. Reporting that forces us to adapt our work to fit the preconceptions of the report designer gobbles up huge amounts of time and stop us doing work that could be genuinely valuable. The preconceptions that underpin many reports and metrics may once have been justified, and have fitted in with contemporary working practices. However, these preconceptions need to be constantly challenged and re-assessed. Reports and metrics do shape the way we work, and the way we are assessed. So we need to keep asking, “just why do you need the report?”

Teddy bear methods

Introduction

I wrote this article for Testing Planet a couple of months ago. Most of the feedback was positive, but I received some criticism on Twitter that I’ve been unfair on Structured Methods. I want to make it clear that I’m talking about the big, formal methodologies such as SSADM, and BIS Structured Analysis & Design, the particular variant we used where I worked. These are based on the work of Yourdon, Constantine and DeMarco.

My problem isn’t with individual techniques, many of which are valuable. I’ve always been obsessive about structured programming when I code. I was trained to regard that as simply good, responsible practice. I’ve also long been a fan of IDEF0, which I’ve personally found useful both as an auditor and a tester to clarify my understanding of what is going on, and what should be happening but maybe isn’t. IDEF0 is definitely a technique from structured methods.

So the problem isn’t with individual components. It’s with the way they are bolted together into a full blown, formal methodology. The development process is linear and rigid. Practitioners are encouraged, often mandated to follow the whole method, and nothing but the method. It is a brave project manager who deviates from the formal method knowing that if he/she delivers anything that is less than perfect there will be hard questions about failure to follow the prescribed process, which implicitly assumes that it is the only route to the Correct Solution. You then see the sort of scenario I’ve described in my article.

And here is the article

When I was a child my father often used to refer to me with an exasperated smile as “lawyer Jim”. He was amused by the way I would always argue and question, not in a disobedient or truculent way. I just wanted to know what was going on, and why things were happening the way they were. I demanded good explanations, and if the explanations didn’t make sense then the adult could expect a thorough cross-examination till we’d got to the heart of the matter.

I didn’t end up as a lawyer, but it’s not really surprising that I did become an auditor and a tester. I’ve always wanted to understand what’s happening around me. It has always struck me as a challenge if someone says “just because”, or “that’s the way we do things round here”.

When I left university I struggled for the first few years to make any sense at all of the places where I worked. I started off training to be a chartered accountant, but I hated that. It wasn’t the day to day auditing, it was the culture. I struggled to see how we were really doing anything useful, how our work added up to valuable information for the stakeholders. It all seemed a charade, and I wasn’t convinced anyone thought differently; it was just a good way to make money.

Structured Methods – something I just didn’t get

I ended up in IT, at a great place where I learned a huge amount; about business, people, organisations, IBM operating systems and utilities, how to have fun coding, all sorts. What I didn’t really learn much about was how to make the prevailing Structured Methods work.

Structured Methods were “The BIG Thing” then, in the mid 80s. They were intended to revolutionise the way that systems were developed. No longer would a bunch of geeky cowboys hack their way through the code till some ramshackle application could be dumped on the poor users. Structured Methods meant that rigorously trained, professional, software engineers would meticulously construct applications that perfectly reflected the requirements of the users.

There were a couple of problems with this, apart from the obvious one that the results were lousy. Firstly, you can’t specify requirements perfectly in advance in the way that Structured Methods assumed. The whole rickety shebang depended on some pretty ludicrous assumptions about requirements; that these could be specified precisely, accurately and in detail up front, and also that they could be largely inferred from the existing system and its flaws. It was a pretty limited view of what was possible.

The other huge problem was the attempt to make development a mechanical process. You churned out the artefacts from one stage, fed them into the next stage and so on along the production line. Fair enough, if there actually was a production line. The trouble was that that the results of the analysis weren’t really connected to the resulting design, not in any coherent and consistent way. The design depended on old-school, unfashionable things like skill, experience and judgement.

Developers would plod agonisingly through the process, generating a skip-load of incomprehensible documentation that no-one would ever read again, then at the crucial point of turning requirements into a physical design, they’d have to wing it.

Anyway, I didn’t get Structured Methods, and managed to arrange my career so I could go my own sweet way and keep enjoying myself.

I knew that Structured Methods didn’t work. That was pretty obvious. What I didn’t realise was that that was irrelevant. People weren’t using them because they worked, or even truly believed deep down that they worked. So what was going on?

Social defences

Let’s wind back a few decades for some interesting insights. Isabel Menzies Lyth was a distinguished psychoanalyst whose particular specialism was analysing the dynamics of groups and organisations.

In 1959 she set off a bomb under the medical profession in the UK with a paper called “The functions of social systems as a defence against anxiety” (PDF, opens in a new tab). She was writing about nursing, and argued that the main factors shaping an organisation’s structure and processes were the primary task, the technology used and the social and psychological needs of the people. The dominant factor was not the task or even the technology; it was the need for managers and nurses to cope with the stress and anxiety of a tough job.

As a result “social defences” were built to help people cope, and specifically to help the managers cope. The defences identified by Menzies Lyth included rigid processes that removed discretion and the need for decision making by nurses, hierarchical staffing structures, increased specialisation, and nurses being managed as fungible (i.e readily interchangeable) units, rather than skilled professionals.

These defences solidified the culture, structure and processes in hospitals. The rigidity damaged performance of the primary task, i.e. caring for patients. This benefited the management but racked up the stress for the nurses themselves. The defences were therefore strengthened. The response was to adapt the job so that nurses became emotionally distanced from the patients and their outcomes. Patients were increasingly regarded as subjects for treatment, rather than people. Sub-standard performance had to be defined as standard. Acceptance of sub-standard performance became part of the defence mechanism.

Practices in the health professions have certainly changed over the last half century, but Menzies Lyth’s insights into how people deal with stressful jobs in organisations have an important lesson for us.

David Wastell and transtional objects

In the 1990s a British academic, David Wastell, linked Menzies Lyth’s insights with work by Donald Winnicott, a paediatrician and psychoanalyst. Winnicott’s big contribution was the idea of the transitional object (PDF, opens in a new tab).

This is something that helps infants to cope with loosening the bonds with their mother. Babies don’t distinguish between themselves and their mother. Objects like security blankets and teddy bears give them something comforting to cling onto while they come to terms with the beginnings of independence in a big, scary world.

Wastell studied development shops that used Structured Methods. He interpreted his findings in the light of the work of Menzies Lyth and Winnicott. Wastell found that the way that developers used the method, and their mindset, meant that Structured Methods had become a transitional object, i.e. a defence mechanism to alleviate the stress and anxiety of a difficult job (see “The fetish of technique, methodology as a social defence”, not free I’m afraid).

Wastell could see no evidence from his own studies, or from the research literature, to suggest that Structured Methods worked. The evidence was that the resulting systems were no better than the old ones, took much longer to develop and were more expensive.

I could recognise the patterns Wastell was describing. Managers became hooked on the technique and lost sight of the true goal.

“Methodology becomes a fetish, a procedure used with pathological rigidity for its own sake, not as a means to an end. Used in this way, methodology provides a relief against anxiety; it insulates the practitioner from the risks and uncertainties of real engagement with people and problems.”

Teddy bears are certainly great for toddlers. They are cute and cuddly, and they help children as they learn about the world. However, Wastell was at pains to point out that it is deeply harmful when developers cling on to their own transitional objects. Systems development has to be a process of learning, he argued (see “Learning dysfunctions in information systems development…”). Seizing on Structured Methods as transitional objects, and obsessing about compliance with a rigid technique, prevented learning.

In chasing a warped vision of “maturity” the proponents and users of Structured Methods were perversely refusing to grow into true maturity. Practitioners became trapped, and never grew beyond the teddy bear stage.

Like the nurses and managers in Menzies Lyth’s study, developers were defining sub-standard levels of performance as the professional standard. The rigid techniques were not really helping the nurses who treating patients. The managerial processes did help the managers to deal with stress and anxiety, but they made the job harder for the nurses to cope with, which in turn led to nurses concentrating on process rather than patients.

Likewise in systems development techniques that help the managers to cope with stress and anxiety, and which give them an illusory, reassuring sense of control, are harmful to the team members. They have to cope by focussing on the technique, mastery of the tool, or compliance with the standard. In doing that they can feel that they are doing a good job – so long as they don’t have to think about whether they are really working towards the true ends of the organisation.

Trusting the method, or asking awkward questions?

Sadly, the more mature and thoughtful nurses in Menzies Lyth’s study were less likely to complete their training and stay in the profession. It was the less thoughtful, less emotionally mature nurses who could adapt more easily to the regime.

Perhaps that was why I was uncomfortable with Structured Methods, and with the idea of standards and rigid processes. My parents always encouraged my awkward questioning. It was important to question, argue and think. If you want to grow up you have to face the reality that worthwhile jobs are going to bring some stress and anxiety. We have to be honest enough to face up to that, rather than pretend that a cute and cuddly new standard, method or technique will conjure up an easier life. Maybe I was right in the first few years after I left university and the people I was watching really didn’t understand why they were working the way they were, or how their activities helped achieve anything worthwhile.

Tough jobs don’t become easy just because you redefine them in a way that makes them feel easier. That way you just risk detaching your activities from the true objectives you’re being paid to achieve. Sooner or later someone is going to catch on and realise that you’re getting paid for something that’s of little value. If you’re not doing anything useful there will always be someone who can do it cheaper.

High quality systems depend on good people, with the right skills and experience. Their managers have to be strong enough, and honest enough, to confront problems, and recognise psychological strains. They shouldn’t evade them. If they want something to help them cope with the stress and anxiety, buy them some teddy bears!