The soft side of audit

I spent six years working as an IT auditor. It took two years before I was confident that I had a reasonable feel for the job. That came only when I started to master the soft skills of audit.

At first I thought the job required me only to understand the principles and practice of audit, and to work feverishly to learn about the different technical areas I was reviewing. Although I came into audit with a high level of technical expertise my knowledge was specialised and many of the audits took me into topics with which I was unfamiliar.

After a couple of years I realised that what I needed was the judgement, the inter-personal skills and the analytical ability to perform effectively. Crucially, what I also needed was the increasing confidence that I was acquiring and the knowledge that if I keep building on those vital soft skills I’d be able do to the job more effectively. I realised that I was well on the way to becoming a good auditor and that I need no longer be afraid of making a fool of myself.

Don’t take on the techies at their own game!

An auditor must never try to beat the technical experts on their own turf. That is a route to disaster. If you try to pretend that you are also an expert then they will probably humiliate you. Sure, you have to acquire an understanding of what you are reviewing, and you do have to be a fast learner. But you need the judgement, the nose, to home in on the important, sensitive issues and understand what the risks are, how they are mitigated, how they can be controlled, how those controls could be evaded.

You have to be constantly churning over in your mind the facts you’ve learned, assessing their significance and varying your next step in response. You have to be able to cut off your research and investigation when you realise that the risk doesn’t justify the effort.

Auditors must try to understand the context of the area under audit and the auditees must have confidence that auditors will appreciate the problems they have to deal with. If auditors can pitch their approach correctly then auditees will see them as potential allies with whom they can work to resolve organisational, or management, problems that make their job harder. If the auditor comes across as the hard nosed suit from head office who is here to beat people up then the auditees will clam up rather than open up.

Interviewing skills are crucial

You have to be able to ask the right questions, so that the answers come from the technical experts – not the auditor firing off half baked proposals based on irrelevant theory and a poor grasp of the subject matter.

The auditor must also have the confidence to ask seemingly stupid questions. That can be hard for inexperienced professionals who are nervous and do not want to risk appearing foolish. They get an incomplete or possibly misleading answer and take it at face value rather than clarifying the point. Saying “I don’t understand, how do you…” or “that doesn’t really make sense to me, can you rephrase/expand your answer…”, or simply “yes, but why…” can be powerful tools.

An auditor is of course a listener. That is a vital skill. Auditors must not only absorb what is being said, but they have to understand the significance of how it is said, and also what is not said. I would never conduct audit interviews over the phone. I might use the phone to gather information, but I would never regard it as a serious interview. You can learn far more in a face to face interview where you can make eye contact and assess body language. As we used to say, “make sure you can look them in the eyes when they’re lying to you”.

An auditor has to be able to operate like an advocate or barrister in a cross examination; assessing the answers and then varying the follow up to gain confirmation or to expose a weakness, evasion or contradiction in the answer.

You also have to be able to vary the style of question. Sometimes you have to ask open ended questions, designed to let the interviewee talk, without being prompted to give the expected answer. At other times you have to ask very precise questions so you can get confirmation of what you want to find out.

However, it should not come across as a courtroom cross-examination. The auditor needs to be relaxed, but serious and professional, able to give auditees the impression that the audit will be rigorous but fair.

There was a fascinating recent example in Scottish politics of an interview that went wrong. Andrew Neil, a hugely experienced journalist and interviewer, was asking Alex Salmond, the Scottish First Minister about whether an independent Scotland would automatically remain a member of the European Union. The interview turned to the question of whether Salmond had taken legal advice on the point. The relevant part is one minute and 19 seconds in.

Neil; “Have you sought advice from your Scottish law officers on this matter?”

Salmond; “We have, yes, in terms of the debate and obviously…”

Neil (interrupting); “And what did they say?”

Salmond; “You can read that in the documents that we’ve put forward that argue that we would be a successor state”.

Neil; “And what did they say?”

Andrew Neil committed three interviewing errors that can wreck an audit interview. He asked an imprecise question. He needed to know whether Salmond had taken legal advice on the specific question of whether Scotland would automatically remain a member of the EU, but asked if he’d taken advice “on this matter”.

He then interrupted the answer to produce his follow up question without clarifying the answer.

Finally, Neil repeated that follow up question without noticing that Salmond had actually said “yes” to a subtly different question; he had taken legal advice on Scotland’s status as a successor state to the UK. That is obviously highly relevant and closely related to the question of automatic EU membership, but it is not exactly the same question. Salmond had evaded the question. He had provided a clear answer to a different question and led Neil off the track the interviewer would have pursued if he had realised that Salmond had not received legal advice on the specific question of EU membership.

The result of the interview was a huge political row, the details of which will be familiar to Scots and tedious to anyone else. It neatly illustrates how difficult it is to interview someone who wishes to present themselves in the best possible light, a situation familiar to any auditor.

Let me tell you a story

Once the interviews have been conducted, all the analyses performed, all the audit tests completed the auditor then has to pull the findings together into a report. This isn’t just a dry list of findings. The auditor has to be a good communicator, able to tell a coherent narrative, setting out the context, the findings and building up to inescapable conclusions and compelling recommendations. I don’t want to dwell on report writing here, but the ability to write clearly, and tell compelling stories is vital.

The perils of audit scripts

Needless to say this is all difficult and requires not just intelligence and analytical ability but the ability to stay calm under pressure. You have to be able to see the bigger picture, and make other people see it too. The skills of a good auditor can be learned, but not everyone is suitable. The temptation for audit departments is to shy away from the problem and take comfort in scripts.

Perhaps the worst management mistake I’ve seen in auditing is a reliance on scripts, or checklists. They can be a useful prompt or starting point, but if auditors rely too heavily on them, at the expense of their own judgement and experience, then they de-skill the job and stop auditors growing into effective professionals.

When I switched from audit into software testing I was intrigued by the parallels with audit, and how the same tension existed between the traditional script driven approach and the more thoughtful and reflective approach required by exploratory testing. The issues and problems in both professions are the same. It is largely a matter of whether you go for the simple but ineffective, or the harder but more effective.

Audit scripts are intended to capture the right information. In practice they limit the information that is gathered. Script driven auditors capture data, not information, that reflects prior assumptions, which may have been formulated long ago, in a different context. These assumptions aren’t challenged and the auditors frequently conduct the audit as a serious of binary questions. “Have you done X to Y? Yes or no?”. The auditees might answer “no, because…” and be cut off. The auditor has an audit finding, and someone will get kicked.

The checklist approach to auditing is a conscious corporate decision. It is not simply a matter of individual auditors taking refuge in the seductive comfort of a checklist. A healthy audit department would instantly spot the problem and take corrective action. There are probably still too many unhealthy audit departments around.

To put it bluntly, over-reliance on checklists can turn auditors into mere compliance checkers. The job can be done by anyone who is literate. I have seen that style of auditing and it is extremely unhealthy. Auditors are feared, but not respected, a deadly combination for the whole organisation.