Questioning auditors questioning testing

Since I chose the title for my tutorial, “questioning auditors questioning testing” at EuroSTAR this year I have become increasingly aware of how relevant both the title and the topic are.

Testers are used to being questioned, whether it’s by project managers, senior management, users – and auditors too of course. It’s easy to get wrapped up in the problems and pressures of our own profession and forget that other people are working under scrutiny too.

When I worked as an internal auditor I always knew we had to demonstrate that we were “adding value” to the company. I dislike that phrase, but the underlying point is crucial. Auditors can go through the motions and produce detailed, unhelpful reports that are of little real value to the organisation. Alternatively they can get to the heart of their role and provide advice that makes a difference. Only by doing so can they justify their salaries.

Auditors are under scrutiny too

There are certainly some audit departments that just go through the motions. A couple of decades ago such auditors would have accounted for the vast majority of the profession. Happily my audit experience was in a company that was in the vanguard of the new approach to audit. Our work was risk based, always aware of the wider context and absolutely never driven by checklists. We were in the minority then but the tide has now turned emphatically. I don’t know whether the time-serving, low value, low quality auditors are now in the minority but they are certainly under serious pressure from their own profession and the regulators to up their game and adjust to the modern world.

The UK branch of the Institute of Internal Auditors has just issued guidance to its members that internal audit departments in financial services should include within their scope…

…the risk and control culture of the organisation. This should include assessing whether the processes (e.g. appraisal and remuneration), actions (e.g. decision making) and “tone at the top” are in line with the values, ethics, risk appetite and policies of the organisation.

Audit departments should have an internal QA team with highly experienced auditors. Their role would be to “ensure” (the IIA’s bold choice of word) that audit plans and reports are risk based, with opinions that are “adequately evidenced”. These uber-auditors would be expected to challenge their colleagues, even their own management, and report directly to the board if there are problems. It sounds an interesting job!

The world has changed since the turn of the millenium. We have seen huge corporations crash, and banks that were too big to fail crash spectacularly. There has been widespread and legitimate disappointment, anger even, at the performance of both external and internal auditors.

Auditors are going to have to be more accountable. They in turn are going to be audited. That’s the way that the world is going.

Will auditors who are increasingly expected to query the culture of a company and the “tone at the top”, who are themselves subject to intense scrutiny, really be content with working their way down a checklist? Will they really be happy to tick boxes as you show them the shelfware, the endless documents meticulously completed to IEEE829 standard templates? Or will they want to sit down with you and understand the risks you identified and investigated so that they can relate them to the risks that keep the stakeholders awake at night?

Just today I was looking at the questions asked about testing practices by a company that sells liability insurance. The questions reflect a dated view of testing, and assume that effective, responsible testing follows the old document driven approach.

Risk management is vital for all companies, but those in financial services have to be particularly skilled. It’s not simply a matter of protecting the company. They are selling their expertise at handling risk though their products.

Any insurer who views testing in the same way as that liability insurer has lost sight of the true risks. That would be a legitimate concern for the auditors of that company. Now, in the UK at least, the auditors will have an explicit responsibility to challenge poor practice.

Why am I telling you this? Testers have to work constructively with auditors, and they have to understand where they are coming from.

Winning friends and influencing auditors

The subtitle of my tutorial is a conscious nod towards Dale Carnegie’s phenomenally successful self-help book “How to win friends and influence people”. You might dismiss Carnegie’s book as a trite collection of obvious pieces of advice, but the sorry truth is that many of the basic truths about human nature are still routinely neglected in business.

I chose the subtitle because I wanted to stress the importance of understanding auditors and where they are coming from, rather than assuming that they are the hard nosed suits from head office. If we are defensive, expecting a battle with auditors then that is what we are likely to get. The relationship between auditors and testers is a human relationship every bit as much as a business one. Whether that relationship is good or bad is largely a matter of how well the personal relationship is handled.

One of Carnegie’s key points is a quote from the Roman writer Publilius Syrus.

We are interested in others when they are interested in us.

If we show an interest in other peoples’ problems then they will be interested in ours. Show an interest in what the auditors are trying to do and they are more likely to take a positive interest in your work.

From personal experience I know how scary it is to embark on a new audit when you can’t use a checklist that pretends to provide all the questions and answers. Look at it from the auditors’ point of view. They have to learn quickly about a new project, a new business area, or some new technology. They have to be able to discuss the important risks and issues with people who are highly experienced and possibly hostile. The auditors know that within a few weeks (at most) they will have to issue an intelligent report that tells a persuasive story identifying problems and possible improvements. It was always a relief to meet people who understood our role, who wanted to work with us and who saw us as valuable allies in their attempt to do a better job.

Auditors can help testers

I have seen both testing and auditing change enormously over the past couple of decades. Enlightened testers and auditors now have far more in common with each other than they do with the old school, checklist/script practitioners in their own professions.

Both testers and auditors are, or should be, enquiring, inquisitive people trying to provide more information about new products and applications and crucially about the risks that their employers and clients are facing.

If auditors are viewed in that positive light then testers should want to get involved with them as early as possible, and to keep speaking to them. It shouldn’t be a one way conversation, with testers justifying themselves. Good auditors will have a different perspective from normal project members. They will help testers to see a bigger picture, to understand the business risks better, and their input should help testers to come up with important new ideas for testing.

My tutorial won’t be a simple matter of telling people the magic tricks, or correct phrases to get the auditors off your back. What I do hope it will provide is an insight into why good auditors will support good testing, rather than impressive documentation. I also hope it will show testers how they can fight back constructively against the poor auditors who are still out there. If auditors are giving you a hard time over your failure to write unnecessary documents then it is good to have the ammunition you need to defend yourself, to turn the tables on them and help them to do a better job!

One thought on “Questioning auditors questioning testing

  1. Pingback: Five Blogs – 9 September 2013 | 5blogs

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s