The Post Office Horizon IT scandal, part 3 – audit, risk & perverse incentives

In the first post of this three part series about the scandal of the Post Office’s Horizon IT system I explained the concerns I had about the approach to errors and accuracy. In the second post I talked about my experience working as an IT auditor investigating frauds, and my strong disapproval for the way the Post Office investigated and prosecuted the Horizon cases. In this, the final part, I will look at the role of internal audit and question the apparent lack of action by the Post Office’s internal auditors.

Independence and access to information

There’s a further aspect to the Horizon scandal that troubles me as an ex-auditor. In 2012, after some pressure from a Parliamentary committee, the Post Office commissioned the forensic IT consultancy Second Sight to review Horizon. Second Sight did produce a report that was critical of the system but they could not complete their investigation and issue a final report. They were stymied by the Post Office’s refusal to hand over crucial documents, and they were eventually sacked in 2015. The Post Office ordered Second Sight to hand over or destroy all the evidence it had collected.

An experienced, competent IT audit team should have the technical expertise to conduct its own detailed system review. It was a core part of our job. I can see why in this case it made sense to bring in an outside firm, “for the optics”. However, we would have been keeping a very close eye on the investigation, assisting and co-operating with the investigators as we did with our external auditors. We would have expected the investigators to have the same access rights as we had, and these were very wide ranging.

We always had the right to scope our audits and investigations and we had the right to see any documents or data that we considered relevant. If anyone ever tried to block us we would insist, as a matter of principle, that they should be overruled. This was non-negotiable. If it was possible to stymie audits or investigations by a refusal to co-operate then we could not do our job. This is all covered in the professional standards of the Institute of Internal Auditors. The terms of reference for the Post Office’s Audit, Risk and Compliance Committee (PDF, opens in new tab) makes its responsibilities clear.

“The purpose of the charter will be to grant Internal Audit unfettered access to staff, data and systems required in the course of discharging its responsibilities to the Committee…

Ensure internal audit has unrestricted scope, the necessary resources and access to information to fulfil its mandate.”

I am sure that a good internal audit department, under the strong management that I knew, would have stepped in to demand access to the relevant records in the Horizon case on behalf of the external investigators, and would have pursued the investigation themselves if necessary. It’s inconceivable that we would have let the matter drop under management pressure.

Internal auditors must be independent of management, with a direct reporting line to the board to protect them from attempted intimidation. “Abdication of management responsibilities” was the nuclear phrase in our audit department. It was only to be used by the Group Chief Auditor. He put it in the management summary of one of my reports, referring to the UK General Manager. The explosion was impressive. It was the best example of audit independence I’ve seen. The General Manager stormed into the audit department and started aggressively haranguing the Chief Auditor, who listened calmly then asked. “Have you finished? Ok. The report will not be changed. Goodbye”. I was in awe. You can’t intimidate good auditors. They tend to be strong willed. The weak ones don’t last long, unless they’re part of a low grade and weak audit department that has been captured by the management.

Risk and bonuses

The role of internal audit in the private sector recognises the divergent interests of the executives and the owners. The priority of the auditors is the long term security and health of the company, which means they will often look at problems from a different angle than executives whose priority might be shaped by annual targets, bonuses and the current share price. The auditors keep an eye on the executives, who will often face a conflict of interest.

Humans struggle to think clearly about risk. Mechanical risk matrices like this one (from the Health and Safety Executive, the UK Government inspectorate responsible for regulating workplace safety) serve only to fog thinking. A near certain chance of trivial harm isn’t remotely the same as a trivial chance of catastrophic damage.

UK HSE risk matrix

Senior executives may pretend they are acting in the interests of the company in preventing news of a scandal emerging but their motivation could be the protection of their jobs and bonuses. The company’s true, long term interests might well require early honesty and transparency to avoid the risk of massive reputational damage further down the line when the original scandal is compounded by dishonesty, deflection and covering up. By that time the executives responsible may have moved on, or profited from bonuses they might not otherwise have received.

A recurring theme in the court case was that the Post Office’s senior management, especially Paula Vennells, the chief executive from 2012 to 2019, simply wanted the problem to go away. Their perception seems to have been that the real problem was the litigation, rather than the underlying system problems and the lives that were ruined.

In an email, written in 2015 before she appeared in front of a Parliamentary committee, Vennells wrote.

“Is it possible to access the system remotely?

What is the true answer? I hope it is that we know it is not possible and that we are able to explain why that is. I need to say no it is not possible and that we are sure of this because of xxx [sic] and we know this because we had the system assured.”

Again, in 2015, Vennells instructed an urgent review in response to some embarrassingly well informed blog posts, mainly about the Dalmellington Bug, by a campaigning former sub-postmaster, Tim McCormack. Vennells made it clear what she expected from the review.

“I’m most concerned that we/our suppliers appear to be very lax at handling £24k. And want to know we’ve rectified all the issues raised, if they happened as Tim explains.”

These two examples show the chief executive putting pressure on reviewers to hunt for evidence that would justify the answer she wants. It would be the job of internal auditors to tell the unvarnished truth. No audit manager would frame an audit in such an unprofessional way. Reviews like these would have been automatically assigned to IT auditors at the insurance company where I worked. I wonder who performed them at the Post Office.

When the Horizon court case was settled Vennells issued a statement, an apology of sorts.

“I am pleased that the long-standing issues related to the Horizon system have finally been resolved. It was and remains a source of great regret to me that these colleagues and their families were affected over so many years. I am truly sorry we were unable to find both a solution and a resolution outside of litigation and for the distress this caused.”

That is inadequate. Expressing regret is very different from apologising. I also regret that these lives were ruined, but I hardly have any responsibility. Vennells was “truly sorry” only for the litigation and its consequences, although that litigation was what offered the victims hope and rescue.

Vennells resigned from her post in the spring of 2019, eight months before the conclusion of the Horizon court case. In her last year as chief executive Vennells earned £717,500 (PDF, opens in new tab), only £800 less than the previous year. She lost part of her bonus because the Post Office was still mired in litigation, but it hardly seems to have been a punitive cut. Over the course of her seven years as chief executive, according to the annual reports, she earned £4.5 million, half of which came in the form of bonuses. In that last year when she was penalised for the ongoing litigation she still earned £389,000 in bonuses.

These bonuses are subject to clawback clauses (according to the annual reports, available at the last link);

“which provide for the return of any over-payments in the event of misstatement of the accounts, error or gross misconduct on the part of an Executive Director.”

Bonuses for normal workers reflect excellent performance. In the case of chief executives the criterion seems to be “not gross misconduct”.

I have dismissed the risk matrix above for being too mechanical and simplistic. There’s a further criticism; it ignores the time it takes for risks to materialise into damage. A risk that is highly unlikely in any particular year might be almost certain over a longer period. It depends how you choose to frame the problem. To apply a crude probability calculation, if the chance of a risk blowing up in a single year is 3%, then there is a 53% chance it will happen at some point over 25 years. If a chief executive is in post for seven years, as Paula Vennells was, there is only a 19% chance of that risk occurring.

These are crude calculations, but there is an important and valid underlying point; a risk that might be intolerable to the organisation might be perfectly acceptable to a chief executive who is incentivised to maximise earnings through bonuses, and push troubling risks down the line for someone else to worry about.

No organisation should choose to remain in the intolerable risk cell, yet Vennells took the Post Office there. That was almost certainly inadvertent, but the system was so perverse that it would have made financial sense for her to make that choice. The Post Office was very likely to lose the Horizon litigation, with massive damage. It wouldn’t happen while she was in post, and it would be extremely unlikely that fighting the case aggressively would be regarded as gross misconduct.

Perverse incentives often tempt managers, and also politicians, to ignore the possibility of dreadful outcomes that are unlikely while they are in post and would force them to incur expense or unpopularity to prepare for. The odds are good that irresponsible management will be rewarded for being wrong and will have left with their hefty bonuses before disaster strikes. On the other hand you can get sacked for doing the right thing long before justification is obvious.

This is, or at least it should be, a big issue for internal auditors who have to keep a sharp eye on risk and misaligned incentives. All too often the only people with a clear eyed, dispassionate understanding of risk are those who are gaming the corporate system. The Post Office’s internal auditors seem to have fallen down on the job here. Even setting aside the human tragedies, the risks to the Post Office posed by the Horizon system and the surrounding litigation should have been seen as intolerable.

Role of internal audit when organisations move from the public to private sector

This all raises questions about corporate governance and the role of internal audit in bodies like the Post Office that sit between the public and private sectors. The Post Office is owned by the UK government, but with a remit of turning itself into a self-sustaining company without government subsidy. The senior executives were acting like private sector management, but with internal auditors who had a public sector culture, focusing on value for money and petty fraud. There are endless examples of private sector internal auditors losing sight of the big picture. However, a good risk-based audit department will always be thinking of those big risks that could take the company down.

Public bodies are backed by the government and can’t fail in the same way as a private company. When they move into the private sector, the management culture and remuneration system exposes the organisation to a new world of risks. So how do their internal auditors respond? In the case of the Post Office the answer is; badly. The problems were so serious that the internal auditors would have had a professional responsibility to bypass the senior executives and escalate them to board level, and to the external auditors. There is no sign that it happened. The only conclusion is that the Post Office’s internal auditors were either complicit in the Horizon scandal, or negligent. At best, they were taking their salaries under false pretences.

Conclusion

At almost every step, over many years, the Post Office handled the Horizon scandal badly, inexcusably so. They could hardly have done worse. There will be endless lessons that can, and will be drawn, from detailed investigation in what must be the inevitable inquiry. However, for software testers and for IT auditors the big lesson they should take to heart is that bad software, and dysfunctional corporate practices, hurt people and damage lives. The Post Office’s subpostmasters and subpostmistresses were hard working, decent, business people trying to make a living and provide for their families. They were ruined by a cynical, incompetent corporation. The compensation they will receive appears substantial, but it’s hardly enough and it will be greatly reduced by the need to cover the massive legal costs that the Post Office ran up with their obstructive and delaying tactics. The subpostmaster and subpostmistresses deserve better.