Usability and external suppliers – part 1

In my time I’ve worked as an in-house developer and as an external supplier. I’ve worked on fast developments that followed the Nike Methodology (just do it – get coding and find out what you can do) and formal, rigid monsters which had big parties when we successfully negotiated our way through the gauntlet of each quality gate.

One thing that intrigues me is the effect on usability of the differing approaches, and in particular the implications of using external suppliers. It’s not just that different approaches have different consequences. There’s nothing surprising in that. The intriguing thing is that, in this case, the implications are not widely acknowledged. It’s an inconvenient truth.

The more distant, formal and contractual the relationship between developers and users, the less likely it is that usability principles will be incorporated and that the end users will get a product they enjoy using.

This isn’t a problem that arose with e-commerce and web developments. It dates way back to the days when all software developments were applications for employees to use. Academics spotted the problems, but their work didn’t really percolate through to the consciousness of the practitioners.

Lewis & Rieman, Holmlid & Artman (both PDFs & open in new windows), Artman on his own, and Grudin all made similar points regarding procurement, external suppliers and contracts.

Unfortunately only the Holmlid and Artman article is free, though the Lewis and Rieman e-book has a modest suggested donation.

The conclusion of these writers was that using external suppliers was likely to have a damaging impact on the usability of the application. With external contracts there is more pressure and temptation to go for a traditional linear approach.

Effective usability engineering entails iteration, which in turn requires flexible contracts with estimates and costs being revised repeatedly. This is a frightening prospect for both sides; with the supplier scared of being committed to a job which cannot be sized effectively, and the client equally scared of being tied into a contract with no cost cap, and no easy exit without writing off the work already paid for.

In 1995 a workshop was held by the ACM’s Special Interest Group on Computer-Human Interaction to consider the challenges of introducing usability to US government developments. In addition to the general problems faced by all IT projects the participants concluded that very few invitations to tender mentioned usability beyond vague and subjective aspirations. Suppliers naturally didn’t build into their costing any features that were not explicitly mandated, and even after winning the contract were reluctant to provide them lest they get a reputation for cost over-runs.

The research is dated, but I don’t think that anything has essentially changed. The problems weren’t rooted in the technology, or development techniques. The problems were a human reaction to the pressures and incentives of a particular style of procurement. People haven’t evolved to a nobler, more altruistic level since the 90s, and people will still react the same way when confronted with the same approach to contracts and procurement.

Effective contracts have to be clear and objective. It is hard enough to specify usability requirements clearly and objectively. Stating them in any useful way at the contractual level is even harder.

Of course none of this necessarily means that external suppliers will do a bad job; far from it. What it does mean is that the external supplier relationship contains problems that must be explicitly acknowledged and addressed. Too often the implicit assumption is that using an external supplier is a low risk option, provided that they can be tied down to a tight contract. That sets up an adversarial relationship that is poison to the flexible approach that is essential if the relationship is really going to work.

No plan survives contact with the enemy. No prototype design survives exposure to the users unchanged. Why pretend that it can?

What are the implications for testers? Well, we are not mere checkers. It is our job to tell it like it us, and shine a light on those awkward facts and truths that others might prefer to ignore.

If your management is keen to follow a route that will have worrying implications, make sure they know what these are so they take the decision with the best information available. Who better than a tester to give them the news that management might not want, but really need to know?

I’ve tried to keep this fairly brief, so I’m not talking about how to make the situation better. That’s for next time (click here).