Usability and external suppliers – part 2

In my last post I talked about how the use of external suppliers can cause problems with the usability of applications. I promised to return to the topic and explain what the solutions are. I’m not sure I’d have bothered if I hadn’t stated that I would do it. The solutions are pretty basic and should be well known.

However, I’ve got to admit that there’s considerable reluctance to acknowledge the problems are real, and that the the solutions are relevant and practical. They won’t guarantee success, but since the traditional alternative load the odds heavily against success they’ve got to be an improvement.

The predictability paradox

There are two uncomfortable truths you have to acknowledge; the general problem and the specific one. In general, you can’t specify requirements accurately and completely up front, and any attempt to do so is doomed. The specific problem is that usability requirements are particularly difficult to specify.

Managers crave certainty and predictability. That’s understandable. What’s dangerous is the delusion that they exist when the truth is that we cannot be certain at the start of a development and we cannot predict the future with the level of confidence that we’d like.

It’s hard to stand up and say “we don’t know – and at this stage we couldn’t responsibly say that we do know”, even though that might be the truthful, honest and responsible thing to say.

It’s always been hugely tempting to pretend that you can set aside a certain amount of time during which you will get the requirements right in a single pass. It makes procurement, planning and management simpler.

Sadly any attempt to do so is like trying to build a house on shifting sand. The paradox is that striving for predictability pushes back the point when you truly can find out what is required and possible.

Mary Poppendieck made the point very well a few years back in an excellent article “Lean Development and the Predictability Paradox” (PDF, opens in new window).

I tried to argue much the same point in an article in 2008, “The Dangerous and Seductive V Model”. an article that has proved scarily popular on my website for the last couple of years. I guess many people still need to learn these lessons.

Unrighteous contracts?

Trying to create predictability by tying suppliers into tight contracts is a result of the delusion that predictability is available at an early stage. This can have perverse results.

Tight and tough contracts committing suppliers right at the start of a project is an understandable attempt to contain risk, or transfer it to the supplier.

Sadly, clients never truly transfer risk to the supplier if the risk is better handled internally. If the requirements are complex and poorly understood at the start then attempting to transfer the risk to the supplier is misguided.

Awarding a contract for the full development creates the danger that it will be awarded to the most optimistic, inexperienced or irresponsible supplier. Those suppliers with a shrewd understanding of what is really likely to be involved will tender cautiously and be undercut by the clueless, reckless or unprincipled.

When the wheels come off the project the client may be able to hold the supplier to the contract, but the cost will be high. Changes will be fought over, and the supplier will try to minimise the work – regardless of the impact on quality.

There seem to be three options about how to handle the contractual relationship with an external supplier if you want a suitable focus on quality, and especially usability.

The orthodox approach is to split the contract in two. The first contract would be to establish, at a high level, what is required, and ideally to deliver an early prototype. The second would be to build the full application.

This approach was recommended way back in 1987 by the US Defense Science Board Task Force On Military Software (PDF – opens in new window), and its motives were exactly the same as I’ve outlined.

The Agile approach would probably be to write contracts that have a budget for time and cost, and allow the scope to vary, possibly giving the client the right to cancel at set points, possibly after every iteration.

Martin Fowler summarises that approach well in this article, “The New Methodology”.

The third, more radical, approach was mooted by Mary Poppendieck in her article “Righteous Contracts”.

These “righteous contracts” would be written after the development, not at the start. I’m not convinced. It doesn’t seem all that different from simply dispensing with a contract and relying on trust, but I urge you to read her piece. She analyses the problems well and has some challenging ideas. I’d like to see them work, but they do seem rather idealistic.

Righteous contracts would work only if the client and supplier trust each other and are committed to a long term relationship. However, I’ve got to concede that without that trust and commitment the client is going to have problems with their suppliers regardless of the contractual style. You can’t create a productive partnership with a contract, but you can destroy it if the contract is poorly written and conceived.

Enough of the contracts! What about UX?

This piece was really prompted by my concerns about the effects of the contractual relationship between clients and suppliers, so not surprisingly I’ve been focussing on contracts.

However, I must admit I’ve found that rather dispiriting. So in order to feel a bit more positive I’m going to provide some pointers about how to try and handle usability better.

I discussed this in an article in 2009. It’s all about Agile and UX, but that’s where the interesting work is being done, and honestly, I’ve nothing positive to say about how traditional, linear techniques have handled usability. It might be possible to smuggle in techniques like Nielsen’s Discount Usability Engineering, or more ambitiously, Constantine & Lockwood’s Collaborative Usability Inspections. However, remember we’re talking about external suppliers and the contractual relationship. These techniques require iteration, and if the contract doesn’t make explicit allowance for iterative development then the odds are stacked hopelessly against you.

So Agile really seems to be the best hope for taking account of usability when dealing with an external supplier. Rather embarrassingly I came across this excellent article by Jeff Paton only a few days ago. It dates from 2008 and I really wish I’d found it earlier. It lists 12 “best practice” patterns that Paton has seen people using to incorporate UX into agile development. There’s some great stuff there.

A somewhat downbeat conclusion

Perhaps I shouldn’t have talked about “solutions” to the usability problems caused by using external suppliers. Maybe I should have used a phrase like “a better way”. I believe that flexible, staged contracts are vital. They are necessary, but they’re not sufficient. A commitment to usability is required regardless of the approach that is taken. It’s just that the traditional fixed price contract for a linear development torpedoes any hope of getting usability right. The damage is far more widespread than that, but usability is the issue that gets me particularly passionate.

I doubt if I’ve set off a light bulb in anyone’s head and they’re now thinking, “that’s what we’ve been doing wrong and I know what to do now”. However, I hope I’ve pointed some people in a useful direction to help them read, think and understand a bit more about what we do wrong.

I don’t know all the answers. Testers never do, and the big decisions that will affect usability and quality are a long way out of our remit. But we can, and should, be aware of the pitfalls of following particular approaches. We’ve a duty to speak out and tell management what can go wrong, and that just might tilt the odds back in our favour.

2 thoughts on “Usability and external suppliers – part 2

  1. Good piece, James. As usual.

    I’ve worked for companies that have used external suppliers primarily down to time constraints and project demands. In retrospect, it would have been easier to do the work in-house given the delays incurred but hindsight is a wonderful thing. The most notable thing was the impact on usability and general ‘look & feel’. New users of the product were able to tell immediately which parts of the system had been contracted out, despite efforts to have a uniform approach to this.


  2. Thanks Del. Lewis & Rieman (see part 1) summarised this problem well in their book.

    “At bottom, your objective as a commercial developer may really not be to improve the quality of somebody else’s work life, but rather to make money for yourself. So you don’t have the right goal to begin with.”

    That’s not a criticism of external suppliers. It’s just the reality. If you bring in people whose work is determined by formal contracts, the implicit threat of legal action if they don’t comply with contracts and the sure knowledge that they will lose their jobs if they lose money then the result will almost inevitably be that they focus obsessively on delivering what the contract requires, and only what it requires.

    There are obvious benefits to using external suppliers. They may have the skills and experience your own staff lack. They can supply them in bulk for a short period when you have only a temporary need. You might reduce the risks and the costs if you manage them carefully. However, going external is never an easy solution to difficult problems. The temptation is to assume that there are loads of benefits and no significant down side, and that any down side can be handled by getting lawyers to draft tough contracts.

    if you care about quality, about usability and if the look and feel of the user interface matter then external suppliers who are managed remotely via the lawyers won’t give you what you want. You have to be a bit more imaginative and flexible and use a contractual style that ensures your supplier’s needs are aligned more closely with your own. That’s where the ideas from people such as Mary Poppendieck come into play. It’s certainly a challenge! However, pretending that you can get the right results without acknowledging the problems is much worse that simply being challenging.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.