The problem is not the problem

There are two issues that have been bugging me for a while now and I mean to get them off my chest this week.

They’re not software testing topics but as with so many psychological, organisational or social problems they do have a very real relevance for testers. We don’t exist in a technical or procedural bubble that insulates us from the blizzard of conflicting and confusing influences that swirl around every organisation.

The first issue is that of problems. Very often the problem is not the problem. The problem we see is not necessarily the real, possibly very damaging, problem.

As testers this shouldn’t be news. We’re used to root cause analysis, delving behind the curtain to find out why things have gone wrong and not being deceived by what we initially see. We should also be familiar with the “five whys”, or some variant on that; the discipline of repeatedly challenging answers till we burrow down to the heart of the problem.

That’s not what I want to talk about here. Things go wrong, in life, at work, and especially in software development. That’s the way of it.

What really matters, certainly in the long run, is often not the problem itself but the way we deal with it.

We can deal with the problem in a brisk and efficient manner, resolving the immediate concerns, learning the right lessons and moving on. Or we can blunder into any one of several traps that could hurt us far more than the original problem did.

The first is the cover up. Watergate is the great historical example. The original crime might have been handled without damaging Richard Nixon, but it was the cover up that ended his presidency.

When we, as software developers, hit problems the worst thing we can do is attempt to cover up. Honesty with customers and users isn’t simply honourable and ethical, it also buys you trust and support.

Own up on the smaller problems and it may just buy you the credibility and goodwill to save your skin when you hit big, career threatening problems.

Denial is the sibling of the cover up. It might not be a vigorous, outspoken denial. It may just be a matter of kidding ourselves that there’s nothing really wrong, that we can carry on as before once we’ve sorted the problem. It was just bad luck, or a bit of carelessness. No big deal. Just move on folks. Nothing to learn here.

The next trap is the one you plunge into if you head off in the opposite direction; over-reaction. Losing your temper and savaging staff is obviously damaging and isn’t tolerated in healthy organisations any more, but a more insidious variant is still too prevalent. That is the tendency to blame individuals.

Someone must be to blame, somebody must have screwed up and be held to account. I once worked somewhere with an appalling blame culture. Problems were something to run away from, not something to fix. No-one wanted to jump in and proactively resolve problems in case they got implicated in the management investigation and fallout.

A more respectable form of over-reaction is the assumption that the problem justifies sweeping changes or drastic action rather than careful tweaking. Instead of a careful analysis of what actually went wrong and what the cause was, there is the presumption that it was confirmation of a trend, or deeper problem, that was previously suspected.

That leads me on to the fourth trap, confirmation bias. We convince ourselves the world should work a certain way. Any and every problem can be interpreted as supporting evidence.

The glaring example in the history of software development was the conviction that software development should be essentially the same as construction engineering. Early experts saw practitioners struggling to manage projects with a more or less structured and orderly linear process. They concluded that the answer to the problem was to try and make the process even more rigidly structured, more ordered and more linear. We got Structured Methods (shudder!).

After all, nobody builds a bridge incrementally, opening it bit by bit. No-one erects a building in several iterations. So software engineering shouldn’t be any different. That was more or less the line of thought. Life might be simpler if software development were a highly predictable and controllable process, but the truth is that development is not like that.

I referred to this problem on my blog earlier this year when I explained the predictability paradox.

The correct response to the problem would have been to go with the grain of the problem, embracing the uncertainty, building iteration and incremental development into the development process rather than pretending that we understand the business problem sufficiently well to proceed down a rigid, linear route.

The problem was assumed to be that software engineering was not sufficiently like construction engineering. Divergence between the two activities was seen not as evidence of a genuine difference, but as proof that the software developers were out of line. The truth was that software development is a quite separate discipline and the real problem was that developers were forced to contort their practices to fit a wildly inappropriate paradigm.

The real problem wasn’t the problem the profession saw. It was their response to the problem.

None of this is new to marketing people. Apparently BMW owners who have a problem with their car are more likely to buy another BMW than those who have no problems. Why? It’s because of the way that BMW deal with problems. Is it true? Well, I don’t have access to the statistics, but that’s BMW’s reputation. Problems have become an asset, rather than remaining a problem.

What does this mean for any of us? It’s not easy to draw a neat solution to this problem of problems. All I can pass on is something I have found useful.

Often we plunge into one of the traps because we choose to respond in a way that suits our instincts or prejudices, or which is just the easy, comfortable response.

Take a breath before committing yourself and ask yourself whether your response is what ought to happen. Or is it the response that you’d really like to be the right one? Is it the easy one? Are you maybe dodging the real issue, going for short term expediency rather than long term results?

Run through the traps I’ve discussed. Do any of these make you wonder about your response?

Is your reaction to the problem going to be seen as the real problem further down the line?

After I wrote this I checked the phrase “the problem is not the problem” to see if anyone had used it before. Initially I was a bit disappointed to see that Tom Peters has, but then I cheered up when I realised he’s a pretty respectable ally! Watch the video in the link or, if you’re in the office, check out the transcript (PDF, opens in a new window). It’s very much in line with my argument, but he puts a bit of extra spin on it.

The other thing I want to get off my chest this week is about making decisions. I’ll be back in a day or so.

Please add any comments. I do like to see what people think.

2 thoughts on “The problem is not the problem

    • Well, if it’s a choice between being right and being original it’s better to be right I suppose! 😉
      Thanks for reminding me about Virginia Satir, someone with whose work I really should be more familiar.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.